Context Engineering & Memory

Context is the model's workspace for one decision, not a document warehouse. Memory is not permanent chat storage; it is a controlled process that extracts reusable facts and retrieves them for an appropriate future task. Reliability depends on what enters the current inference, not how much text sits in a database.

---

Separate four kinds of state

The OpenAI Agents SDK makes the same distinction between local context and model-visible context. A tool can access application dependencies without sending a database connection or secret to the model.

business data and dependencies → local context → tool
                                  │
                                  └─ only task results enter LLM context

past events → memory write gate → memory store
                                      │
current task → retrieval + policy ────┘ → working set → LLM

---

The goal is not a full context window

Anthropic describes context engineering as continuously selecting the tokens most likely to support the desired behaviour. The practical target is a smallest sufficient set.

Organise one model call in layers:

1. Stable instructions: role, safety boundaries, and output contract.
2. Current objective: user outcome and acceptance criteria.
3. Working state: completed steps, open questions, and remaining budget.
4. On-demand knowledge: retrieved passages or memories with sources.
5. Tool definitions: only the tools available for this decision.
6. Recent observations: results that still affect the next action.

Do not hard-code a rule that context must consume a particular percentage of the window. Models, tasks, and output needs vary. Reserve output and tool-result capacity first, then set thresholds through evals.

---

Allocate a context budget

type ContextBudget = {
	modelLimit: number;
	reservedOutput: number;
	stableInstructions: number;
	taskState: number;
	retrievedKnowledge: number;
	toolDefinitions: number;
	recentObservations: number;
};

function assertBudget(b: ContextBudget) {
	const inputBudget = b.modelLimit - b.reservedOutput;
	const planned =
		b.stableInstructions +
		b.taskState +
		b.retrievedKnowledge +
		b.toolDefinitions +
		b.recentObservations;
	if (planned > inputBudget) throw new Error('CONTEXT_BUDGET_EXCEEDED');
}

When over budget, remove duplication, truncate low-value tool output, compact old observations, reduce retrieval passages, and defer unused tools. Stable safety instructions should not be the first content removed.

---

What belongs in memory

“The user said it” does not mean “remember it forever.” Before writing, ask whether the information is explicit, likely to change, useful later, permitted to store, and subject to expiry.

---

Put a gate in front of memory writes

type MemoryCandidate = {
	tenantId: string;
	userId: string;
	kind: 'fact' | 'preference' | 'event' | 'procedure';
	value: string;
	sourceRef: string;
	observedAt: string;
	confidence: number;
	expiresAt?: string;
	sensitivity: 'public' | 'internal' | 'personal' | 'restricted';
};

function mayPersist(candidate: MemoryCandidate) {
	return (
		candidate.confidence >= 0.9 &&
		candidate.sensitivity !== 'restricted' &&
		Boolean(candidate.tenantId && candidate.userId && candidate.sourceRef)
	);
}

A real system also applies consent, regional retention, deletion requests, and field-level encryption. The model may propose a candidate; deterministic policy decides whether to persist it.

---

Retrieval is more than vector similarity

current task
  → hard tenant/user/region filter
  → kind, time, and permission filter
  → semantic or keyword recall
  → deduplication and conflict detection
  → relevance + recency + confidence reranking
  → token-budget truncation
  → inject with sourceRef

const memories = await memoryStore.search({
	tenantId: run.tenantId,
	userId: run.userId,
	query: run.objective,
	kinds: ['fact', 'preference'],
	notExpiredAt: new Date().toISOString(),
	limit: 12
});

const workingSet = rerank(memories)
	.filter(item => item.score >= retrievalThreshold)
	.slice(0, maxMemoryItems)
	.map(({ value, sourceRef, observedAt }) => ({ value, sourceRef, observedAt }));

Tenant and user constraints belong in the database query. Never retrieve across tenants and ask the model to decide what it may reveal.

---

Conflicts, expiry, and correction

Memory is not immutable truth. If a user first lives in Brisbane and later explicitly moves to Sydney, preserve provenance and mark the older fact as superseded.

{
	"memoryId": "mem_204",
	"key": "home_city",
	"value": "Sydney",
	"observedAt": "2026-08-25T11:10:00+10:00",
	"sourceRef": "message_991",
	"status": "active",
	"supersedes": "mem_102"
}

When a conflict cannot be resolved deterministically, ask the user. For volatile facts such as prices, laws, job roles, and exam rules, prefer an authoritative current source over an old memory.

---

Preserve recoverable state during compaction

## Objective

Deploy checkout-api after regression tests pass.

## Confirmed facts

-   Current branch: release/checkout-42 [source: git]
-   Payment E2E is failing at 3DS callback [source: run_87]

## Decisions

-   Do not bypass the 3DS test.
-   Use provider sandbox account only.

## Completed

-   Unit tests passed at commit 1a2b3c4.

## Open work

-   Fix callback signature verification.

## External actions

-   No deployment has occurred.

Remove small talk, duplicate logs, and observations that no longer affect decisions. Preserve the objective, hard constraints, evidence, external side effects, and next safe action.

---

Prevent persistent prompt injection

Web pages, email, and tool results are untrusted data. If attack text is summarised into long-term memory, it can influence unrelated future tasks.

The model must not be the sole authority deciding that arbitrary text is safe to remember permanently.

---

Evaluate context and memory behaviour

Include same-name users, conflicting preferences, expired facts, malicious documents, empty retrieval, and oversized tool output. “Remember that I like coffee” is not a sufficient production eval.

---

Common mistakes

Practice: a cross-session project assistant

1. Define schemas for facts, preferences, and events.
2. Reject passwords, one-time codes, and unconfirmed inferences at the write gate.
3. Filter by tenant, user, expiry, and sourceRef.
4. Simulate a city change and verify the new fact supersedes the old one.
5. Add malicious instructions to a retrieved document and confirm they never enter long-term memory.
6. Record answer quality, input tokens, retrieval precision, and cross-user leakage.

Definition of done

• I can distinguish local state, LLM context, working memory, and long-term memory.
• Code and policy control writes; the model does not decide alone.
• Retrieval hard-filters tenant and user before semantic search.
• Every long-term fact has source, time, status, and deletion path.
• Compaction preserves objectives, decisions, evidence, and side effects.

Related reading

• Context Fundamentals
• Context Compression
• Claude Code Context Management
• Advanced Agent Systems

Official references

• https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
• https://openai.github.io/openai-agents-js/guides/context/
• https://openai.github.io/openai-agents-python/running_agents/