pgmem
Guides

Tune retrieval

Strategies, depths, and the token budget.

compile_context decides what the model sees. Three levers shape it: the strategy (how it weights the memory tiers), the retrieval depths (how many items per tier), and the token budget (the hard ceiling). All three are inspectable after the fact.

Prerequisites

Steps

Pick a strategy per turn

strategy re-weights the memory tiers for the kind of turn you're handling:

context = await session.compile_context(query=user_text, strategy="recall")
  • conversation — balanced (default).
  • task — favours session-scoped episodic recall to keep doing the work.
  • recall — favours cross-session narrative for "what happened before."

Set a token budget and inspect what was dropped

context = await session.compile_context(query=user_text, token_budget=12_000)
print(context.token_count, context.dropped)   # e.g. 11_840 ['T4']

Under pressure the compiler sheds lowest-priority first: T4 → T3 → T5 → T6 → R. dropped tells you exactly which tiers were cut.

Override depths — globally or per call

Set depths once on CompilerConfig.strategy_depths (every built-in strategy must be present), or override for a single compile with retrieval_depths:

from pgmem import PgMem, PgMemConfig, CompilerConfig, RetrievalDepths

mem = await PgMem.create(dsn, embedder=embedder, llm=llm, config=PgMemConfig(
    compiler=CompilerConfig(strategy_depths={
        "conversation": RetrievalDepths(episodic=5, narrative=3, semantic=5),
        "task":         RetrievalDepths(episodic=8, narrative=0, semantic=3),
        "recall":       RetrievalDepths(episodic=3, narrative=8, semantic=8),
    }),
))

# one-off override:
context = await session.compile_context(
    query=user_text,
    retrieval_depths=RetrievalDepths(episodic=10, narrative=0, semantic=2),
)

0 disables a tier. Depths are retrieval limits, not token reservations — the budget still trims what's recalled.

Verify

Inspect the provenance to see what each lever did:

print(len(context.recalled_facts))     # T3 episodic
print(len(context.recalled_sagas))     # T5 narrative
print(len(context.recalled_semantic))  # T4 graph
print(context.injected)                # T6 live

Gotchas

The token budget counts message content only — not your provider's chat framing or tool-call overhead. Leave headroom in token_budget, or you'll overflow the real context window.

  • The semantic tier (T4) is only retrieved when an embedder is wired; without one it's silently skipped regardless of depth.
  • CompilerConfig.strategy_depths must define every built-in strategy — a partial map raises at construction (so an upgrade can't silently change a depth).

On this page