Latency & voice safety

For a voice agent, a blocked turn is a dropped beat in the conversation. pgmem is designed so the response path does only one thing — assemble the prompt — and everything expensive runs after the reply. This page is the production checklist for holding that line.

The one rule

Only compile_context runs on the response path. It's pure indexed reads under a token budget — no LLM call. Everything else (add_episode, compaction, projection refresh, and narrative rebuilding does real LLM/embedding work and must run off-turn.

on the turn:   add_turn(user) → [fetch live ∥ compile_context] → your model → add_turn(assistant)
after reply:   compact_async() · observe(...) · session.end() → projection workers

Checklist

• Compact off-turn. Use compact_async after replying; never await compress/compact inline. See Compaction off the critical path.
• Fetch live state concurrently. Read systems of record in parallel with your turn work and pass the result via the per-call injected_context= — not a construction-time provider, which runs on the turn.
• Refresh projections off-turn. session.end() only enqueues; the projection worker refreshes derived claim views in the background.
• Size the pool. pool.max_size >= 2 so off-turn work (compaction, worker) doesn't contend with the turn's own connection. Size up for concurrent groups.
• Mind the tokenizer. compile_context's budget counts message content only; reserve headroom for provider framing so you don't overflow the real window.

Why it's safe to restart

Because the response path touches only Postgres reads and your model call, an agent worker holds no irreplaceable in-memory state. A crashed or rescaled worker calls session.resume(session_id, group_id) and continues — the execution tree and recent turns come straight back from the database.

Gotchas

Related

• How a turn flows — the on-turn vs off-turn model
• LiveKit / voice agents — the stateless-worker pattern
• Compaction off the critical path
• Use live system-of-record state