​
ADR-0050: Digital Worker local embeddings for semantic history search

​
Status

Accepted (2026-07-22 — spec review completed; see docs/superpowers/specs/2026-07-22-dw-semantic-history-search-design.md)

​
Tags

digital-worker, memory, learning, ai, search, embeddings, pgvector, cost, multilingual

​
Decision

Digital Worker gains semantic search over chat history using local embeddings (bge-m3 via the same Ollama host already running local triage), stored in Postgres/pgvector. This is an addition to, not a replacement of, the existing full-text search (ADR-0034’s SearchAgentHistory over dw_chat_events). The decision has two parts:
  1. DW adopts embeddings, deliberately reversing the no-vector posture of ADR-0028 and ADR-0034 — but only under strict conditions that keep those ADRs’ principles intact:
    • Local and unbilled. Embedding runs on the local model; it MUST NOT route through AccessGate. This preserves ADR-0028’s billed-AI cost discipline — the objection there was per-message API cost, which local compute removes.
    • Search-only, off the hot path. Embeddings power history retrieval, not the chat reply path and not memory injection. Memory injection stays recency/usage-ranked exactly as ADR-0034 specifies; this ADR does not touch it.
    • Asynchronous. The dw_chat_events row is stored synchronously; a periodic watermark sweep embeds closed conversation windows, never blocking ingest. An empty watermark provides historical backfill without a separate job.
    • Substantive windows only. Substance is evaluated once per completed conversation window; every message in a qualifying window is retained.
  2. Embedding-storage convention (applies repo-wide, satisfied here by DW):
    • Each domain owns its own embedding table (dw_message_embeddings), sized to its model. Do not reuse another module’s embedding table.
    • Store the model identity as a column and always filter queries by it. Two models never share a vector space — even at equal dimensions — so a cross-model comparison is meaningless.
    • Embeddings are a regenerable cache, not source of truth. message_text is the truth; a model swap is a re-embed backfill job, not a migration. Regenerable holds only within the event retention window: the retention purge deletes embedding rows at the same cutoff as their source dw_chat_events rows (the embedding table stores chunk text, so outliving the purge would violate the org’s retention promise). A model swap re-embeds live rows only.
    • Storage is halfvec(1024) + halfvec_cosine_ops.

​
Why

DW history search today uses plainto_tsquery('simple', ...). The simple FTS config does no Chinese word segmentation and no stemming. For DW’s real data — ~80% English but ~40% Malay, 10–20% Chinese, and heavily code-switched within single messages — keyword FTS is already broken for the non-English portions. Malay and Chinese content is effectively unsearchable. Semantic embeddings are the only approach that works for zh/ms/rojak recall, and they enable cross-lingual retrieval (a query in one language finds messages in another). The reason to record this as an ADR is precisely that it looks like it contradicts ADR-0034, which told a future reader not to bolt AI onto the FTS-based history search. It does not contradict it: 0034’s rejection was of lossy LLM compaction on the write path and billed AI where FTS sufficed. Here the AI is local (unbilled), additive (FTS remains), search-only, and async — none of 0034’s or 0028’s actual concerns apply. Memory budgeting and injection are untouched. The storage convention is recorded because the same rule is already violated elsewhere: AIStorageAdapter.Search (ai_repository.go:157) runs vector search without a model filter, which silently breaks the moment that table holds two models. Writing the rule down makes that a known, fixable bug rather than a latent trap each new embedding consumer re-discovers. Rejected alternatives:
  • Keep FTS only. Broken for Chinese/Malay/rojak — the whole motivation.
  • Paid embeddings (OpenAI/Voyage). Per-token cost that grows with history and sends all internal chat to a third party; local ties or beats them on our languages. Rejected on cost + privacy.
  • LLM-normalize (translate) → embed. A full generative call per message, non-deterministic, and lossy exactly on code-switched input. Rejected; bge-m3 handles rojak natively in one deterministic step.
  • Reuse the ai module’s embeddings table (vector(1536)). Different model and dimension; mixing models in one column is meaningless. Rejected — own table.
  • Treat every message as its own embedding / embed inline on ingest. Per-message embeddings fragment conversational meaning; inline embedding blocks ingest and re-introduces a hot-path cost. Rejected — substantive conversation windows, embedded asynchronously by the sweep.
  • MRL / dimension truncation. Solves storage/speed at millions of vectors; the DW corpus is thousands. Rejected as premature.

​
Consequences

  • New dw_message_embeddings table and a single periodic watermark sweep (per org/provider/channel/model) that embeds closed conversation windows — the sweep alone provides async embedding, retry (watermark stalls on transient failure), historical backfill (empty watermark), and model-swap rebuild (new model = empty watermark). Poison windows are recorded and skipped so they cannot stall the watermark forever.
  • Because embeddings are regenerable, the embedding model is a config value (default bge-m3), swappable later via a re-embed backfill under a new model.
  • Every embedding query MUST filter by (org_id, model); the ai_repository.go missing-model-filter bug must be fixed to uphold the rule (separate small change, different module).
  • A local reranker (bge-reranker-v2-m3) is a deferred phase-2 addition, run as a separate inference service — not part of this decision.
  • The whole feature is behind a master feature flag (ADR-0016); off → the sweep is a no-op and search remains FTS-only. Ingest is unchanged in either state.
  • ADR-0034 remains fully in force for memory: injection stays recency-ranked; this ADR governs history search only.

​
Rules for agents

  • Scope every embedding query by (org_id, model). Never compare vectors across models, even at equal dimensions.
  • Keep history embedding local and unbilled — never route it through AccessGate. It is not a billed AI call.
  • Treat embeddings as a regenerable cache; dw_chat_events.message_text is the source of truth. A model change is a re-embed job over live rows.
  • Embedding rows die with their source: the retention purge deletes dw_message_embeddings at the same cutoff as dw_chat_events. Never let embedded chunk text outlive the org’s event retention.
  • Embed only substantive completed conversation windows. Retain every message inside a qualifying window, and embed asynchronously via the watermark sweep — never on the ingest hot path.
  • Do not add semantic ranking to memory injection here — that stays ADR-0034 recency/usage-ranked until the over-budget signal justifies it.
  • When adding any new embedding consumer, give it its own model-scoped table and filter by model; do not reuse another domain’s embedding table.