Retrieval and context

A system that answers badly is usually not thinking badly, it is reading the wrong thing. Retrieval is a data engineering discipline with a search problem inside it, and it responds to the same treatment as any other data problem: look at what came back, label it, measure it, change one thing, measure again. We spend the early weeks of most engagements here, because a better prompt over the wrong context is a rounding error.

What we actually do

• Chunking chosen against your document structure rather than a default token count, and evaluated by whether a retrieved chunk actually contains the answer
• Metadata design so filters do the coarse work before the vector search does the fine work: source, effective date, jurisdiction, owner, permission scope
• Permission propagation, so the index carries the same access rules as the source and a retrieval cannot surface a document the asker could not open
• Hybrid retrieval where keyword and vector search disagree, with the merge strategy chosen by measurement rather than by fashion
• Version awareness for policy and reference corpora, so an answer cites the version that applied on the date in question rather than the current one
• Re-ranking and context assembly under an explicit token budget, with what got dropped visible in the trace
• A labelled retrieval set separate from the end to end evaluation set, because a system can answer correctly on retrieved rubbish and hide the fault

What we instrument

Measurements, not results. These are the readouts we put in place so that you end up with numbers about your own system. There are no values on this page because a value here would be somebody else’s.

• Recall at k against a labelled question and passage set you own
• Proportion of answers where every cited passage is actually relevant
• Retrieval latency at the median and the tail, separated from generation latency
• Index freshness: age of the oldest document that should have been re-indexed
• Filter selectivity, so you can see when metadata has stopped narrowing anything

Where this stops working

• Retrieval cannot invent an answer that is not in the corpus. If the knowledge lives only in people's heads, the first piece of work is writing it down, and we will say so rather than build an index over a gap
• Corpora that change hourly need a re-indexing pipeline with its own monitoring, which is a build, not a setting
• Scanned documents without a text layer need an extraction step first, and extraction quality then caps everything downstream

What we work with

Named as plain text. None of these is a partnership, an endorsement or a default: the right one is chosen per engagement, usually the one your team already runs.

• Vector stores including Pinecone and pgvector
• PostgreSQL and its full text search
• OpenSearch and Elasticsearch
• Document extraction and OCR pipelines
• Airflow and scheduled ingestion