Reflex: retrieval built for recurring documents

Most retrieval systems treat every request as new work. A document that was searched a minute ago can be embedded, transferred and scored again as if the system had never seen it. For RAG applications built over policies, product catalogs, support libraries, source code or tenant knowledge bases, that is the wrong economic model: queries change constantly, but the evidence often does not.

Reflex is built around that asymmetry. It turns recurring document representations into a reusable asset, combines semantic similarity with deterministic lexical evidence and returns a calibrated order without adding a pairwise language-model pass to every query. The result is a retrieval layer whose latency and unit cost improve as the same evidence continues to serve new questions.

Retrieval should remember the documents, not just answer the query

Dense embeddings are effective because they place related ideas close together even when their wording differs. Production search, however, also depends on literal evidence: a product code, a statute, an acronym, a person's name, an exact phrase or two terms appearing close to each other.

Reflex brings those signals together in one path. It first establishes semantic relevance, then calibrates the surviving candidates with candidate-local lexical evidence. Its lexical stage combines BM25-like term strength, query coverage, compact co-occurrence and constrained fuzzy matching. Because the calibration is deterministic, it requires no additional model inference once the embeddings are available.

That distinction matters operationally. A dedicated cross-encoder evaluates query-document pairs jointly. Reflex reuses independent document representations and spends its online work on the changing query and a bounded candidate set. Repeated documents stop behaving like repeated model jobs.

One retrieval layer, two integration paths

Reflex supports two ways of fitting into an application:

1. Ad hoc reranking. Send a query and an existing list of document strings. Reflex returns their original indices in calibrated order, so an application can add semantic and lexical reranking without operating a permanent collection.
2. Indexed RAG. Embed documents during indexing, retain their vectors with the collection and encode only the query online. Reflex can then participate as the final ranking stage before context is assembled for the model.

The evaluated V1 configuration uses EmbeddingGemma 300M with normalized 768-dimensional vectors. The boundary itself is encoder-independent: another compatible embedding model can supply the vectors without changing the lexical engine or the retrieval contract.

For requests with up to 100 documents, Reflex computes exact cosine similarity across the complete submitted set. Larger requests use compact sign sketches to narrow the search before exact cosine scoring. On eligible small sets, lexical evidence may recover up to three threshold-passing candidates that would otherwise sit below the semantic shortlist.

The API keeps the original cosine similarity as the public relevance score. Calibration changes the returned order, not the meaning of the score. That preserves threshold interpretability while still allowing literal evidence to resolve close semantic candidates.

Competitive ranking without another pairwise model pass

We evaluated Reflex against Qwen3-Reranker-0.6B, Llama Nemotron Rerank VL 1B V2 and Cohere Rerank 4 Fast over the same prepared candidates. The suite combines TREC DL 2019, TREC DL 2020, NFCorpus, SciDocs, FiQA-2018 and TREC-COVID: 301 queries and 29,870 query-document pairs per system, with a common top-10 cutoff.

Reflex reached aggregate nDCG@10 of 0.478, slightly above Qwen at 0.474. Llama Nemotron and Cohere reached 0.502 and 0.500. In other words, Reflex landed within 0.024 nDCG of the strongest observed point estimate while avoiding an additional pairwise inference stage.

Aggregate nDCG@10

301 queries · point estimates and marginal query-bootstrap intervals

00.250.500.751

Reflex 0.478

Qwen 0.474

Nemotron 0.502

Cohere 0.500

The quality result establishes the operating range: Reflex preserves competitive top-10 ordering without making a second model the mandatory final step of every retrieval request. The benchmark uses prepared candidate pools, so it isolates reranking quality; like any reranker, Reflex can only order evidence that reaches its input.

Reuse turns into measurable latency

Cache awareness is not a secondary optimization in Reflex. It is part of the retrieval architecture. We measured three document-reuse states with 100 requests per phase, 60 medium documents per request, top 10 and concurrency four.

The all-document-miss path averaged 1.593 seconds. A new query over cached documents averaged 496 milliseconds, a 69% reduction and a 3.21x acceleration. Exact replay averaged 209 milliseconds, showing a 7.62x acceleration when both content and request shape repeat.

End-to-end mean latency

100 requests per phase · 60 documents · concurrency 4

All documents miss 1,593 ms

Documents cached 496 ms

Exact replay 209 ms

The middle path is the important one for recurring-corpus RAG: the question is new, but the documents are already known. Policies, manuals, catalogs and support content naturally create this traffic shape. Reflex converts that repetition into less online work rather than charging the full cold path indefinitely.

The cost curve improves as the corpus repeats

To compare billing models on a common task, we normalized 1,500 query-document pairs to 750,000 assumed billable units for token-metered systems. The Cohere row treats the same task as 15 searches of 100 single-chunk documents. At the reference rates used in the calculation, a Reflex document miss costs $0.01125 for the task and a document hit costs $0.00225.

The warm Reflex path is 30% of the $0.00750 Qwen and Llama Nemotron reference rows, and 7.5% of the $0.03000 Cohere row. Even the fully cold Reflex path remains below the Cohere reference task.

Estimated reference-task cost

1,500 pairs · normalized billing scenario · US dollars

Reflex · miss $0.01125

Reflex · hit $0.00225

Qwen $0.00750

Nemotron $0.00750

Cohere $0.03000

The cold and warm rates also reveal the break-even point. Under this normalized model, Reflex moves below the two $0.00750 token-metered reference rows when roughly 42% of document units are served from cache. The more stable the corpus, the more the economics shift toward the warm path.

The architecture changes more than one benchmark number

Reflex is designed as an operating layer for retrieval, not as a single opaque score endpoint.

Together, these choices make Reflex especially natural for tenant-scoped search, support collections, knowledge bases, policy libraries, application catalogs and any RAG system where a changing stream of questions repeatedly reaches a comparatively stable body of evidence.

What Reflex is - and what it is not

Reflex occupies a specific layer in the retrieval stack. Its boundaries are deliberate:

• It is not an embedding model. It orchestrates a compatible encoder, so the representation layer can evolve independently from the lexical calibration logic.
• It is not corpus-wide BM25. Its lexical statistics are computed inside the current semantic candidates, keeping the final stage bounded and focused.
• It is not a pairwise cross-encoder. It reaches its operating point without jointly running a language model over every query-document pair.
• It is not an exact-response cache. Reused document embeddings still pass through semantic selection and lexical calibration, so a new query receives a newly ranked result.
• It is not a replacement for first-stage recall. No final reranker can promote evidence that never reaches its candidate pool.

Those constraints define where Reflex compounds. Entirely cold, one-off batches cannot use document reuse. Repeated-corpus retrieval can.

Open source, inspectable and coming soon

Reflex is being released as an open-source project under the MIT License. The public repository is being prepared and will be available soon.

The release will include the source, a frozen V1 configuration, evaluation scripts and sanitized benchmark artifacts. The goal is to make the retrieval path inspectable: how candidates are selected, how lexical evidence changes order, how cache accounting works and how the reported measurements can be reproduced.

That matters for infrastructure expected to sit between private documents and generated answers. Reflex is designed to be understood, measured and adapted - not treated as an unexplained ranking service.

A retrieval layer whose economics improve with use

Reflex starts from a practical observation: in production RAG, documents repeat more often than questions. Its architecture converts that repetition into reusable embeddings, lower warm-path latency and a cost curve that improves with cache utilization.

The first evaluation places its top-10 ranking quality in the same competitive range as dedicated rerankers: slightly above Qwen on aggregate nDCG@10 and within 0.024 of the strongest observed point estimate. The production experiment shows a 3.21x acceleration for new queries over cached documents. The normalized cost model puts the warm task at 30% of the token-metered comparator references.

That combination is the Reflex proposition: strong retrieval order, literal evidence where semantic similarity needs help, and an operating model that rewards the document stability already present in real knowledge systems.