Rerank RAG Results Without a Cross-Encoder: When to Use Reflex

To rerank RAG search results without a cross-encoder, Reflex uses semantic similarity and bounded lexical evidence rather than jointly encoding every query-document pair. It can reuse document representations within an AIVAX account, so a document that appears in many queries does not need to be embedded again each time. Whether this approach is faster, cheaper, or accurate enough for your application requires a workload-specific comparison.

The useful distinction is between repeated evidence and repeated answers. A support manual can serve many different questions without its text changing. Reflex can reuse the representation of that text while computing a new ranking for each question; it does not cache a finished answer.

How does Reflex rank candidates without a cross-encoder?

Reflex embeds the query and candidate documents separately, uses semantic similarity to select candidates, and can refine their order with lexical signals. Those signals include BM25-style term matching, term coverage, and proximity. They operate on the candidates being ranked, not as a corpus-wide keyword search. The lexical contribution is conditional; some query shapes use semantic scores alone.

This avoids a joint query-document model pass after embeddings are available. It does not eliminate embedding work on uncached inputs, request overhead, or the need to retrieve useful candidates first.

The response preserves each document's original input index. Its relevance_score remains the semantic similarity score, while the returned order can reflect lexical adjustments. Preserve the response order rather than sorting it again by score, and do not treat the score as a probability that an answer is correct.

For the boundary between finding candidates and ordering them, see Do I need a reranker for RAG?. The Reflex documentation explains the collection-less workflow.

When do recurring documents make reuse useful?

Representation reuse is relevant when new queries repeatedly reach unchanged policies, manuals, catalogs, or support articles. The cache is scoped to the account and distinguishes the embedding task, model, and exact input text. The same document can therefore be reused across different questions, but altered text or another account does not share that entry.

Reuse is conditional on the representation still being cached. A cold request or a one-off batch may need every representation computed again. Cache reuse also does not remove the need to send candidate strings to the standalone API: it saves embedding work inside the service, not request payload size.

Inspect usage.input_tokens and usage.cached_input_tokens to distinguish newly processed input from cache hits. Compare cold and repeated-document requests separately instead of averaging them into a single latency or cost claim.

If AIVAX should store and retrieve a persistent corpus, use Semantic Search. Collection retrieval and Reflex reranking are separate stages: stored collection vectors do not mean that all Reflex candidate representations will already be cached.

When should you choose Reflex instead of another reranker?

Consider Reflex when you already have useful candidates, the evidence repeats, and you want to evaluate semantic-plus-lexical ranking without a pairwise model pass. Product identifiers, names, and domain terminology are useful cases to include in that evaluation, not a guarantee of better relevance.

Compare it with an available Cohere, Jina, or Qwen reranker when your queries require distinctions that similarity and term evidence may miss, such as negation, exceptions, or several interacting conditions. Do not assume that removing the pairwise pass preserves a cross-encoder's ranking quality. This article makes no measured quality, latency, or cost claim against those models; use the current reranker selection guide and test the options on your own documents.

Keep a hybrid BM25/vector retrieval baseline if you already have one. Rank fusion combines retrieval signals; Reflex evaluates the supplied text candidates. Neither approach fixes missing source material. In AIVAX, rrf is available after collection retrieval, not through the standalone reranking endpoint.

If the expected evidence never reaches the candidate set, address retrieval or document preparation first. The RAG pipeline checklist helps distinguish those failures, and the PDF ingestion quality checklist covers source-text problems that ranking cannot repair.

What determines cost and capacity?

As of this update, Reflex's base metered rates are $0.015 per million uncached input tokens and $0.003 per million cached input tokens. Query and document input both contribute to usage. Actual charges depend on subscription coverage and applicable discounts; cached input is not inherently free. See RAG and Collections pricing for current rates.

The standalone Reflex API accepts up to 10,000 candidate documents and returns at most 200 results. Treat these as request limits, not a recommended batch size or a quality guarantee. Narrow large candidate pools and keep each document focused enough to preserve the evidence needed for the query.

A smaller top_n limits the output; it does not remove submitted documents from the processing and billing path. Reranking request quotas are separate from token-based charges. Consult Plans and Limits before sizing production traffic.

How should you evaluate the trade-off?

Use a fixed set of representative queries and label which candidate documents contain the required evidence. Then compare no reranking, Reflex, and the alternative ranker you are considering:

1. Hold the candidate pool constant. Otherwise, a retrieval change can look like a ranking improvement.
2. Measure useful evidence near the top. Record whether the required document reaches the context window and whether distractors displace it.
3. Separate cold from reused inputs. Include changed documents as well as repeated text; record latency and the returned usage for each case.
4. Test the answer, not just the score. Check whether the downstream response uses the evidence correctly, especially for exceptions and conflicting passages.

Choose the ranker that meets your quality requirement within your measured latency and cost budget. Document reuse is an operational advantage to test, not a substitute for relevance evaluation.

What should you know before enabling Reflex?

Can Reflex replace a cross-encoder?

It can occupy the reranking stage when your evaluation shows that its ordering is sufficient. It uses a different scoring mechanism, so there is no general guarantee of equivalent quality or lower end-to-end cost and latency.

When should you avoid Reflex?

Do not use it to compensate for missing candidates or broken source text. One-off inputs offer little document-cache reuse, and tasks that depend on subtle query-document relationships need comparison with a pairwise ranker before switching.

How do you enable or disable it for collection search?

For a semantic collection query, omitting reranker selects Reflex. Set "reranker": "@aivax/reflex-v1" to select it explicitly, or "reranker": "none" to disable reranking. The smart alias also resolves to Reflex. Filter-only queries do not run semantic retrieval or reranking.

How do you use Reflex without a collection?

Authenticate with an AIVAX API key using the Authorization: Bearer <API_KEY> header, set Content-Type: application/json, and send this body to POST https://inference.aivax.net/api/v1/generations/rerank:

{
  "model": "@aivax/reflex-v1",
  "query": "How can I rotate an API key without interrupting service?",
  "documents": [
    "Create a replacement key, update the application, verify requests, then revoke the old key.",
    "Billing invoices are available in account settings."
  ],
  "top_n": 1,
  "min_score": 0
}

The document strings are illustrative inputs, not an AIVAX key-management procedure. Omitting model also selects Reflex. Join returned results[].index values to your input array; no collection is required. The standalone endpoint rejects none and rrf because neither is an autonomous reranker.

How does minScore interact with reranking?

Collection search applies minScore to semantic candidates before reranking. Reflex cannot restore a candidate removed by that threshold. The standalone endpoint uses a different field, min_score, to filter returned relevance scores after ranking and before the final top_n cut. For Reflex, those public scores are semantic scores, not the internal lexical-adjusted ordering scores. Tune thresholds on your own queries and retest them when changing rankers.