A narrow beam of cold light passes through a facade of silver and frosted-glass louvers and falls on three pale-blue glass spheres on an icy floor
All posts

RAG metadata filtering: pre-filter by tags, metadata and date

Filter RAG results by metadata, tags and date before ranking, or run a filter-only browse. Syntax, why filters return nothing, and a working AIVAX request.

To filter RAG results by metadata, store tags and metadata with each document, then send a filter with the query so the search ranks only documents that match. On AIVAX that is the filter field of POST /api/v2/query: a string such as tags has "billing" and metadata.region = "latam" and createdAt >= now-30d, or an array of strings, one condition each. The filter runs before the search terms are embedded, so documents outside it are never ranked. You can also send a filter with no search terms to POST /api/v2/query/browse to list matching documents without any embedding. The earlier POST /api/v1/query still works but has been deprecated since 25 October 2026 (changelog).

This matters because similarity can rank an outdated, regional or draft policy next to the applicable one. A refund question matches last year's policy as well as the current one, and the model then answers from text that is on topic and wrong.

A filter selects documents first, and only those are ranked by similarity Four stages from left to right. All indexed documents in the requested collections enter a filter step that evaluates tags, metadata, name, content and dates. The filter produces the set of allowed documents. Only the allowed documents are ranked by similarity to the search terms, which returns the top results. A branch below the filter step shows that when no document matches, the request returns an empty result without generating embeddings. Indexed documents every version, region, draft and tenant 1. Filter tags, metadata, name, content, dates 2. Allowed set only these documents can be returned 3. Rank by similarity search terms are embedded and compared to the set No match: empty result, no embedding generated
The filter selects the allowed documents first. Similarity ranking sees only that set, and a filter that matches nothing ends the request before any embedding is generated.

Should you filter before or after vector search?

Before, whenever the filter expresses a hard constraint such as region, version, tenant or "not a draft". Post-filtering searches first and discards non-matching results afterwards, so a strict filter can discard the whole candidate list and return nothing, even though matching documents exist. Pre-filtering avoids that by ranking only documents that already pass.

Vector engines handle this differently, and the trade-offs are real. Qdrant's write-up on pre-filtering versus post-filtering describes how pre-filtering gets costly when a filter matches broad parts of a collection, while post-filtering can return too little when the filter is strict. In the benchmark behind that post, which is specific to Qdrant's HNSW index (its filtered search article has the method), a broad-value filter lowered recall to 90.8% and an AND over two broad values to 39.7%, and both recovered to 100% with ACORN, a query-time traversal repair.

AIVAX resolves the filter first and ranks only the documents that pass. When the allowed set is small enough, every allowed document is scored exactly; for larger sets the search applies its usual candidate selection, restricted to the allowed documents. What the behavior gives you is a guarantee about ranking: a document that fails the filter is never a search result, and a filter that matches nothing costs no embedding. Reference expansion is the one exception, covered below.

Sending a filter with a query

Authenticate with an AIVAX API key from a trusted backend and send the filter field next to the search terms. An array is combined with and, which keeps each constraint on its own line:

curl -X POST https://inference.aivax.net/api/v2/query \
  -H "Authorization: Bearer $AIVAX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "collections": ["<collection-id>"],
    "terms": "How do I request a refund?",
    "limit": 5,
    "filter": [
      "tags has \"billing\"",
      "not tags has \"draft\"",
      "metadata.region = \"latam\""
    ]
  }'

A single collection can carry its own filter, sent as { "id": "<collection-id>", "filter": "..." } inside collections; it is combined with the top-level filter for that collection. The same field works on answer generation (POST /api/v1/answer) and on POST /api/v2/query/similar. The field name is filter; a misspelled filters is ignored and the search runs unfiltered. An invalid filter returns 400 Bad Request with the problem and its character position, such as Unknown field 'author'. Expected name, content, reference, tags, createdAt, updatedAt or metadata.<key>. (at 0).

What you can filter on

Filterable fields, what they hold, and the operators each accepts
FieldHoldsOperators
name, content, referenceDocument name, indexed text, and the reference that groups related documents such as chunks of one source= != in contains startswith endswith
tagsList of text tagshas in
createdAt, updatedAtTimestamps of the document= != > >= < <=, with ISO dates or now-7d
metadata.<key>Your own JSON, nested keys allowedequality, in, has (arrays), exists, numeric ordering, and text operators

Conditions combine with and, or, not and parentheses. not binds tighter than and, and and tighter than or. An array in filter is combined with and; for or, write it inside one string and use parentheses to group. Text comparison ignores case and accents, so name = "relatorio" matches a document named Relatório. Relative dates such as now-30d work only on createdAt and updatedAt, in units of minutes, hours, days, weeks, months and years, and both fields carry the document's AIVAX timestamps — a date that belongs to your content belongs in metadata as a number. The full grammar, units and limits are in the document filters reference.

Some filters you will use often:

tags in ("finance", "legal") and not tags has "draft"
metadata.region = "latam" and metadata.public = true
metadata.pages > 10 and metadata.pages <= 200
metadata.languages has "pt-BR"
name startswith "manual-v2-"
updatedAt >= "2026-09-01" and updatedAt < "2026-10-01"
(tags has "billing" or metadata.region = "latam") and createdAt >= now-1mo

Storing tags and metadata that filters can use

Filters can only select on what the document carries. When you import a document through JSONL, __tags takes a list of strings and __meta takes any JSON object:

{"docid":"refund-policy-2026","text":"Refund requests are accepted within 30 days...","__tags":["billing","policy"],"__meta":{"region":"latam","year":2026,"public":true}}

Decide the keys before you import, because a few behaviors shape the schema. Types are never converted: metadata.year = 2026 matches the number 2026 and does not match the string "2026", so keep one type per key, and if old data mixes both, list both with metadata.year in (2026, "2026"). Metadata dates cannot be compared as dates, because ordering operators on metadata work only on numbers — store 20260915 or a Unix timestamp, or filter on createdAt and updatedAt. Those two describe the AIVAX document, not the event in your text, so they are the wrong tool for "policies published in 2025".

Re-imports have one more trap. The collections docs state that a JSONL entry changing only the reference or tags is skipped, while a metadata-only change updates without reindexing. Change the text, or use the single-document endpoint, when tags must change, and check this in a test collection before relying on it for a large import. For ingestion in general, see our guide to converting a PDF to Markdown and chunks for RAG.

Why does your metadata filter return no results?

These are the cases that return an error or silently match something other than you expected. The quickest diagnosis is to send the filter alone, with no terms, to POST /api/v2/query/browse and look at what it selects.

Filters that fail or surprise, with the reason and the fix
FilterWhat happensWrite instead
tags = "billing"Rejected: tags supports only has and intags has "billing"
name == "x"Rejected: the operator is a single =name = "x"
metadata.price > "100"Rejected: ordering needs a numbermetadata.price > 100, stored as a number
createdAt >= "15/06/2025"Rejected: dates must be ISO 8601createdAt >= "2025-06-15"
content contains "ai"Rejected: contains needs at least 3 charactersA longer term
metadata.year = 2026Skips documents that store "2026" as textmetadata.year in (2026, "2026") until the data is cleaned
metadata.region != "emea"Also matches documents with no region key or no metadataAdd and metadata.region exists

The last row follows from two-valued logic: x != v means exactly not x = v, and a missing key never matches a positive condition. A filter like not tags has "draft" therefore includes documents that have no tags at all, which is usually what you want for "exclude drafts". Equality and contains also disagree about trailing spaces: equality ignores them, so metadata.code = "A1 " matches a stored "A1", while contains, startswith and endswith compare exactly.

Browsing with a filter and no search terms

POST /api/v2/query/browse accepts a request with only filter; POST /api/v2/query rejects a request without terms. Browse returns the matching documents oldest first unless you set sort, up to limit per page, and each result carries document without scores. No embedding is generated, no vector search runs, and no reranker is applied. Without a filter, browse lists every indexed document in the collections. Code that relied on the old filter-only order, newest first, now has to send "sort": { "by": "createdAt", "dir": "desc" }. The request contract is in the semantic search docs.

{
  "collections": ["<collection-id>"],
  "limit": 10,
  "sort": { "by": "createdAt", "dir": "desc" },
  "filter": "tags has \"incident\" and createdAt >= now-7d"
}

Use it for "show me what changed this week", for building a review queue, or to check what a filter selects before adding search terms. Rate limits still apply to the request.

Limits, cost and performance

  • A filter string is limited to 2,048 characters, 32 conditions, 8 levels of nesting, 100 values per in list, 8 keys in a metadata path and a relative amount capped at 100,000.
  • A filter must finish within 10 seconds. Conditions on content, endswith and tags examine every document in the requested collections. Metadata conditions have used an index of metadata values since November 2, so they stay fast in large collections. name with =, in or startswith is the cheapest. For very large corpora, split them into smaller collections or add a selective name condition.
  • Dates without Z or an offset are read in the server's local time zone, which the documentation gives as America/Sao_Paulo (UTC−03:00). Add an offset when you need an exact instant.
  • Reference expansion is not filtered. With "include": ["references"], a result is returned together with other documents that share its __ref, and that lookup does not re-apply the filter. It is off by default; keep it off when the filter is a hard constraint.
  • Automatic gateway RAG, which searches before the model call, does not apply filters. Filters reach a model through tools: the search tool of the Collections MCP, the query tool of gateways that use the QueryFunction strategy, and memory_search.

Cost follows the same shape: a filter-only query and a filter that matches nothing generate no query embedding, while a query with both terms and a matching filter still embeds the terms. See Pricing for current rates.

Filters are not access control

A filter can scope results, but only as strongly as the code that builds the filter, and it is not an access-control system. A filter is a string sent with the request. If a browser or an end user can supply it, they can change it. Build the filter on your backend from the authenticated user, never concatenate user-written text into it, and keep references out of include. Authorize the collection IDs on the backend as well. When the boundary is a legal or contractual one, prefer a separate collection per tenant over one shared collection with a tenant key, and still check access to each collection ID in your own code. For per-user scoping of agent memory, see our post on protecting agent memory from poisoning.

When filtering is not enough

A filter narrows the candidate set; it does not make the remaining candidates relevant. If the right document is in the allowed set but ranks too low, that is a ranking problem. Our guide on semantic search versus reranking explains how to tell the two apart. If you are still deciding what to build around a vector store, RAG versus a vector database lists what sits between vector search and a usable answer.

To try a filter without writing code, the collection playground has a Filters section where each line is a filter and all lines must match.


Filter behavior in this post was checked against the AIVAX implementation and its maintained documentation as read on 2 November 2026; the mistake table was additionally run through the filter parser on a four-document sample when the post was first published. We did not benchmark recall or latency under filters, and the Qdrant recall figures cited above describe that engine's benchmark, not AIVAX.