A Vector Database Is Not a RAG System
A vector database stores and searches vectorized records; a managed RAG system coordinates the wider path from source content to maintained, retrievable context.
A vector database can return the nearest records to a query vector and still leave you with most of a RAG system to build. The missing work starts before the first vector exists and continues after search returns: extracting source content, deciding chunk boundaries, tracking updates, reranking candidates, assembling context, measuring failures, and rebuilding representations when the corpus or model changes.
Vector databases solve a difficult, bounded problem, and some now include embedding models, lexical retrieval, hybrid search, and reranking. The practical question is how much additional application code, maintenance, and risk sits between their searchable records and your changing source material.
A vector database starts with records that are ready to index
At its core, a vector database stores vectors and associated data, builds indexes over those vectors, and executes similarity queries. pgvector, for example, adds vector similarity search to Postgres and supports exact search plus approximate indexes such as HNSW and IVFFlat. It gives an application a capable vector storage and query layer inside a familiar database.
That layer does not inherently know that a row came from page 37 of a revised employee handbook. It does not decide whether the page header is content or noise, whether a table should remain intact, whether a section needs to overlap its neighbor, or whether a deleted source file should remove twelve derived chunks. The application must either arrive with those decisions made or connect another service that makes them.
The unit mismatch explains much of the confusion. Source systems contain files, pages, tickets, records, media, permissions, and revision histories. Vector indexes contain searchable units with identifiers, vectors, text or payloads, and metadata. A production ingestion path has to translate one into the other without losing provenance, access rules, or update semantics.
If your application already produces clean, stable chunks, a vector database may be exactly the right abstraction. You keep control over parsing, embedding, deployment, and data movement while the database handles indexing and search. The mistake is treating that choice as if the surrounding responsibilities disappeared.
Managed RAG owns more of the path from source to context
“Managed RAG” is not a perfectly standardized product category. Services expose different connectors, parsers, stores, models, retrieval methods, and generation integrations. The useful definition is operational: the service coordinates a larger part of the lifecycle that turns source content into retrievable context and keeps that lifecycle running.
In production, this sequence repeats as sources change, parsing fails, embeddings time out, indexes lag, retrieval quality drifts, and permissions evolve. The operating system must detect each condition, preserve enough state to retry safely, and show which source revision produced the context used for an answer.
A managed service can assume more of that work, but only where its documentation and contract say it does. Vertex AI RAG Engine, for example, describes ingestion, transformation and chunking, embedding, indexing, retrieval, and generation as distinct parts of its RAG process. Amazon Bedrock Knowledge Bases explicitly distinguishes a Managed Knowledge Base, where AWS manages ingestion, indexing, storage, and retrieval infrastructure, from a customer-managed option where the customer controls the pipeline and vector store.
Integrated database features narrow the gap without erasing it
A fair comparison must account for how vector databases have expanded. Pinecone can accept vectors directly, but an index with integrated embedding can also accept source text, which Pinecone converts with the hosted embedding model associated with that index. Its search surface includes hybrid retrieval, and it can rerank initial results with hosted or external models.
Those features remove real integration work: with clean text records, stable identifiers, correct metadata, and an update stream, an application can use integrated embeddings to avoid a separate embedding client, combine semantic and lexical evidence through hybrid search, and rerank results without another deployment boundary.
They do not automatically connect to the system of record or determine how a PDF, wiki tree, database row, or video becomes those clean records. Nor do they establish deletion propagation, document-level access policy, source revision tracking, dead-letter handling, or evaluation criteria unless another documented feature supplies them. Apply the same scrutiny to managed RAG products because some still leave parsing or synchronization to the customer.
The responsibility matrix exposes what each purchase includes
| Responsibility | Vector database | Managed RAG service | What the buyer still decides |
|---|---|---|---|
| Source connection and synchronization | Usually supplied by the application or another ingestion product. | May provide connectors, scheduled sync, and deletion propagation. | Source authority, freshness target, and failure policy. |
| Parsing and extraction | Usually expects vectors, text records, or documents in an accepted schema. | May parse supported files, OCR scans, or describe non-text media. | Fidelity requirements and review of extraction errors. |
| Chunking | Usually receives already bounded records, although adjacent tools may help. | May transform source content into chunks and retain source relationships. | Chunk policy, overlap, context preservation, and domain exceptions. |
| Embeddings | Accepts customer-generated vectors; some products also embed submitted text. | Often coordinates an embedding model as part of ingestion. | Model fit, language coverage, dimensions, cost, and migration plan. |
| Indexing and retrieval | Core responsibility: storage, index maintenance, filtering, and vector or hybrid query. | Operates or integrates the retrieval store behind a higher-level knowledge interface. | Recall target, filters, candidate depth, latency, and tenant isolation. |
| Reranking and context assembly | May offer integrated or standalone reranking; context assembly is often external. | May rerank, cap, cite, and inject selected evidence into a model request. | Relevance judgments, context budget, citation policy, and fallback behavior. |
| Operations and reindexing | Exposes database health and index operations; upstream lineage remains external. | May track ingestion jobs, retries, source status, retrieval traces, and coordinated reindexing. | Service-level objectives, audit retention, evaluation gates, and what happens when models change. |
“May” is deliberate throughout the table. OpenAI File Search, for example, is a hosted tool that searches previously uploaded files through semantic and keyword search in vector stores. It supports a documented set of file types and metadata filtering. That is a larger managed surface than a bare vector index, but applications still decide which files to upload, how to assign metadata, when to replace them, and whether retrieved evidence is acceptable for the task.
Bedrock's managed option goes further in documented areas, including source connectors, Smart Parsing for several document and media types, service-managed embedding and reranking defaults, access-control-list filtering for supported connectors, and native observability. Its customer-managed Knowledge Base intentionally returns more control and more operational responsibility to the customer.
Managed operation does not delegate product judgment. Your team still decides which identities may retrieve each document, which metadata is safe to expose, how much latency and ingestion cost the application can accept, and what evidence qualifies an answer as supported. It must also test chunking and retrieval with domain-specific questions, because a service can execute a configured policy reliably while that policy remains wrong for the corpus. Evaluation belongs outside the provider's health dashboard: measure source coverage, candidate recall, ranking quality, citation support, and access-control behavior against cases that represent the application.
Cost needs the same end-to-end view. Storage price alone omits extraction, embedding, repeated synchronization, query volume, reranking, and full-corpus rebuilds. Compare the bill across the lifecycle you will operate instead of extrapolating from an isolated demo query. Even when the provider runs the machinery, your team defines acceptable results.
RAG failures often originate at the handoffs
A nearest-neighbor query can be correct while the answer is wrong. The index may faithfully return a chunk that lost its heading during extraction, carries stale permissions, or omits the exception stated in the next paragraph. Retrieval cannot reconstruct information removed upstream.
Chunking is one common fault line. Boundaries affect whether a chunk is independently understandable, whether tables survive, and whether a query can match the right level of detail. There is no universal chunk size that settles those questions. A managed chunker reduces implementation work, but you still need representative queries and relevance judgments to decide whether its output fits your corpus.
Synchronization creates another. Stable identifiers must connect derived chunks to source revisions so updates and deletions do not leave contradictory evidence behind. Reindexing must distinguish a metadata-only change from text that needs a new embedding. If the embedding model changes, old and new vectors may not belong in the same index space. Do not assume a managed service automatically migrates them. Ask whether it pins model versions, coordinates reindexing, supports a staged rebuild, and charges for the work.
The retrieval handoff has its own failure mode: a reranker only sees candidates selected by the first stage. Our guide to semantic search and reranking explains how to diagnose whether the relevant document is missing or merely ordered too low. Managed reranking does not compensate for poor candidate recall, just as a managed index does not compensate for missing source content.
Observability must therefore cross stage boundaries. Useful evidence includes source revision, extraction and chunking status, embedding model, index timestamp, query and filters, retrieved candidates, reranked order, final context, latency, and cost. A green database health check covers only part of that chain.
AIVAX keeps preparation, indexing, and ranking as explicit choices
AIVAX illustrates why product-specific boundaries matter. AIVAX Collections store persistent documents for semantic search. Direct and JSONL imports receive prepared document text with a stable name: Name in the single-document API or docid in JSONL. When text changes, the document is queued for reindexing; a metadata-only update does not reindex the text.
That import path does not claim that every submitted source is automatically segmented. If you have a source file rather than prepared documents, the dashboard's Media Injector can process it into RAG documents. If an application wants reviewable text boundaries first, Text Segmentation returns segments to the application but does not create embeddings or store the submitted document. For audio, images, video, or PDFs that need structured extraction, Media Descriptions is another explicit preparation step.
Once documents are indexed, AIVAX Semantic Search searches one or more collections. An optional reranker can adjust the order of retrieved candidates, while the standalone reranking API reorders candidate strings supplied by the application. Neither reranking path searches for text absent from its input.
These separate surfaces let an application choose where preparation happens instead of pretending that one import call fits every corpus. They also make the contract inspectable: source-file processing, segmentation, persistent indexing, retrieval, and reranking are related operations, not synonyms.
Match the service contract to the work your team can operate
Start with the artifacts you already have. If your system reliably produces clean chunks, stable IDs, vectors, metadata, and change events, a vector database gives you control without paying another service to repeat that work. If your inputs are changing files and media from several systems, the source-to-index pipeline may cost more to build and operate than the search layer itself.
Resolve four questions that the feature list and responsibility matrix cannot answer for your workload:
- When the embedding model or version changes, who coordinates a staged reindex, validates compatibility, and accounts for rebuild cost?
- Can every retrieved chunk be traced to its exact source revision and the access policy enforced for that request?
- Can telemetry connect ingestion to retrieved and reranked context, and can your evaluation measure coverage, recall, ranking quality, citation support, and access-control behavior?
- During partial failure, model deprecation, or a full reindex, which retries, rollback paths, and availability guarantees apply?
Record these answers in the architecture decision and assign each remaining operation to a team before selecting a product.
References
- pgvector: vector similarity search for Postgres
- Pinecone: upsert records and integrated embeddings
- OpenAI: File Search
- Amazon Bedrock Knowledge Bases
- Vertex AI RAG Engine overview
- AIVAX Collections and Documents
- AIVAX Media Injector
- AIVAX Semantic Search
- AIVAX Text Segmentation
- AIVAX Media Descriptions
- AIVAX Rerankers