Most teams agonize over vector database selection like it determines the success of their AI project. ChromaDB versus Pinecone versus pgvector versus Weaviate versus Qdrant versus whatever launched last Tuesday. The comparison matrices multiply. The benchmarks get cited. The architecture diagrams get drawn.
Then the project ships and the vector database choice turns out to be approximately the fifth most important decision they made.
The choice matters. But the conversation around vector database selection has drifted into territory that serves vendors more than developers. Meanwhile, the harder problems — chunking strategy, token budget management, retrieval augmented generation pipeline design — get less attention because they lack comparison matrices.
What Vector Databases Actually Do
At the conceptual level, a vector database stores numerical representations of data — embeddings — and retrieves them based on similarity rather than exact matching. A query like "authentication middleware" does not need to match those exact words in your codebase. Instead, it finds code whose embedding is mathematically close to the embedding of that query.
The core operations are: 1. Insert — Store vectors with associated metadata 2. Query — Find the N most similar vectors to a given input 3. Filter — Narrow results by metadata before or after similarity search 4. Delete — Remove vectors when underlying data changes
That is the job. Everything else — distributed architecture, persistence options, hosting models, query languages, indexing algorithms — is implementation detail. Important implementation detail, but detail nonetheless.
The actual hard problem in semantic search is not where the vectors live. It is whether the vectors mean anything useful in the first place. A RAG pipeline with poor semantic chunking returns garbage regardless of which vector store backs it.
The Benchmarks Everyone Cites, The Reality Nobody Mentions
Vector database benchmarks measure queries per second, recall at various K values, and latency distributions. These numbers are real and reproducible. They are also almost entirely irrelevant to most production use cases.
The typical codebase semantic search scenario: - Database size: 5,000 to 50,000 chunks - Query volume: A few queries per minute, maybe less - Latency requirement: Under a second is fine, under 100ms is nice
At this scale, every major vector database performs identically in practice. ChromaDB on SQLite handles it. Pinecone handles it. pgvector handles it. The billion-scale benchmarks that vendors publish are solving a different problem than most teams have.
The benchmarks that would actually matter — embedding quality degradation over time, query drift as codebases evolve, retrieval precision for domain-specific terminology, context compression effectiveness — those do not get published. Harder to measure. Harder to market.
ChromaDB: The Default That Mostly Works
ChromaDB became the default for Python projects building RAG pipelines and semantic search. There are reasons for this beyond hype.
Setup is a single pip install. No server to run, no connection strings to manage, no infrastructure decisions to make on day one. The Python API is straightforward enough that a working prototype exists within an hour.
For local development and moderate-scale production, ChromaDB on SQLite is sufficient. The chunks live in a file. Queries run fast enough. Life is simple.
Where ChromaDB requires thought: - Persistence across restarts — SQLite mode handles this; in-memory does not - Scaling beyond single-machine — The client-server mode exists but adds deployment complexity - Multi-tenancy — Collections work but isolation is at the application level
The honest assessment: ChromaDB is a good choice for projects where the vector database is a component, not the product. If semantic search is one feature among many, optimizing the storage layer is premature.
Pinecone: Managed Simplicity at Managed Prices
Pinecone's value proposition is not performance. It is not having to think about vectors at all.
There is no database to deploy, no indexes to tune, no storage to monitor. Vectors go in through an API. Vectors come out through the same API. The infrastructure is someone else's problem.
This is genuinely valuable for teams where: - DevOps capacity is limited - Time-to-production matters more than unit economics - Scale is unpredictable and might spike
Where Pinecone creates friction: - Cost at scale — The pricing model assumes you are building something that will eventually make money. Hobby projects and internal tools feel expensive. - Vendor lock-in — The proprietary query language and hosting model make migration nontrivial - Latency for local development — Every query hits the network, which matters when iterating rapidly
The calculation is straightforward. Is avoiding infrastructure management worth the monthly invoice? For some teams, unambiguously yes. For others, the answer changes as scale increases and the bill arrives.
pgvector: The Boring Option That Keeps Winning
If the application already uses Postgres, pgvector is the obvious choice that teams consistently underrate.
Adding vector search to an existing Postgres deployment means: - No new infrastructure to manage - No new backup strategy to implement - No new monitoring to configure - No new vendor relationship to negotiate - Transactions that span vectors and relational data
The SQL integration is the underrated advantage. Filtering by metadata is just a WHERE clause. Joining vectors with relational data is just a JOIN. The query optimizer understands both sides.
Performance at moderate scale is fine. The IVFFlat and HNSW indexing options cover most use cases. For a 50,000-chunk codebase, query times are measured in low milliseconds.
Where pgvector requires adjustment: - Index tuning matters more — Default settings are not optimized for vector workloads - Dimension limits — Very high-dimensional embeddings need configuration - Pure vector workloads — If there is no relational data, adding Postgres for vector search alone is overhead
The pattern that works well: start with pgvector because the Postgres instance already exists, migrate to a dedicated solution if and when vector operations become the bottleneck. This migration path usually never gets triggered.
The Decision That Actually Matters
The choice between ChromaDB, Pinecone, pgvector, or any other vector store is a second-order decision. The first-order decision is: what embeddings go into the database?
Generic embedding models trained on web text encode a version of language that does not match how developers write code. The abbreviation svc might embed nowhere near the word service. The function handleAuthCallback might not cluster with authentication concepts at all. Team-specific terminology — the internal names, the project abbreviations, the domain jargon — exists outside the training distribution.
A poorly chosen embedding model in a perfectly optimized vector database still returns irrelevant results. A well-tuned embedding model in a SQLite-backed prototype returns useful results. The lost in the middle problem — where models ignore information buried in long context windows — does not care which database stored the chunks.
Most teams optimize in the wrong order. They compare query-per-second benchmarks between vector stores while using off-the-shelf embeddings that do not understand their codebase. The database selection conversation is easier than the embedding quality conversation. Vendor comparison matrices are more shareable than "we need to evaluate whether our embeddings actually encode our domain."
Where This Is Heading
The vector database market is consolidating around a few patterns:
Postgres is eating the embedded use case. pgvector keeps improving. For applications that already have Postgres, adding vector capabilities is increasingly the path of least resistance. The "best tool for the job" philosophy is losing to the "tool I already have that does the job well enough" reality.
Managed services are competing on ecosystem, not performance. Pinecone, Weaviate Cloud, and others are differentiating through integrations, SDKs, and developer experience rather than raw query speed. At scales where performance matters, teams are building custom solutions anyway.
Local-first is gaining ground. ChromaDB, LanceDB, and similar projects optimize for the developer laptop use case. The ability to iterate without network latency or token cost overhead matters for prototyping and development workflows.
The embedding model decision is finally getting attention. Domain-specific fine-tuning, retrieval augmented embeddings, and hybrid search combinations are where actual retrieval quality improvements come from. This was always true. The industry is slowly acknowledging it.
The Honest Take
Vector database selection is a solved problem for most teams. Pick the one that matches your existing infrastructure. If you have Postgres, add pgvector. If you want zero infrastructure, use Pinecone and accept the cost. If you are building a Python prototype, ChromaDB is fine.
The agonizing comparison between options is usually a form of productive procrastination — it feels like work without requiring the harder decisions about embedding quality, chunking strategy, token efficiency, and retrieval evaluation.
The vendors benefit from extended evaluation cycles. Developer time spent comparing vector stores is developer time not spent questioning whether the fundamental approach is right.
The teams that ship working semantic search usually spent more time on embedding selection and evaluation than on vector database benchmarking. The teams still evaluating vector stores after three months have usually optimized for the wrong variable.
The database choice matters less than vendors admit, and more than "just pick one" suggests. The nuance is that it matters in ways that benchmarks do not capture — developer experience, operational overhead, migration paths, token budget implications, total cost over time. These are not numbers. They are judgment calls that depend on context.
Most comparison articles will not tell you that. It is harder to optimize for judgment than for queries per second.