Why Vector Database Architecture Matters

Vector databases solve a fundamentally different problem than traditional databases. Instead of exact key matching, they perform approximate nearest neighbor (ANN) search across high-dimensional embeddings. The difference in data structure requirements is profound: a B-tree index that works perfectly for SQL WHERE clauses is useless for finding the 10 most semantically similar documents among 10 million vectors.

The core tradeoff: accuracy versus speed versus memory. You can build an index that returns perfectly accurate nearest neighbors by comparing every vector (exact search), but it takes linear time—O(n) complexity that becomes unusable above 100,000 vectors. Approximate algorithms like HNSW or IVF trade accuracy for speed by organizing vectors into navigable structures, but these structures consume memory proportional to your vector count and dimensionality.

This matters because embedding dimensions vary wildly. OpenAI's text-embedding-3-small uses 1536 dimensions. Cohere's embed-multilingual-v3.0 uses 1024. Open-source models like sentence-transformers offer 384-768 dimensions. Each dimension adds 4 bytes per vector (for float32), so 1 million vectors at 1536 dimensions consumes 6GB just for the raw embeddings before indexing overhead, which typically multiplies storage by 1.5-3x.

The Hidden Cost of Hybrid Search

Most production AI applications need hybrid search: vector similarity combined with metadata filtering. You're not just finding similar documents—you're finding similar documents where user_id matches, created_at is within the last 30 days, and status equals "published". Naive implementations execute the vector search first, then filter results, which destroys performance when filters eliminate 99% of candidates.

Better implementations pre-filter before vector search, but this requires bitmap indexes or inverted indexes alongside your vector index. Now you're managing two index types, coordinating updates, and dealing with consistency challenges. Some vector databases handle this elegantly. Others bolt filtering on as an afterthought, resulting in queries that are fast for pure vector search but slow by 10-100x when you add a simple WHERE clause.

This architectural difference separates production-ready vector databases from research projects. If your application only needs pure semantic search with no filtering, any database works. The moment you need to scope search by user, tenant, time range, or category, you need a database that treats hybrid search as a first-class concern, not a feature addition.

Pinecone: Managed Vector Database for Production Scale

Pinecone is a fully managed vector database designed for production workloads where operational complexity is the primary constraint. You don't manage indexes, sharding, or replicas—you push vectors via API and query them. This managed approach trades flexibility for reliability: you can't tune low-level index parameters, but you also don't spend weekends debugging cluster rebalancing failures.

Architecture and Performance Characteristics

Pinecone uses a pod-based architecture where each pod provides a fixed amount of vector capacity. A p1 pod stores approximately 1 million vectors at 1536 dimensions with metadata. Query latency typically ranges from 50-200ms at p95 depending on index size and query complexity. The database scales horizontally by adding pods and vertically by using larger pod types (p1, p2, s1 for storage-optimized workloads).

// Initialize Pinecone client
import { Pinecone } from '@pinecone-database/pinecone';

const pc = new Pinecone({
  apiKey: process.env.PINECONE_API_KEY
});

// Create index with specific configuration
await pc.createIndex({
  name: 'semantic-search',
  dimension: 1536,
  metric: 'cosine',
  spec: {
    serverless: {
      cloud: 'aws',
      region: 'us-east-1'
    }
  }
});

// Insert vectors with metadata for hybrid search
const index = pc.index('semantic-search');
await index.upsert([
  {
    id: 'doc-1',
    values: embedding, // 1536-dimensional array
    metadata: {
      userId: 'user_123',
      category: 'technical',
      createdAt: 1709856234
    }
  }
]);

// Query with metadata filtering
const results = await index.query({
  vector: queryEmbedding,
  topK: 10,
  filter: {
    userId: { $eq: 'user_123' },
    category: { $in: ['technical', 'tutorial'] }
  },
  includeMetadata: true
});

The metadata filtering is where Pinecone shows architectural maturity. Filters execute before vector search when possible, using optimized indexes. In benchmarks with 1 million vectors, adding a filter that matches 10% of documents increases query latency by ~20-30ms, not 10x. This efficiency comes from maintaining separate indexes for metadata fields marked as filterable.

When Pinecone is the Right Choice

Pinecone excels when operational simplicity outweighs cost optimization. You're paying a premium for managed infrastructure—starter plans begin at $70/month for 1 million vectors—but you're buying engineering hours back. No Kubernetes clusters to maintain, no index tuning, no capacity planning beyond "add another pod".

It's particularly strong for teams building RAG applications that need fast iteration. The developer experience is excellent: client libraries in 7 languages, clear documentation, and sensible defaults. You can have a working semantic search implementation running in production within hours, not weeks.

Where Pinecone Shows Limitations

The pod-based pricing becomes expensive at scale. Each pod has fixed capacity regardless of actual usage. If you have 800K vectors, you still pay for a full pod. At 5 million vectors, you need 5 pods at ~$350/month before considering replication or multi-region deployments.

The serverless option (launched in 2023) addresses this with pay-per-query pricing, but introduces cold start latency. The first query after idle periods can take 1-2 seconds while the index loads into memory. This makes serverless unsuitable for user-facing applications with strict latency requirements.

Weaviate: Open-Source Vector Database with GraphQL Interface

Weaviate positions itself as the vector database for applications that need to combine vector search with complex data relationships. Unlike databases that treat metadata as simple key-value pairs, Weaviate models data as a graph with typed relationships. This architectural choice makes it powerful for knowledge graphs and recommendation systems where entities connect in meaningful ways.

Schema-First Architecture

Weaviate requires you to define a schema before inserting data. This feels restrictive compared to schemaless options, but it enables sophisticated query capabilities. You're explicitly declaring that a Product has a "belongsToCategory" relationship with a Category, and Weaviate can traverse these relationships during vector search.

// Define schema with relationships
const schemaConfig = {
  class: 'Article',
  properties: [
    {
      name: 'title',
      dataType: ['text']
    },
    {
      name: 'content',
      dataType: ['text']
    },
    {
      name: 'author',
      dataType: ['Author'], // Reference to Author class
    },
    {
      name: 'category',
      dataType: ['string']
    },
    {
      name: 'publishedAt',
      dataType: ['date']
    }
  ],
  vectorizer: 'text2vec-openai', // Auto-generate embeddings
  moduleConfig: {
    'text2vec-openai': {
      model: 'text-embedding-3-small',
      vectorizeClassName: false
    }
  }
};

await client.schema.classCreator().withClass(schemaConfig).do();

// Insert with automatic vectorization
await client.data.creator()
  .withClassName('Article')
  .withProperties({
    title: 'Vector Database Comparison',
    content: 'Article content here...',
    author: { beacon: 'weaviate://localhost/Author/uuid-123' },
    category: 'technical',
    publishedAt: new Date().toISOString()
  })
  .do();

// Query with relationship traversal
const result = await client.graphql
  .get()
  .withClassName('Article')
  .withNearText({ concepts: ['database performance'] })
  .withFields('title content _additional { distance } author { ... on Author { name } }')
  .withWhere({
    operator: 'And',
    operands: [
      { path: ['category'], operator: 'Equal', valueString: 'technical' },
      { path: ['publishedAt'], operator: 'GreaterThan', valueDate: '2024-01-01' }
    ]
  })
  .withLimit(10)
  .do();

The vectorizer modules are Weaviate's killer feature. Instead of generating embeddings in your application code, Weaviate can automatically vectorize text using OpenAI, Cohere, or open-source models. This simplifies your ingestion pipeline—you send text, Weaviate handles embedding generation and storage atomically.

Performance and Scaling Characteristics

Weaviate uses HNSW indexing with configurable parameters. You can tune efConstruction (build-time accuracy vs speed) and ef (query-time accuracy vs speed) based on your workload. Higher values increase accuracy but consume more memory and CPU. The defaults (efConstruction: 128, ef: -1 for dynamic) work well for most use cases, but you have the option to optimize.

Sharding is manual in self-hosted deployments. You define shard count at class creation, and Weaviate distributes data across shards using consistent hashing. This gives you control over data distribution but requires capacity planning—adding shards later requires reindexing. Weaviate Cloud Services (WCS) handles this automatically, similar to Pinecone's model.

When to Choose Weaviate

Weaviate is optimal when your vector search must respect complex relationships. Building a recommendation engine where you need "products similar to this one, but from different brands within the same category, purchased by users with similar preferences" is where Weaviate's graph capabilities shine. Other vector databases require multiple queries and application-level joins.

It's also strong for teams that want to self-host on Kubernetes. The deployment model is cloud-native: Docker containers, Kubernetes operators, horizontal pod scaling. If you're already running stateful workloads on Kubernetes and have operational expertise, self-hosting Weaviate is viable and cost-effective compared to managed alternatives.

Where Weaviate Requires More Work

The schema requirement adds friction. You can't prototype by just pushing vectors—you must design your data model first. This is architectural discipline, but it slows initial development. For RAG applications that just need "find similar documents," Weaviate's capabilities are overkill.

The GraphQL interface is powerful but unfamiliar. Most developers know REST or SQL. Learning GraphQL query syntax, understanding how fragments work, and debugging complex queries takes time. The JavaScript and Python clients abstract much of this, but when queries fail, you're debugging GraphQL.

Qdrant: Performance-Focused Vector Database in Rust

Qdrant is built in Rust with a singular focus: query performance. Benchmarks consistently show Qdrant achieving lower latency than competitors at the same accuracy levels. This performance comes from careful engineering of the index structures and aggressive use of memory-mapped files to keep working sets in OS page cache.

Architecture and Query Engine

Qdrant's differentiator is its filtering architecture. Instead of treating filters as an afterthought, Qdrant builds payload indexes (their term for metadata indexes) that work together with vector indexes. When you query with filters, Qdrant can choose between filter-first or vector-first execution depending on filter selectivity, similar to how SQL query planners choose index strategies.

// Initialize Qdrant client
import { QdrantClient } from '@qdrant/js-client-rest';

const client = new QdrantClient({
  url: process.env.QDRANT_URL,
  apiKey: process.env.QDRANT_API_KEY
});

// Create collection with specific distance metric and index config
await client.createCollection('documents', {
  vectors: {
    size: 1536,
    distance: 'Cosine'
  },
  optimizers_config: {
    memmap_threshold: 20000 // Use mmap for segments > 20K vectors
  },
  hnsw_config: {
    m: 16, // Connections per layer
    ef_construct: 100, // Build-time accuracy
    full_scan_threshold: 10000 // Use brute force below 10K vectors
  }
});

// Insert with rich payload
await client.upsert('documents', {
  points: [
    {
      id: 1,
      vector: embedding,
      payload: {
        title: 'Vector database performance',
        author: 'john_doe',
        tags: ['database', 'performance', 'ai'],
        metadata: {
          userId: 'user_123',
          timestamp: 1709856234,
          category: 'technical'
        }
      }
    }
  ]
});

// Query with complex filtering
const results = await client.search('documents', {
  vector: queryEmbedding,
  limit: 10,
  filter: {
    must: [
      {
        key: 'metadata.userId',
        match: { value: 'user_123' }
      },
      {
        key: 'tags',
        match: { any: ['database', 'ai'] }
      }
    ],
    must_not: [
      {
        key: 'metadata.category',
        match: { value: 'archived' }
      }
    ]
  },
  with_payload: true,
  with_vector: false
});

The filtering DSL is more expressive than most competitors. You can combine AND, OR, NOT operations, use range queries on numeric fields, and filter on nested JSON structures. The query planner examines filter cardinality and chooses whether to filter before vector search (when filters are highly selective) or after (when filters match most vectors).

Memory-Mapping Strategy

Qdrant's use of memory-mapped files is sophisticated. Vectors are stored in segments. Small segments (below memmap_threshold) stay in RAM for maximum speed. Large segments use mmap, letting the OS manage which pages stay in memory. This hybrid approach keeps hot data fast while allowing indexes much larger than available RAM.

In practice, this means you can run a 50GB index on a server with 16GB RAM. Query performance depends on access patterns—frequently accessed vectors stay in page cache, while cold vectors trigger page faults. For applications with localized queries (like user-specific search where each user's vectors are spatially clustered), this works remarkably well.

When Qdrant is the Best Choice

Qdrant excels for applications where query latency directly impacts user experience. If your AI feature is in the critical path of page rendering and you need p95 latencies under 50ms, Qdrant's performance edge matters. Benchmarks show 20-40% lower latency than alternatives at equivalent accuracy.

It's also strong for cost-conscious teams willing to self-host. Qdrant's memory efficiency means you can handle more vectors per dollar of infrastructure. A server with 32GB RAM can handle collections that would require 64GB+ in less optimized databases.

Operational Considerations

Self-hosting requires Rust expertise for deep troubleshooting, though normal operations don't. Qdrant provides Docker images and Kubernetes Helm charts. The HTTP API is straightforward, and the gRPC option exists for lower latency (saves ~10-15ms per request by eliminating HTTP parsing overhead).

The managed Qdrant Cloud service launched in 2023 eliminates operational burden. Pricing is competitive with Pinecone—$0.10 per GB-hour for storage, $0.60 per compute hour for query nodes. For 1 million vectors at 1536 dimensions (~6GB), expect ~$45/month plus compute costs.

PostgreSQL with pgvector: Vector Search in Postgres

pgvector extends PostgreSQL with vector data types and similarity search. This matters because it eliminates a database in your stack. Instead of syncing data between Postgres and a separate vector database, you store vectors alongside relational data and query them together. For applications where vector search is a feature, not the core product, this architectural simplification is compelling.

Capabilities and Limitations

pgvector supports vectors up to 2,000 dimensions and provides three distance metrics: L2 (Euclidean), inner product, and cosine similarity. It uses HNSW indexes for approximate search above ~100K vectors. Below that threshold, it falls back to sequential scan, which is fine for small collections but becomes prohibitively slow at scale.

-- Create table with vector column
CREATE TABLE documents (
  id SERIAL PRIMARY KEY,
  content TEXT,
  embedding vector(1536),
  user_id INTEGER,
  category TEXT,
  created_at TIMESTAMP DEFAULT NOW()
);

-- Create HNSW index for fast ANN search
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

-- Create traditional indexes for filtering
CREATE INDEX ON documents(user_id);
CREATE INDEX ON documents(category);

-- Hybrid query: vector similarity + SQL filtering
SELECT id, content, 1 - (embedding <=> $1) AS similarity
FROM documents
WHERE user_id = $2
  AND category = ANY($3)
  AND created_at > NOW() - INTERVAL '30 days'
ORDER BY embedding <=> $1
LIMIT 10;

The query combines vector similarity (the <=> operator) with standard SQL WHERE clauses. Postgres's query planner can use indexes for both the filter conditions and the vector search. This integration is pgvector's primary advantage—you're writing SQL, not learning a new query language.

Performance Reality Check

pgvector is significantly slower than dedicated vector databases. In benchmarks with 1 million vectors, Qdrant averages 20-30ms p95 latency while pgvector averages 100-200ms. This gap widens with collection size. Above 10 million vectors, pgvector struggles to maintain acceptable query times even with properly tuned indexes.

The reason is fundamental: Postgres wasn't designed for vector workloads. HNSW index operations compete with OLTP query optimization, WAL writes, and vacuum operations. During high write throughput, vector query latency becomes unpredictable. Dedicated vector databases optimize for this single workload and consistently deliver lower latencies.

When pgvector Makes Sense

pgvector is ideal for applications where vectors are a secondary feature. If you're building a documentation search feature for your SaaS product and you already use Postgres, adding pgvector is simpler than deploying Pinecone. Your vector count stays below 1 million, query latency under 200ms is acceptable, and operational simplicity trumps raw performance.

It's also valuable during prototyping. You can validate that semantic search improves your user experience before committing to dedicated vector infrastructure. Migrating from pgvector to a specialized database later is straightforward—vectors are just arrays, easily exported and imported.

When to Choose a Dedicated Vector Database

When vector queries are in the critical path and latency directly impacts UX, pgvector's performance gap becomes unacceptable. When your collection exceeds 5-10 million vectors, you'll spend more time tuning pgvector than it would take to deploy Qdrant or Pinecone. When you need specialized features like filtered search with minimal overhead, automatic vectorization, or multi-vector queries, dedicated databases provide capabilities pgvector lacks.

Milvus: Large-Scale Vector Database for Research and Production

Milvus targets the high end of vector database requirements: billions of vectors, distributed across clusters, with guarantees around consistency and durability. It's architecturally complex—separating storage, indexing, and query layers—but this separation enables true horizontal scaling that simpler databases can't match.

Architecture for Scale

Milvus uses a disaggregated architecture inspired by cloud-native data systems. The log broker (using Pulsar or Kafka) handles write replication. Object storage (S3, MinIO) stores segments. Query nodes are stateless and can be scaled independently of data nodes. This complexity pays off when you need to scale read and write throughput independently or when durability requirements demand multi-region replication.

For small to medium workloads (under 10 million vectors), this architecture is overkill. The operational overhead of managing a Milvus cluster—coordinating Pulsar, object storage, query nodes, and data nodes—exceeds the complexity of running Qdrant or Weaviate. But above 100 million vectors, especially when write throughput is high, Milvus's architecture enables scaling patterns that aren't possible in simpler systems.

Index Options and Tuning

Milvus supports more index types than alternatives: IVF_FLAT, IVF_SQ8, IVF_PQ, HNSW, and ANNOY. Each has different memory vs accuracy vs query speed tradeoffs. IVF_PQ (inverted file with product quantization) compresses vectors aggressively, enabling larger-than-RAM indexes at the cost of accuracy. HNSW provides better accuracy but consumes more memory. The ability to choose indexes per collection based on workload is valuable at scale.

When Milvus is Appropriate

Milvus is designed for organizations where vector search is core infrastructure serving multiple applications. If you're building a search platform that will support dozens of internal use cases, Milvus's multi-tenancy features (collections with isolated resources) and operational tooling justify the complexity. For single-application vector search, simpler alternatives require less operational investment for equivalent functionality.

Chroma: Developer-Friendly Embedding Database

Chroma positions itself as the embedding database for LLM applications. It started as a Python library you could run in-process (like SQLite for vectors) but has evolved to support client-server architecture for production deployments. The value proposition: get started in 3 lines of code, scale up when needed.

Developer Experience Focus

import chromadb

# In-memory database for development
client = chromadb.Client()

# Create collection with auto-embedding
collection = client.create_collection(
    name="documents",
    metadata={"hnsw:space": "cosine"}
)

# Add documents - Chroma handles embedding generation
collection.add(
    documents=[
        "This is a document about vector databases",
        "RAG systems need efficient retrieval"
    ],
    metadatas=[
        {"category": "database", "user_id": "user_123"},
        {"category": "ai", "user_id": "user_123"}
    ],
    ids=["doc1", "doc2"]
)

# Query with natural language
results = collection.query(
    query_texts=["tell me about databases"],
    n_results=5,
    where={"user_id": "user_123"}
)

This developer experience is Chroma's differentiator. You don't manage embeddings explicitly—Chroma generates them using configurable embedding functions (OpenAI, Cohere, sentence-transformers). For prototyping RAG applications, this eliminates boilerplate.

Production Limitations

Chroma is younger than alternatives and shows it. Production deployments require the client-server mode, which adds network latency. Query performance lags behind Qdrant and Pinecone in benchmarks. The filtering syntax is less expressive than competitors. For applications past the prototype stage, you'll likely migrate to more mature infrastructure.

Chroma is best viewed as a development tool: excellent for validating that semantic search improves your application, but not the database you'll run in production at scale. Use it to prove the concept, then evaluate production-grade alternatives based on your specific requirements.

Decision Framework: Matching Database to Requirements

Choose based on your constraints, not features. The database with the longest feature list isn't necessarily the right choice—the one that aligns with your operational capabilities and performance requirements is.

Scenario 1: RAG Application for SaaS Product

You're adding semantic search to an existing SaaS application. Vector count: 100K-1M. Query latency requirement: under 200ms. Engineering resources: 1-2 developers, no dedicated infrastructure team.

Recommendation: Pinecone or Qdrant Cloud. The managed approach eliminates operational complexity. Pinecone if you prioritize simplicity and vendor stability. Qdrant Cloud if you need the lowest latency or want to self-host in the future (the API is identical).

Scenario 2: Recommendation Engine with Complex Relationships

Building a content recommendation system where recommendations must respect category hierarchies, user preferences, and content relationships. Vector count: 1M-10M. Need to traverse relationships during search.

Recommendation: Weaviate. The graph capabilities and schema-based relationships are purpose-built for this use case. Self-host on Kubernetes if you have the expertise, or use Weaviate Cloud Services if you want managed hosting.

Scenario 3: Small-Scale Semantic Search

Adding semantic search to a Postgres-backed application. Vector count: under 500K. Already running Postgres in production. Team has strong SQL skills but no vector database experience.

Recommendation: pgvector. The simplicity of keeping everything in Postgres outweighs performance limitations at this scale. Migrate to a dedicated database if query latency or vector count grows beyond pgvector's capabilities.

Scenario 4: Large-Scale Multi-Tenant Vector Search

Building vector search infrastructure for multiple internal applications. Vector count: 100M+. Need strict tenant isolation, multi-region deployment, and independent read/write scaling.

Recommendation: Milvus. The architectural complexity is justified at this scale. You need a dedicated platform team to operate it, but the scaling characteristics and multi-tenancy features aren't available in simpler alternatives.

Cost Analysis: TCO Across Vector Databases

Managed services trade dollars for engineering time. Self-hosted options trade engineering time for infrastructure costs. Neither is universally better—the right choice depends on your team's opportunity cost.

Managed Database Costs (1M vectors, 1536 dimensions)

Pinecone: ~$70/month for a single p1 pod, ~$140/month with replication for high availability. Serverless starts at $0.081 per 1M queries plus $0.10 per GB-hour storage.

Qdrant Cloud: ~$45/month for storage (6GB at $0.10/GB-hour) plus compute costs (~$0.60/hour for query nodes). With one query node running 24/7: ~$478/month. Use autoscaling to reduce compute costs during low-traffic periods.

Weaviate Cloud Services: Pricing similar to Qdrant, starting around $50/month for small deployments, scaling based on usage.

Self-Hosted Costs

Hardware requirements: For 1M vectors at 1536 dimensions, plan for 16-32GB RAM (depending on index type), 2-4 CPU cores. AWS c6a.2xlarge (~$250/month) or equivalent handles this workload with room for growth.

But self-hosting includes operational costs: monitoring, backups, updates, security patching, on-call rotations. If you value engineering time at $150/hour and spend 10 hours/month on database operations, that's $1,500 in labor costs. Managed services start looking economical unless you're operating at scale where labor costs amortize across many databases.

Migration Strategy: Changing Vector Databases

You'll likely change vector databases as your requirements evolve. Design for this from the start by abstracting database operations behind an interface. Your application code should call findSimilar(vector, filters, limit), not Pinecone-specific or Qdrant-specific APIs.

Abstraction Layer Pattern

// Vector database interface
interface VectorStore {
  insert(id: string, vector: number[], metadata: Record): Promise;
  search(vector: number[], filters: Filter[], limit: number): Promise;
  delete(id: string): Promise;
}

// Pinecone implementation
class PineconeStore implements VectorStore {
  async search(vector: number[], filters: Filter[], limit: number) {
    // Translate generic filters to Pinecone format
    const pineconeFilter = translateFilters(filters);
    const results = await this.index.query({
      vector,
      filter: pineconeFilter,
      topK: limit
    });
    return results.matches.map(m => ({
      id: m.id,
      score: m.score,
      metadata: m.metadata
    }));
  }
}

// Qdrant implementation
class QdrantStore implements VectorStore {
  async search(vector: number[], filters: Filter[], limit: number) {
    // Translate generic filters to Qdrant format
    const qdrantFilter = translateFilters(filters);
    const results = await this.client.search('collection', {
      vector,
      filter: qdrantFilter,
      limit
    });
    return results.map(r => ({
      id: r.id.toString(),
      score: r.score,
      metadata: r.payload
    }));
  }
}

With this abstraction, migrating databases means implementing a new class and running a data migration script. Your application code doesn't change. The migration itself: export vectors from the old database, transform IDs/metadata if needed, batch import to the new database, run validation queries, switch traffic.

Frequently Asked Questions

Can I use multiple vector databases in the same application?

Yes, and this is sometimes optimal. Use pgvector for small per-user vector collections that benefit from transactional consistency with user data. Use Pinecone for the large shared corpus that powers semantic search. Keep the databases serving different workloads rather than duplicating data across both.

How do I handle embedding model updates?

Embeddings from different models aren't comparable—you can't query with gpt-3.5-turbo embeddings against a corpus embedded with text-embedding-ada-002. Model updates require reindexing your entire corpus. Budget for this: maintain the old index while building the new one, validate quality on a sample, then switch traffic. This is why picking a stable embedding model matters.

What's the difference between cosine and dot product similarity?

For normalized embeddings (L2 norm = 1), cosine similarity and dot product are equivalent. OpenAI and Cohere embeddings are normalized, so either works. For non-normalized embeddings, cosine measures angle between vectors (direction) while dot product measures angle and magnitude. If your embedding model doesn't document normalization, use cosine to be safe.

How many vectors can I store before performance degrades?

Depends on the database and query patterns. pgvector struggles above 1-2M vectors. Pinecone and Qdrant handle 10M+ vectors with proper configuration. Milvus scales to billions. The real constraint is often query latency—larger indexes mean longer paths through the HNSW graph. Test with your target dataset size and measure p95 latency.

Should I store vectors and source data in the same database?

Store minimal metadata with vectors (enough to render results), keep full documents in your primary database. Vector databases optimize for similarity search, not complex queries or transactions. When you need the full document content, fetch it by ID from Postgres/MongoDB after the vector query returns IDs. This separation keeps vector queries fast.

How do I handle deletions in vector databases?

Most vector databases delete asynchronously—deleted vectors may appear in query results briefly. If immediate deletion is required (GDPR right to erasure), maintain a deletion blocklist in your application layer and filter results after the vector query. Or use a database like Weaviate that supports consistent deletes at the cost of write throughput.

Can I update metadata without reindexing vectors?

Yes in most databases. Metadata and vectors are stored separately. Updating metadata doesn't require recomputing the vector index. However, if you're filtering on that metadata field, the metadata index needs updating, which can be slow for large collections. Batch metadata updates when possible.

How do I choose between managed and self-hosted?

Use managed services unless you have dedicated infrastructure engineers and need the cost savings at scale. The operational burden of running distributed systems is high. Managed services are expensive per unit of compute, but cheap per unit of engineering time. Self-host when the cost delta justifies hiring platform engineers.

What's the relationship between accuracy and performance?

ANN search trades accuracy for speed. At 95% recall (95% of true nearest neighbors found), queries are 10-100x faster than exact search. At 99% recall, maybe 5-10x faster. At 99.9%, approaching exact search speed. Most applications don't notice the quality difference at 95% recall—user satisfaction with search results is influenced more by embedding quality than index accuracy.

How do I benchmark vector databases for my use case?

Use your actual data and query patterns. Public benchmarks use random vectors, which don't reflect real embedding distributions. Export 100K vectors from your application, run queries representative of your traffic mix, measure p50/p95/p99 latency and recall. Test with and without metadata filtering since filtering performance varies dramatically across databases.

Conclusion

The best vector database is the one that matches your operational capabilities and performance requirements, not the one with the most features. For most developers building AI applications, start with Pinecone or Qdrant Cloud to validate that vector search delivers user value. Once you've proven the concept and understand your query patterns, you can optimize for cost or performance by self-hosting or choosing specialized databases.

Avoid premature optimization. Running Milvus for a prototype with 50K vectors wastes engineering time on operational complexity that doesn't matter yet. Similarly, avoid premature commitment—design abstraction layers that let you migrate between databases as requirements evolve. Your first vector database likely won't be your last, and that's fine. Build for the scale you have, not the scale you hope to reach.