Infrastructure

Vector Database

A database optimised for storing and querying high-dimensional embedding vectors via approximate nearest-neighbour search.

Definition

Vector databases store embedding vectors (numerical representations of text, images, audio) and enable fast similarity search across billions of vectors. Traditional databases handle structured data with exact matches; vector DBs find the k most similar vectors using approximate nearest-neighbour (ANN) algorithms like HNSW, IVF, and LSH.

They are foundational for RAG systems, semantic search, recommendation engines, duplicate detection, and any application requiring "find items semantically similar to this query." Key metrics are recall (fraction of true nearest neighbours found) and queries per second at a given recall.

The market leader is Pinecone (purpose-built, fully managed); open-source alternatives include Weaviate, Qdrant, Milvus, and Chroma. PostgreSQL extensions (pgvector) add vector search to existing databases. The vector DB market grew rapidly with the LLM boom from 2022.

Examples

  • Pinecone
  • Weaviate
  • Qdrant
  • Chroma
  • pgvector (PostgreSQL extension)