Embeddings and vector stores

Retrieval augmented generation (RAG) gives a model the facts it needs at question time. Your documents are split into chunks, each chunk is turned into an embedding (a list of numbers that captures its meaning), and the embeddings go into a vector store. A question is embedded the same way, the store returns the nearest chunks, and those go into the prompt. The model answers from your content instead of from memory.

An embedding model makes the vectors. Amazon Titan Text Embeddings V2 is AWS's own, and lets you choose the vector size (1,024, 512 or 256 dimensions); smaller vectors cost less to store and search and lose a little accuracy. Cohere's embedding models are also on Bedrock. The rule that matters most: the question and the documents must be embedded with the same model and settings. Change the model and you must re-embed everything.

Where to keep the vectors. Amazon Bedrock Knowledge Bases is the managed option: point it at a data source (such as an S3 bucket), choose an embedding model and a vector store, and it chunks, embeds, stores and retrieves for you. For the store itself, Amazon OpenSearch Service (managed clusters or Serverless) gives vector search with keyword search beside it; Amazon Aurora PostgreSQL with the pgvector extension suits teams that already run Postgres and want vectors next to their tables; Amazon Neptune Analytics adds graph relationships; Amazon S3 Vectors stores vectors cheaply in S3. Check the current list of stores Knowledge Bases supports.

Metadata makes retrieval precise. Attach fields like department, product, language, access level and date to each document (in Knowledge Bases, a metadata file beside each S3 object). At query time a filter such as department = "HR" AND year >= 2025 narrows the search before similarity ranks what's left. It's also how you keep users from retrieving documents they shouldn't see.

At scale, OpenSearch performance depends on its design: enough shards to spread the index, the right vector engine and index type, and sometimes separate indexes for very different domains (legal and engineering documents, say) so each search is smaller and more relevant.

A vector store goes stale as documents change. Knowledge Bases syncs a data source with an ingestion job, which handles new, changed and deleted files; schedule it (for example with Amazon EventBridge) or start it when S3 reports a change. Connectors for sources like Confluence, SharePoint, Salesforce and web pages, and Amazon Kendra or Amazon Q Business for enterprise search, bring in content that isn't in S3.