Embeddings and Vector Search: Finding Semantic Similarity in AI

An embedding is a numerical representation of text—think of it as a fingerprint that captures meaning. When you feed a sentence into an embedding model, it converts that sentence into a list of hundreds or thousands of numbers. Sentences with similar meanings produce numerically similar embeddings. This is the foundation for semantic search: matching by meaning, not just keywords.

Here's why this matters: if you're looking for "tips for sleeping better," a keyword search might miss "strategies to improve sleep quality" because the words don't overlap perfectly. But embedding-based search understands that these sentences mean roughly the same thing, because their numerical representations are close together in what's called "vector space." Distance between vectors = difference in meaning.

How Vector Search Works in Practice

Modern AI workflows increasingly use vector databases to organize and retrieve information. Tools like NotebookLM automatically embed your source documents and use vector search to find relevant passages when you ask questions. Cursor uses embeddings to understand code context and suggest relevant lines. The process: (1) documents are embedded and stored in a database, (2) your query gets embedded in the same way, (3) the system finds the closest vectors mathematically, (4) those documents are fed to the language model as context.

This is why RAG (Retrieval-Augmented Generation) systems work so well. Instead of relying solely on what the model memorized during training, the system retrieves relevant external documents first, then uses those documents to ground the response. RAG + embeddings = a way to give AI models access to current information or your proprietary data without retraining.

The Embedding Model Matters

Different embedding models capture different kinds of meaning. OpenAI's embedding model is strong for general English. Specialized domains (scientific papers, code, multilingual content) might need different embeddings. Using the wrong embedding model for your domain can degrade search quality significantly. If you're embedding legal documents, a general-purpose embedding might miss domain-specific nuances.

Embedding dimension is another consideration. A 1536-dimensional embedding (OpenAI's standard) captures more nuance than a 384-dimensional one, but uses more storage and compute. Most use cases land on 1024–1536 dimensions as a practical sweet spot.

Common Limitations and Edge Cases

Embeddings capture semantic similarity, not factual correctness. Two sentences can be semantically similar but factually contradictory. A vector search might retrieve high-quality results that the language model then misinterprets. This is why RAG + vector search requires careful prompt engineering around how to handle retrieval results.

Another edge case: embeddings are sensitive to phrasing. "The cat sat on the mat" and "The mat sat under the cat" have very different meanings but might have surprisingly similar embeddings because word-level similarity can mislead the model. Always validate vector search results before feeding them into decision-making processes.

Building Retrieval Workflows

If you're designing a workflow that retrieves documents and synthesizes insights, start with vector search (fast, semantic matching) to narrow the candidate set, then use a language model to extract and reason. This two-stage approach is more efficient and reliable than just dumping all potential documents into a prompt and hoping the model figures out relevance.

NotebookLM is a practical example: it embeds your documents, then when you chat, it retrieves relevant sections semantically and feeds them to the language model. This lets you interact with documents more intelligently than traditional document search.

Try this: In NotebookLM, upload a document and ask a question. Then hover over cited sections—notice how the system found passages that are semantically related to your question, not just keyword matches. This is vector search in action. Try a deliberately oblique question (rephrase the meaning without using key terms) and see whether the semantic retrieval still works.