Local embedding models: nomic-embed-text and friends

An embedding model turns text into a list of numbers that captures its meaning. It's the engine behind "chat with your documents" and any search that finds results by meaning instead of exact words. For local use, start with nomic-embed-text in Ollama. It's small, fast, and good enough for most personal and small-team projects.

What embeddings are for

A chat model writes text. An embedding model only reads it. You give it a sentence or a paragraph and it returns a vector, a few hundred to a few thousand numbers. Texts with similar meanings get similar vectors, so "how do I reset my password" lands near "forgot login credentials", even with no words in common.

That's what makes these work:

• Document chat (RAG). Your files are split into chunks and each chunk is embedded once. When you ask a question, it's embedded too, the closest chunks are found, and they're handed to a chat model to answer from. See chat with your documents.
• Semantic search over notes, tickets or email.
• Grouping and de-duplicating similar items.
• Classifying text by comparing it to examples.

Run one with Ollama

Embedding models are tiny next to chat models, typically a few hundred megabytes up to a couple of gigabytes. They run fine on a CPU and fly on any GPU.

You don't chat with it. Your tools call it through Ollama's API. To see what it returns:

You'll get back a long list of numbers. Open WebUI, Continue, many note apps and most RAG libraries can be pointed at Ollama to use it. See the Ollama API guide.

Choosing an embedding model

Ollama's library has several embedding models, and Hugging Face has many more. What to compare:

nomic-embed-text is a good default for English. If your documents are in several languages or results seem off, try a multilingual model and compare on your own questions.

Rules that save headaches

Use the same model for indexing and searching. Vectors from different models aren't comparable. If you switch embedding models, re-embed everything.

Some models want a prefix. A few embedding models expect a short label in front of the text, one for documents and another for queries. It's on the model card, and skipping it quietly hurts results. See how to read a model card.

Chunking matters more than the model. Chunks that are too big blur several topics together; too small and they lose context. A few hundred words with some overlap is a common starting point. See local RAG that actually works.

Embeddings find; the chat model answers. If answers are wrong but the right chunks were found, upgrade the chat model. If the right chunks weren't found, fix the embedding side.

Pairing it with a chat model

The embedding model barely needs memory, so the chat model sets your hardware. On a 12 GB card, the embedding model plus Qwen3 14B fits comfortably. On 8 GB, pair it with Qwen3 8B. Here's what a 12 GB RTX 3060 runs, from our calculator:

When embedding speed matters

For a few hundred documents, indexing takes minutes on almost anything. For tens of thousands, a GPU helps a lot, and the first index is the slow part; after that you only embed new files and each question. If you're indexing a big archive, run it overnight.

Ready to build it? Start with chat with your documents, then check every GPU for the chat model your card can run.