Embeddings & Vector Databases - Semantic Search with Ollama and Qdrant

Status: Active
Last Updated: 2026-08-26
Category: AI/ML - Phase 3: Vector Databases
Prerequisites: ollama-setup, ollama-api, docker-compose-patterns
Time: 1-2 hours
Tags: embeddings, vectors, qdrant, semantic-search, ollama, nomic-embed-text, cosine-distance, docker-compose

Summary

Embeddings turn text into long arrays of numbers that represent semantic meaning, so "how do I reset my password?" and "I forgot my login credentials" land close together in vector space even though they share almost no words. This guide explains what embeddings and cosine distance actually are, pulls a local embedding model with Ollama (nomic-embed-text), runs Qdrant via Docker Compose, and builds a working semantic search demo with the Python client — create collection, upsert, query, results.

🎯 What You'll Learn

By the end of this article, you'll be able to:

• ✅ Explain what an embedding is and why similar meanings produce nearby vectors
• ✅ Compute and interpret cosine similarity / cosine distance
• ✅ Pull and use an Ollama embedding model (nomic-embed-text)
• ✅ Run Qdrant locally with Docker Compose (REST :6333, gRPC :6334, Web UI dashboard)
• ✅ Create collections with correct vector dimensions
• ✅ Upsert points and run semantic search from Python
• ✅ Build an end-to-end local semantic search demo over your own notes

---

Table of Contents

1. What Are Embeddings?
2. Similarity: Cosine Distance Explained
3. Generating Embeddings Locally with Ollama
4. Running Qdrant with Docker Compose
5. Collections, Upserts, and Search in Python
6. Semantic Search Demo
7. Troubleshooting & Common Pitfalls
8. Sources & Related
9. Change Log

---

1. What Are Embeddings?

An embedding model is trained specifically to map text into a fixed-length numeric vector where geometric closeness encodes semantic similarity.

"How do I reset my password?"  →  [0.021, -0.114, 0.337, ... ]   # 768 floats
"I forgot my login credentials" →  [0.019, -0.109, 0.342, ... ]  # very close!
"Best pizza toppings 2026"      →  [-0.442, 0.087, 0.011, ...]   # far away

Key properties:

Property	Meaning
Dimensionality	Fixed length per model. nomic-embed-text outputs 768 dims; mxbai-embed-large outputs 1024; all-minilm outputs 384
Context window	Max input tokens per call. nomic-embed-text supports ~2K tokens — chunk longer documents first
One model, one space	You must embed queries and documents with the same model. Vectors from different models live in incompatible spaces

Why this matters for search: keyword search fails on paraphrases ("car repair" vs "auto fix"), while embedding search matches on meaning. This is the foundation for RAG (Phase 4).

2. Similarity: Cosine Distance Explained

The standard metric is cosine similarity: the cosine of the angle between two vectors, ignoring their magnitude.

cos(θ) = (A · B) / (‖A‖ × ‖B‖)

+1.0  → identical direction (same meaning)
 0.0  → orthogonal (unrelated)
-1.0  → opposite directions

Most vector databases (including Qdrant) report cosine distance = 1 − cosine similarity, so:

• distance ≈ 0.0–0.3 → strong match
• distance ≈ 0.4–0.6 → plausibly related
• distance > 0.7 → probably unrelated (tune thresholds per model empirically)

Rule of thumb: don't hardcode a threshold from a blog post — print scores on your own data, look at the gap between good and bad matches, then set a cutoff.

3. Generating Embeddings Locally with Ollama

Pull an embedding model:

ollama pull nomic-embed-text   # 274MB, 137M params, 2K context, 768 dims

Other good options (per Ollama's embedding models docs): mxbai-embed-large (334M params) for higher quality, all-minilm (23M) when resources are tight.

REST API

curl http://localhost:11434/api/embed -d '{
  "model": "nomic-embed-text",
  "input": "Llamas are members of the camelid family"
}'
# {"model":"nomic-embed-text","embeddings":[[0.010,-0.049,...]],...}

Note: the modern endpoint is /api/embed (batch-friendly). The older /api/embeddings takes "prompt" instead of "input" and returns one vector at a time — prefer /api/embed.

Python library

pip install ollama

import ollama

resp = ollama.embed(model="nomic-embed-text", input="Llamas are members of the camelid family")
vector = resp["embeddings"][0]
print(len(vector))  # 768

# Batch multiple inputs in one call:
resp = ollama.embed(model="nomic-embed-text",
                    input=["First doc", "Second doc", "Third doc"])
vectors = resp["embeddings"]  # list of 3 x 768

4. Running Qdrant with Docker Compose

Qdrant exposes REST on port 6333 (plus a Web UI at /dashboard) and gRPC on 6334. The quickstart runs it with docker run, but for a persistent home-lab service, Docker Compose is cleaner (see docker-compose-patterns):

# docker-compose.yml
services:
  qdrant:
    image: qdrant/qdrant:latest
    container_name: qdrant
    ports:
      - "6333:6333"   # REST API + Web UI dashboard
      - "6334:6334"   # gRPC API
    volumes:
      - ./qdrant_storage:/qdrant/storage
    restart: unless-stopped
    healthcheck:
      test: ["CMD-SHELL", "bash -c ':> /dev/tcp/127.0.0.1/6333' || exit 1"]
      interval: 30s
      timeout: 5s
      retries: 3

On Windows/WSL, prefer a named volume over a bind mount (qdrant_data:/qdrant/storage with a top-level volumes: entry) to avoid filesystem permission issues.

Bring it up and verify:

docker compose up -d
curl http://localhost:6333/collections          # should return {"collections":[...]}
open http://localhost:6333/dashboard            # built-in Web UI

5. Collections, Upserts, and Search in Python

pip install qdrant-client

Create a collection

The vector size must match your embedding model's output dimension (768 for nomic-embed-text). Distance COSINE stores normalized-friendly vectors:

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams

client = QdrantClient(url="http://localhost:6333")

if not client.collection_exists("notes"):
    client.create_collection(
        collection_name="notes",
        vectors_config=VectorParams(size=768, distance=Distance.COSINE),
    )

Upsert points

Each point needs a unique id, the vector, and an arbitrary JSON payload (source file, title, etc.) returned alongside search hits:

import uuid

client.upsert(
    collection_name="notes",
    points=[{
        "id": str(uuid.uuid4()),
        "vector": vector,
        "payload": {"text": "Reset passwords from the admin panel", "source": "admin.md"},
    }],
)

Search

hits = client.query_points(
    collection_name="notes",
    query=query_vector,
    limit=5,
).points

for hit in hits:
    print(round(hit.score, 4), hit.payload["text"])
    # score here is similarity (higher = better) for COSINE

6. Semantic Search Demo

Complete working demo — index a few sentences and query by meaning:

"""semantic_search.py — minimal Ollama + Qdrant semantic search."""
import ollama
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams

MODEL = "nomic-embed-text"

docs = [
    {"id": 1, "text": "Reset your password from the account settings page."},
    {"id": 2, "text": "API keys are managed under developer options."},
    {"id": 3, "text": "Our office serves pizza every Friday."},
]

client = QdrantClient(url="http://localhost:6333")
client.recreate_collection(
    collection_name="demo",
    vectors_config=VectorParams(size=768, distance=Distance.COSINE),
)

resp = ollama.embed(model=MODEL, input=[d["text"] for d in docs])
client.upsert(collection_name="demo", points=[
    {"id": d["id"], "vector": v, "payload": {"text": d["text"]}}
    for d, v in zip(docs, resp["embeddings"])
])

query = "how do I change my login credentials?"
qv = ollama.embed(model=MODEL, input=query)["embeddings"][0]

hits = client.query_points(collection_name="demo", query=qv, limit=3).points
print(f"Query: {query}\n")
for h in hits:
    print(f"  {h.score:.4f}  {h.payload['text']}")

Expected output shape (scores vary slightly by version):

Query: how do I change my login credentials?

  0.62  Reset your password from the account settings page.
  0.48  API keys are managed under developer options.
  0.11  Our office serves pizza every Friday.

Note the magic: the query shares zero keywords with the top result, yet ranks it first because the meaning is close. That's the retrieval primitive RAG is built on.

Troubleshooting & Common Pitfalls

Click to expand

Problem	Cause	Fix
Validation error: vector size mismatch	Collection dim ≠ model dim	Recreate collection with correct size (768 for nomic-embed-text) or switch models consistently
Garbage search results	Query embedded with a different model than docs	Use one model everywhere; record the model name in collection metadata/payload
Connection refused to :6333	Qdrant not running or wrong port mapping	docker compose ps; check curl localhost:6333/collections
Truncated/degraded embeddings	Input exceeded the model context (~2K tokens for nomic-embed-text)	Chunk documents before embedding — see rag-pipeline
Using deprecated /api/embeddings	Old endpoint returns single vector via "prompt"	Migrate to /api/embed with "input" (supports batches)
Bind-mount permission errors on Windows/WSL	Host FS incompatibility	Use a named Docker volume instead of a bind mount
Everything scores ~0.5	Very short texts embed weakly	Give chunks more context (title + body); try mxbai-embed-large

Sources & Related

Web sources consulted during research:

• https://qdrant.tech/documentation/quickstart/
• https://ollama.com/blog/embedding-models
• https://ollama.com/library/nomic-embed-text

Related knowledge-base articles:

• model-selection-guide — picking chat vs embedding models
• ollama-api — full REST/Python API reference
• rag-pipeline — build on these primitives into full RAG
• docker-compose-patterns — service patterns used above

Change Log

• 2026-08-26: Initial draft created via headless-browser web research session.