fogserv.cloud fogserv.cloud

Qdrant Setup - Self-Hosted Vector Database

Status: Active
Last Updated: 2026-08-26
Category: AI/ML - Phase 3: RAG Infrastructure
Prerequisites: docker-basics, embeddings-vector-db
Time: 2-3 hours
Tags: qdrant, vector-database, docker, rag, embeddings, filtering

Summary

Deploy Qdrant in Docker on your homelab server, create collections with the right vector configuration, understand distance metrics, filter by payload metadata, and talk to it from Python. Qdrant is the recommended self-hosted vector store for our RAG stack.

🎯 What You'll Learn

By the end of this article, you'll be able to:

Context / Why This Matters

embeddings-vector-db explains what embeddings are; this article stands up the database that stores them. When your document set outgrows brute-force numpy similarity (document-chunking → hundreds of files), you need an ANN index plus metadata filtering — that's Qdrant: a single Rust binary, trivially containerized, with a clean REST + gRPC API and good Python support. It pairs directly with the pipeline described in rag-pipeline.

Implementation / Core Content

Deploy With Docker

mkdir -p /srv/qdrant/storage /srv/qdrant/snapshots

docker run -d \
  --name qdrant \
  --restart unless-stopped \
  -p 127.0.0.1:6333:6333 \
  -p 127.0.0.1:6334:6334 \
  -v /srv/qdrant/storage:/qdrant/storage \
  -v /srv/qdrant/snapshots:/qdrant/snapshots \
  qdrant/qdrant:v1.12.4

Notes:

Enable API key auth via env vars (or a config file mounted at /qdrant/config/config.yaml):

docker rm -f qdrant
docker run -d --name qdrant --restart unless-stopped \
  -p 127.0.0.1:6333:6333 -p 127.0.0.1:6334:6334 \
  -e QDRANT__SERVICE__API_KEY="change-me-long-random" \
  -e QDRANT__SERVICE__READ_ONLY_API_KEY="optional-read-key" \
  -v /srv/qdrant/storage:/qdrant/storage \
  qdrant/qdrant:v1.12.4

Health check:

curl http://localhost:6333/healthz          # -> "healthz"
curl http://localhost:6333/collections      # list collections

Or use compose alongside Ollama on one network — see docker-compose-intro.

Collections and Vector Config

A collection holds points (id + vector + payload) sharing one vector schema. The critical parameter is the dimension, which must match your embedding model exactly:

Embedding model (Ollama) Dim
nomic-embed-text 768
mxbai-embed-large 1024
snowflake-arctic-embed 1024

Mismatched dimension = every insert rejected. Decide the model first (embeddings-vector-db); changing models later means re-embedding the whole corpus into a new collection.

from qdrant_client import QdrantClient, models

client = QdrantClient(host="localhost", port=6333, api_key="change-me-long-random")

if not client.collection_exists("docs"):
    client.create_collection(
        collection_name="docs",
        vectors_config=models.VectorParams(
            size=768,                                  # match embedding model!
            distance=models.Distance.COSINE,
        ),
    )

For most homelab corpora (up to a few million points) defaults are fine; Qdrant builds HNSW indexes automatically. Named vectors (vectors_config={...} dict form) allow multiple vector spaces per point — skip until you need hybrid dense+sparse search.

Distance Metrics

Metric Use when
Cosine Default choice. Ignores magnitude, compares direction. Matches normalized embedding models (nomic, bge, etc.)
Dot Equivalent to cosine for already-normalized vectors, slightly faster
Euclid Spatial/embedding models trained for L2 (some image encoders); rare for text

Rule: use cosine unless your embedding model card explicitly says otherwise. Consistency matters more than the specific pick — never mix metrics across a migration.

Points, Payloads, and Filtering

Every point: numeric ID, vector, and arbitrary JSON payload. Payload design is what makes filtered retrieval possible later:

client.upsert(
    collection_name="docs",
    points=[models.PointStruct(
        id=1,                                    # unique int or UUID
        vector=[0.12, -0.44, ...],               # len == 768
        payload={
            "source": "runbooks/backup-failures.md",
            "chunk_index": 3,
            "category": "ops",
            "tags": ["backup", "restic"],
            "updated": "2026-07-02",
            "text": "If restic exits with code 3, the snapshot was partial...",
        },
    )],
)

Store the chunk text in the payload — you need it back at query time to build the prompt, and it makes the collection self-contained.

Search with filters (payload indexes get created automatically on demand):

hits = client.query_points(
    collection_name="docs",
    query=embedding_vector,
    limit=5,
    with_payload=True,
    query_filter=models.Filter(
        must=[
            models.FieldCondition(key="category", match=models.MatchValue(value="ops")),
            models.Range(key="updated", gte="2026-01-01"),
        ],
        must_not=[
            models.FieldCondition(key="tags", match=models.MatchAny(any=["archived"])),
        ],
    ),
).points

for h in hits:
    print(round(h.score, 3), h.payload["source"], h.payload["text"][:80])

Filter semantics: must = AND, should = OR (at least one), must_not = exclusion. Filters narrow the candidate set before ranking — far better than retrieving then discarding in Python.

Client Options

Ops Basics

# snapshot a collection (writes into the snapshots volume)
curl -X POST http://localhost:6333/collections/docs/snapshots

# collection info: count, status, indexed vectors
curl http://localhost:6333/collections/docs

Include /srv/qdrant/storage in normal filesystem backups too, or schedule snapshots and back those up (restic-backups pattern applies).

Practical Examples

Example 1: End-to-End Mini Pipeline (Embed + Search)

"""mini_rag.py — index markdown notes, search them."""
import glob, ollama
from qdrant_client import QdrantClient, models

client = QdrantClient(host="localhost", port=6333)
MODEL, DIM = "nomic-embed-text", 768

def embed(texts: list[str]) -> list[list[float]]:
    return ollama.embed(model=MODEL, input=texts)["embeddings"]

client.recreate_collection("notes",
    vectors_config=models.VectorParams(size=DIM, distance=models.Distance.COSINE))

points = []
for pid, path in enumerate(sorted(glob.glob("notes/*.md"))):
    text = open(path).read()
    points.append(models.PointStruct(
        id=pid, vector=embed([text])[0],
        payload={"source": path, "text": text}))
client.upsert("notes", points)

qvec = embed(["how do I restore a restic snapshot?"])[0]
for h in client.query_points("notes", query=qvec, limit=3, with_payload=True).points:
    print(f"{h.score:.3f} {h.payload['source']}")

Expected: the backup-related note scores highest (cosine similarity ~0.6–0.8), unrelated notes ~0.2–0.4.

Example 2: Metadata-Restricted Search

Only ops-tagged documents updated this year:

results = client.query_points(
    "docs", query=qvec, limit=5,
    query_filter=models.Filter(must=[
        models.FieldCondition(key="tags", match=models.MatchAny(any=["ops"])),
        models.FieldCondition(key="updated", match=models.MatchText(text="2026")),
    ]),
).points

Example 3: Verify Persistence

docker rm -f qdrant
docker start qdrant           # or re-run the docker run command
curl http://localhost:6333/collections | grep notes   # still there

Troubleshooting & Common Pitfalls

Problem Cause Fix
Insert fails: dimension mismatch Vector size ≠ collection size Match embedding model dim exactly; recreate collection if wrong
Search results irrelevant Wrong distance metric, or mixing embed models between write & query Cosine + same model both directions; re-embed corpus on model change
Data lost after container upgrade No volume mount on storage path Always mount /qdrant/storage; pin image versions
Client can't connect from another container Bound to 127.0.0.1 only Put both containers on a shared Docker network and use container name as host
Slow queries on large collections Payload-heavy unindexed filters Add payload indexes (create_payload_index); use gRPC client
403 responses API key enabled but client missing key Pass api_key= in client constructor
Scores look like tiny floats, not 0–1 Metric semantics differ (Euclid returns distances) Cosine scores are similarities; normalize expectations per metric
Collection won't accept writes: 409 Duplicate point ID with different vector Use deterministic IDs (hash of source+chunk_index) for idempotent re-indexing

Next Steps / Ops Actions

Sources & Related

External references consulted:

Related knowledge-base articles:

Change Log

2026-08-26

Choose Theme

Your selection is saved locally.

Neural Cacophony
Aperture v2
Flux v1
Mosaic Chaos
Nexus v1
Nexus Zest
Prism v2
Synapse