LangChain Integration - Ollama + Qdrant With LangChain

Status: Active
Last Updated: 2026-08-26
Category: AI/ML - Phase 3: RAG Infrastructure
Prerequisites: ollama-api, qdrant-setup
Time: 3 hours
Tags: langchain, rag, ollama, qdrant, retrievers, abstraction

Summary

Use LangChain to wire document loaders, splitters, Ollama embeddings/LLMs, and a Qdrant vector store into a working RAG pipeline — plus an honest look at where LangChain's abstractions help, where they hurt, and when plain API calls (or LlamaIndex) are the better call.

🎯 What You'll Learn

By the end of this article, you'll be able to:

✅ Build an ingest → embed → store pipeline with LangChain components
✅ Create an Ollama-backed retriever over Qdrant and answer questions with it
✅ Know which LangChain layers to avoid in production code
✅ Decide between LangChain, LlamaIndex, and raw clients for a given task

Context / Why This Matters

Everything so far was explicit: you embedded with ollama.embed (python-llm-integration), chunked by hand (document-chunking), and queried Qdrant directly (qdrant-setup). That's the right way to learn the stack — and often the right way to run it. LangChain packages the same pieces behind standard interfaces (Document, Embeddings, VectorStore, Retriever), trading control and transparency for speed of assembly. This article shows both the assembly and the fine print.

Implementation / Core Content

Install

pip install langchain langchain-community langchain-qdrant langchain-ollama
# note: langchain-core is pulled transitively; the integration moved OUT of
# core into per-provider packages (langchain-ollama, langchain-qdrant).

The package split matters: most old tutorials import from langchain.chat_models or langchain.vectorstores and break on modern versions.

The Four Interfaces You Actually Touch

Interface	Role	Our implementation
`DocumentLoader`	Files/URLs → list[Document] (page_content + metadata)	DirectoryLoader, TextLoader
`TextSplitter`	Documents → chunked Documents	RecursiveCharacterTextSplitter
`Embeddings`	text → vector	`OllamaEmbeddings(model="nomic-embed-text")`
`VectorStore`	store/similarity search of vectors	`QdrantVectorStore`
`ChatOllama`	chat model wrapper	used for generation

Ingestion Pipeline

from langchain_community.document_loaders import DirectoryLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_ollama import OllamaEmbeddings
from langchain_qdrant import QdrantVectorStore
from qdrant_client import QdrantClient

docs = DirectoryLoader("./kb-notes", glob="**/*.md", show_progress=True)
splitter = RecursiveCharacterTextSplitter(
    chunk_size=1200,
    chunk_overlap=150,          # ~10-15%: see document-chunking.md
)
chunks = splitter.split_documents(docs.load())
for c in chunks:
    c.metadata["source"] = c.metadata.get("source", "unknown")

client = QdrantClient(host="localhost", port=6333)

vs = QdrantVectorStore.from_documents(
    chunks,
    embedding=OllamaEmbeddings(model="nomic-embed-text"),  # dim must match collection if reusing one
    collection_name="kb",
    client=client,
)

LangChain auto-creates the collection with the model's dimension if it doesn't exist. If you point at an existing collection, dimension and distance must match what qdrant-setup created (768/cosine for nomic-embed-text).

Retrieval + Generation Chain

from langchain_ollama import ChatOllama
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableParallel, RunnablePassthrough

retriever = vs.as_retriever(search_kwargs={"k": 4})

prompt = ChatPromptTemplate.from_template(
    "Answer using ONLY the context below. If it's insufficient, say so.\n\n"
    "<context>\n{context}\n</context>\n\nQuestion: {question}"
)

def format_docs(docs):
    return "\n---\n".join(d.page_content for d in docs)

rag = (
    RunnableParallel(context=retriever | format_docs, question=RunnablePassthrough())
    | prompt
    | ChatOllama(model="llama3.1:8b", temperature=0, num_ctx=8192)
    | StrOutputParser()
)

print(rag.invoke("How do I restore a restic snapshot?"))

This is LCEL (LangChain Expression Language): components compose with |, and the result is a runnable with .invoke(), .stream(), and .batch() for free. Streaming works without extra code: for tok in rag.stream("...").

Add retrieval filtering via a configurable retriever or by constructing the underlying search_kwargs with Qdrant filters:

from qdrant_client import models
vs.as_retriever(search_kwargs={"k": 4, "filter": models.Filter(must=[
    models.FieldCondition(key="doc_type", match=models.MatchValue(value="runbook")),
])})

The Abstraction Tax — Read Before Adopting

Where LangChain genuinely helps:

Swappable providers: change Ollama→cloud or Chroma→Qdrant with one class.
Standard loaders for PDFs/HTML/notion etc., saving parser boilerplate.
Free streaming/batching semantics on composed chains.
Fast prototyping of multi-step flows before you know what you need.

Where it hurts, learned repeatedly in practice:

Opaque prompts. Prebuilt chains ("load_qa_chain", legacy RetrievalQA) hide the exact prompt sent to the model. When output quality dips you're debugging a string you never wrote. Prefer explicit ChatPromptTemplate chains like above; avoid deprecated loader-style chains entirely.
Deep nesting. One line of LCEL can span five classes across three packages; tracebacks and docs navigation get hard. Keep chains shallow — retrieve, format, prompt, generate.
Version churn. Integration packages split out of core repeatedly; tutorials rot within months. Pin versions in requirements and read the changelog before upgrading.
Silent defaults. Splitters default to sizes that don't suit your corpus; retrievers default k=4 with no filter. Every default relevant to quality should be set explicitly (chunk_size, k, temperature, num_ctx).

Debugging tip: insert a print runnable anywhere in the chain to see intermediate values:

from langchain_core.runnables import RunnableLambda
debug = RunnableLambda(lambda x: (print(x), x)[1])
rag_with_debug = (RunnableParallel(...) | debug | prompt | ...)

Rule of thumb adopted for our stack: ingestion may use LangChain loaders/splitters; the serving path stays close-to-metal — direct qdrant-client queries plus hand-built prompts (prompt-engineering) — unless composition complexity genuinely demands a framework.

LlamaIndex Alternative

If your workload is mostly "index documents, ask questions", LlamaIndex is worth evaluating alongside: its ingestion and index/retriever abstractions are more RAG-centric and arguably cleaner, at the cost of being less general-purpose than LangChain. Both integrate with Ollama and Qdrant through dedicated provider packages (llama-index-llms-ollama, llama-index-vector-stores-qdrant). For our homelab KB use-case either works; pick one and stay consistent rather than mixing frameworks.

Practical Examples

Example 1: Incremental Re-Ingest Script

"""reingest.py — upsert changed files without wiping the collection."""
import hashlib
from langchain_community.document_loaders import TextLoader

def stable_id(doc):
    h = hashlib.sha1(f"{doc.metadata['source']}:{doc.page_content}".encode())
    return h.hexdigest()

new_chunks = splitter.split_documents(TextLoader("kb-notes/new-page.md").load())
ids = [stable_id(c) for c in new_chunks]
vs.add_documents(new_chunks, ids=ids)     # same id+content => no duplicate effect

Deterministic IDs make reruns idempotent — same lesson as raw-Qdrant ingestion in document-chunking.

Example 2: Streaming Answer With Sources

sources = set()
context_docs = retriever.invoke("how do I rotate letsencrypt certs?")
for d in context_docs:
    sources.add(d.metadata["source"])

answer = ""
for token in rag.stream("how do I rotate letsencrypt certs?"):
    print(token, end="", flush=True); answer += token
print("\nSources:", *sorted(sources), sep="\n  - ")

Always surface sources to users — unattributed RAG answers erode trust fast.

Example 3: Verify What the Model Actually Saw

chain_to_prompt = RunnableParallel(context=retriever | format_docs,
                                   question=RunnablePassthrough()) | prompt
rendered = chain_to_prompt.invoke("question here")
print(rendered.to_string())      # inspect the literal prompt before blaming the model

Troubleshooting & Common Pitfalls

Problem	Cause	Fix
`ImportError: ... not found in langchain`	Integrations moved to provider packages	Import from `langchain_ollama` / `langchain_qdrant`; install them
Dimension mismatch creating store	Existing collection has different dim/distance	Match nomic-embed-text = 768 cosine; or let LangChain create a fresh collection
Retrieval returns junk despite good data	Default k / no filters / bad chunk size	Set `search_kwargs={"k": ..., "filter": ...}` explicitly
Answers ignore retrieved context	Prompt doesn't instruct grounding	Use explicit template with delimiters ("answer ONLY from context")
Everything works locally, breaks after upgrade	Unpinned langchain versions	Pin exact versions; upgrade deliberately with tests
Slow first query	Cold Ollama model load	Prewarm model at service start; see python-llm-integration
Duplicates after every re-ingest	Random IDs on add_documents	Deterministic IDs from source+content hash
Hard-to-debug empty answers	Hidden chain state	Insert print `RunnableLambda`s; render the final prompt

Next Steps / Ops Actions

Improve what goes into the retriever: document-chunking
Tune the generation side of the chain: prompt-engineering
Go beyond single-shot retrieval with tool-using loops: agent-frameworks
Run the whole stack as services under compose/k8s: docker-compose-intro

Sources & Related

External references consulted:

Change Log

2026-08-26

Initial creation by KB build session.