practice-rag
The full RAG loop built by hand and run entirely on local hardware: Ollama embeddings into a persistent Chroma store over the IMDB top 100, retrieval into a LangChain prompt, generation by a local llama3.2. No API keys anywhere.
// I had built RAG search at work on a hosted model. I wanted to rebuild the whole loop myself with a local LLM, local embeddings, and a vector store, to understand each step.
The brief
A deliberately small repo with one job: build the canonical
retrieval-augmented-generation pipeline from primitives, with no hosted API at
any step. Each movie in an IMDB top-100 CSV becomes a LangChain document,
embedded with Ollama’s mxbai-embed-large into a persistent Chroma
collection. A REPL then answers movie questions: the retriever pulls the five
most similar films and a prompt-plus-model chain grounds llama3.2’s answer
in them.
What it demonstrates
- The RAG loop with nothing hidden: embed, store, retrieve, ground, generate, each step visible in about a page of code.
- A fully local model stack. Both embeddings and generation run through Ollama on the machine, which changes the economics and privacy story compared to hosted APIs.
- Index-once discipline: the store persists to disk and skips re-embedding on subsequent runs.
The honest bit
This is a practice repo and says so in its README. It even ships a small bug (a typo means movie metadata never attaches to the documents), which is the kind of thing you find when you read code instead of README claims. The production-scale version of this idea is the RAG knowledge base I built at IBM for a 70-person team; this repo is the fundamentals, owned end to end.