About
How the pipeline works and why it is built this way.
What this is
An engineering platform for building, evaluating and benchmarking Retrieval-Augmented Generation systems — not a chatbot. A chatbot answers questions; this answers whether the retrieval behind those answers is any good, and how you know.
Pipeline
- 1EmbeddingBAAI/bge-small-en-v1.5, 384-dim, CPU
- 2DenseFAISS IndexFlatIP, exact inner-product search
- 3SparseBM25 over the same chunks
- 4FusionReciprocal Rank Fusion, k = 60
- 5RerankerCross-encoder ms-marco-MiniLM-L-6-v2 — opt-in per query, off by default
- 6GenerationGroq llama-3.1-8b-instant, temperature 0
Measured quality
Why accuracy metrics are not on the Query page
Precision, Recall and MRR are undefined without ground truth, and a question typed a second ago has none. The Query page therefore shows only what is directly observable — scores, latency, tokens, whether dense and sparse agreed — while accuracy lives on the Evaluation page, over the labelled dataset. Any Precision@5 rendered next to an ad-hoc answer would be fabricated.
Ground truth
Evaluation labels are anchored to content spans, not to chunk IDs. Chunk IDs are a function of chunk size and separators, so re-chunking silently re-points every label at different text — once moving MRR from 1.000 to 0.143 with no error raised anywhere. Each span is now resolved against the live index at evaluation time and must match exactly one chunk, so a stale label fails the build instead of corrupting a number.
Built with
- Frontend
- Next.js 16 · React 19 · TanStack Query · Tailwind
- API
- FastAPI · Pydantic · SSE streaming
- Retrieval
- FAISS · BM25 (rank-bm25) · Reciprocal Rank Fusion
- Models
- bge-small-en-v1.5 · ms-marco-MiniLM-L-6 · llama-3.1-8b
- Storage
- SQLite (documents) · numpy vectors · JSON metadata
- Worker
- In-process thread, or Redis when configured
No LangChain and no framework abstractions: every stage above is written directly, which is what makes the trace on the Query page able to report what each one actually did.