First Aid QA RAG Assistant

A Retrieval-Augmented Generation (RAG) assistant that answers first-aid questions in real time. A person at an accident scene can ask a natural-language question and receive an accurate, context-grounded answer drawn from a curated first-aid FAQ dataset.

The system combines hybrid search (BM25 text search + HNSW vector search with Reciprocal Rank Fusion) with an LLM to generate answers, automatically rewrites queries to improve retrieval, and records every conversation in PostgreSQL for monitoring and feedback collection.

Problem

People at accident scenes often need fast, reliable first-aid guidance but cannot quickly search through documentation. This assistant provides a conversational interface where any question — however informally phrased — is reformulated for precision, searched against a structured first-aid knowledge base, and answered using an LLM grounded in retrieved context.

Architecture

User query
  │
  ▼
Query rewriting (LLM)
  │
  ▼
Hybrid search ──── BM25 text search (SQLite FTS)
  │            └── HNSW vector search (ONNX embeddings)
  │                      ↕ Reciprocal Rank Fusion
  ▼
Prompt building + LLM answer (Mistral ministral-3b via LiteLLM)
  │
  ├── Save conversation → PostgreSQL
  ├── Auto-judge relevance (LLM-as-judge) → PostgreSQL feedback
  └── OpenTelemetry traces → SQLite

Stack: Python 3.12 · Streamlit · LiteLLM · Mistral AI · SQLite (search + traces) · PostgreSQL (conversations + feedback) · ONNX Runtime · OpenTelemetry

Project structure

first-aid-qa-rag-assistant/
├── src/
│   ├── app.py              # Streamlit UI entry point
│   ├── assistant.py        # RAG assistant factory
│   ├── config.py           # Model name, LLM client factory, cost pricing
│   ├── embedder.py         # ONNX sentence embedder (all-MiniLM-L6-v2)
│   ├── ingest.py           # Load FAQ data, build text + vector indices
│   ├── judge.py            # LLM-as-judge relevance scoring
│   ├── search.py           # text_search, vector_search, RRF fusion
│   ├── rag/                # RAG class hierarchy
│   │   ├── rag_helper.py   # RAGBase, prompt templates, query rewriting
│   │   ├── rag_usage.py    # RAGWithUsage (hybrid search + usage tracking)
│   │   ├── metrics.py      # RAGWithMetrics (cost + latency recording)
│   │   └── starter.py      # RAGTraced (OTel) + RAGProduction (full stack)
│   ├── db/                 # PostgreSQL persistence
│   │   ├── init.py         # Schema creation
│   │   ├── conversations.py
│   │   └── feedback.py
│   ├── evaluation/         # Retrieval + answer quality evaluation
│   │   ├── evaluation.py
│   │   └── evaluation_utils.py
│   └── telemetry/          # OpenTelemetry tracing → SQLite
│       ├── tracer.py
│       └── exporter.py
├── notebooks/              # Jupyter experiment notebooks
├── scripts/
│   └── download.py         # Download ONNX model from Hugging Face
├── data/
│   └── firstaidqa_v1-first-half.json   # 2,775 first-aid Q&A pairs
├── models/                 # ONNX model files (not in git — see setup below)
├── sqlite/                 # SQLite DB files (not in git — see setup below)
├── Dockerfile
├── docker-compose.yml
└── pyproject.toml

Prerequisites

Tool	Version	Install
Python	3.12	python.org
uv	≥ 0.11	`curl -LsSf https://astral.sh/uv/install.sh \| sh`
Docker + Docker Compose	any recent	docs.docker.com
Mistral AI API key	—	console.mistral.ai

Dependency versions

All dependencies are fully pinned in uv.lock. Key packages:

Package	Version
litellm	1.92.0
streamlit	1.59.2
numpy	2.5.1
onnxruntime	1.27.0
opentelemetry-api	1.44.0
opentelemetry-sdk	1.44.0
pandas	3.0.3
psycopg	3.3.4
sqlitesearch	0.1.2
tokenizers	0.23.1
tqdm	4.68.4

Quick start — local (without Docker)

1. Clone and install

git clone https://gitfront.io/r/manishgitfront/q9AfQg84m4bX/first-aid-rag-assistant.git
cd first-aid-qa-rag-assistant
uv sync          # creates .venv and installs all pinned dependencies

2. Configure environment

Copy the example env file and fill in your values:

cp .env.example .env

# .env
MISTRAL_API_KEY=your_mistral_api_key_here

POSTGRES_HOST=localhost
POSTGRES_DB=first_aid_qa_assistant
POSTGRES_USER=user
POSTGRES_PASSWORD=password

3. Download the ONNX embedding model

The models/ directory is excluded from git (binary files ~90 MB). Run once after cloning:

uv run python scripts/download.py

This downloads all-MiniLM-L6-v2 (ONNX) from Hugging Face into models/Xenova/all-MiniLM-L6-v2/.

Offline / air-gapped: place model.onnx and tokenizer.json manually into models/Xenova/all-MiniLM-L6-v2/.

4. Create SQLite index directories

The sqlite/ directory is excluded from git. Create the required subdirectories once:

mkdir -p sqlite/text sqlite/vector sqlite/otel

The text and vector search indices are built automatically on first run of the application.

5. Start PostgreSQL

Using Docker for just the database:

docker compose up postgres -d

Or use any PostgreSQL 15+ instance and update .env accordingly.

6. Initialise the database schema

Run once after starting PostgreSQL:

uv run python src/db/init.py

This creates the conversations and feedback tables (safe to re-run — uses IF NOT EXISTS).

7. Run the app

uv run streamlit run src/app.py

Open http://localhost:8501 in your browser.

Quick start — Docker Compose (full stack)

1. Configure environment

cp .env.example .env
# Edit .env with your MISTRAL_API_KEY and Postgres credentials

2. Download the ONNX model

The model must be present before building the image:

uv sync --group dev
uv run python scripts/download.py

3. Create SQLite directories

mkdir -p sqlite/text sqlite/vector sqlite/otel

4. Initialise the database

Start Postgres first, then initialise the schema:

docker compose up postgres -d
# Wait ~5 seconds for Postgres to be ready, then:
uv run python src/db/init.py

5. Build and start everything

docker compose up --build

Open http://localhost:8501.

Dataset

File: data/firstaidqa_v1-first-half.json Size: 2,775 first-aid Q&A pairs Format: JSON array of {"id": str, "question": str, "answer": str} Source: Included in the repository — no external download required.

The full dataset (data/firstaidqa_v1.json) and clinical evaluation set (data/clinical_eval_set.csv) are also included for notebook experiments.

Notebooks

All experiment notebooks are in notebooks/. Each notebook adds the project root to sys.path via import path_setup so they work from the notebooks/ directory without installation.

Notebook	Purpose
`01-data-gen.ipynb`	Generate ground-truth Q&A pairs
`02-sqlite-text-search-evals.ipynb`	BM25 retrieval evaluation
`03-clinical-data-gen.ipynb`	Clinical dataset generation
`04-embedding-model-evals.ipynb`	Embedding model comparison
`05-sqlite-vector-search-mode-evals.ipynb`	Vector index mode comparison
`06-sqlite-vector-num-of-results-evals.ipynb`	Vector search top-k tuning
`07-sqlite-text-search-num-of-results-evals.ipynb`	Text search top-k tuning
`08-sqlite-hybrid-search-evals.ipynb`	Hybrid search (RRF) evaluation
`09-llm-as-judge.ipynb`	LLM-as-judge answer quality evaluation
`010-agent-evals.ipynb`	Agent evaluation
`010-monitoring.ipynb`	Monitoring dashboard from OTel traces
`pre-processing.ipynb`	Dataset pre-processing

Environment variables reference

Variable	Required	Default	Description
`MISTRAL_API_KEY`	Yes	—	Mistral AI API key
`POSTGRES_HOST`	No	`localhost`	PostgreSQL host
`POSTGRES_DB`	No	`first_aid_qa_assistant`	Database name
`POSTGRES_USER`	No	`user`	Database user
`POSTGRES_PASSWORD`	No	`password`	Database password

Setup checklist

After cloning, run these steps in order:

[ ] uv sync
[ ] cp .env.example .env  (fill in MISTRAL_API_KEY)
[ ] uv run python scripts/download.py        # download ONNX model (~90 MB)
[ ] mkdir -p sqlite/text sqlite/vector sqlite/otel
[ ] docker compose up postgres -d            # or start your own Postgres
[ ] uv run python src/db/init.py             # create DB tables
[ ] uv run streamlit run src/app.py          # start the app