# First Aid QA RAG Assistant

A Retrieval-Augmented Generation (RAG) assistant that answers first-aid questions in real time. A person at an accident scene can ask a natural-language question and receive an accurate, context-grounded answer drawn from a curated first-aid FAQ dataset.

The system combines **hybrid search** (BM25 text search + HNSW vector search with Reciprocal Rank Fusion) with an LLM to generate answers, automatically **rewrites queries** to improve retrieval, and records every conversation in PostgreSQL for monitoring and feedback collection.

---

## Problem

People at accident scenes often need fast, reliable first-aid guidance but cannot quickly search through documentation. This assistant provides a conversational interface where any question — however informally phrased — is reformulated for precision, searched against a structured first-aid knowledge base, and answered using an LLM grounded in retrieved context.

---

## Architecture

```
User query
  │
  ▼
Query rewriting (LLM)
  │
  ▼
Hybrid search ──── BM25 text search (SQLite FTS)
  │            └── HNSW vector search (ONNX embeddings)
  │                      ↕ Reciprocal Rank Fusion
  ▼
Prompt building + LLM answer (Mistral ministral-3b via LiteLLM)
  │
  ├── Save conversation → PostgreSQL
  ├── Auto-judge relevance (LLM-as-judge) → PostgreSQL feedback
  └── OpenTelemetry traces → SQLite
```

**Stack:** Python 3.12 · Streamlit · LiteLLM · Mistral AI · SQLite (search + traces) · PostgreSQL (conversations + feedback) · ONNX Runtime · OpenTelemetry

---

## Project structure

```
first-aid-qa-rag-assistant/
├── src/
│   ├── app.py              # Streamlit UI entry point
│   ├── assistant.py        # RAG assistant factory
│   ├── config.py           # Model name, LLM client factory, cost pricing
│   ├── embedder.py         # ONNX sentence embedder (all-MiniLM-L6-v2)
│   ├── ingest.py           # Load FAQ data, build text + vector indices
│   ├── judge.py            # LLM-as-judge relevance scoring
│   ├── search.py           # text_search, vector_search, RRF fusion
│   ├── rag/                # RAG class hierarchy
│   │   ├── rag_helper.py   # RAGBase, prompt templates, query rewriting
│   │   ├── rag_usage.py    # RAGWithUsage (hybrid search + usage tracking)
│   │   ├── metrics.py      # RAGWithMetrics (cost + latency recording)
│   │   └── starter.py      # RAGTraced (OTel) + RAGProduction (full stack)
│   ├── db/                 # PostgreSQL persistence
│   │   ├── init.py         # Schema creation
│   │   ├── conversations.py
│   │   └── feedback.py
│   ├── evaluation/         # Retrieval + answer quality evaluation
│   │   ├── evaluation.py
│   │   └── evaluation_utils.py
│   └── telemetry/          # OpenTelemetry tracing → SQLite
│       ├── tracer.py
│       └── exporter.py
├── notebooks/              # Jupyter experiment notebooks
├── scripts/
│   └── download.py         # Download ONNX model from Hugging Face
├── data/
│   └── firstaidqa_v1-first-half.json   # 2,775 first-aid Q&A pairs
├── models/                 # ONNX model files (not in git — see setup below)
├── sqlite/                 # SQLite DB files (not in git — see setup below)
├── Dockerfile
├── docker-compose.yml
└── pyproject.toml
```

---

## Prerequisites

| Tool | Version | Install |
|---|---|---|
| Python | 3.12 | [python.org](https://www.python.org/downloads/) |
| uv | ≥ 0.11 | `curl -LsSf https://astral.sh/uv/install.sh \| sh` |
| Docker + Docker Compose | any recent | [docs.docker.com](https://docs.docker.com/get-docker/) |
| Mistral AI API key | — | [console.mistral.ai](https://console.mistral.ai/) |

---

## Dependency versions

All dependencies are fully pinned in `uv.lock`. Key packages:

| Package | Version |
|---|---|
| litellm | 1.92.0 |
| streamlit | 1.59.2 |
| numpy | 2.5.1 |
| onnxruntime | 1.27.0 |
| opentelemetry-api | 1.44.0 |
| opentelemetry-sdk | 1.44.0 |
| pandas | 3.0.3 |
| psycopg | 3.3.4 |
| sqlitesearch | 0.1.2 |
| tokenizers | 0.23.1 |
| tqdm | 4.68.4 |

---

## Quick start — local (without Docker)

### 1. Clone and install

```bash
git clone https://gitfront.io/r/manishgitfront/q9AfQg84m4bX/first-aid-rag-assistant.git
cd first-aid-qa-rag-assistant
uv sync          # creates .venv and installs all pinned dependencies
```

### 2. Configure environment

Copy the example env file and fill in your values:

```bash
cp .env.example .env
```

```dotenv
# .env
MISTRAL_API_KEY=your_mistral_api_key_here

POSTGRES_HOST=localhost
POSTGRES_DB=first_aid_qa_assistant
POSTGRES_USER=user
POSTGRES_PASSWORD=password
```

### 3. Download the ONNX embedding model

The `models/` directory is excluded from git (binary files ~90 MB). Run once after cloning:

```bash
uv run python scripts/download.py
```

This downloads `all-MiniLM-L6-v2` (ONNX) from Hugging Face into `models/Xenova/all-MiniLM-L6-v2/`.

> **Offline / air-gapped:** place `model.onnx` and `tokenizer.json` manually into `models/Xenova/all-MiniLM-L6-v2/`.

### 4. Create SQLite index directories

The `sqlite/` directory is excluded from git. Create the required subdirectories once:

```bash
mkdir -p sqlite/text sqlite/vector sqlite/otel
```

The text and vector search indices are built automatically on first run of the application.

### 5. Start PostgreSQL

Using Docker for just the database:

```bash
docker compose up postgres -d
```

Or use any PostgreSQL 15+ instance and update `.env` accordingly.

### 6. Initialise the database schema

Run once after starting PostgreSQL:

```bash
uv run python src/db/init.py
```

This creates the `conversations` and `feedback` tables (safe to re-run — uses `IF NOT EXISTS`).

### 7. Run the app

```bash
uv run streamlit run src/app.py
```

Open [http://localhost:8501](http://localhost:8501) in your browser.

---

## Quick start — Docker Compose (full stack)

### 1. Configure environment

```bash
cp .env.example .env
# Edit .env with your MISTRAL_API_KEY and Postgres credentials
```

### 2. Download the ONNX model

The model must be present before building the image:

```bash
uv sync --group dev
uv run python scripts/download.py
```

### 3. Create SQLite directories

```bash
mkdir -p sqlite/text sqlite/vector sqlite/otel
```

### 4. Initialise the database

Start Postgres first, then initialise the schema:

```bash
docker compose up postgres -d
# Wait ~5 seconds for Postgres to be ready, then:
uv run python src/db/init.py
```

### 5. Build and start everything

```bash
docker compose up --build
```

Open [http://localhost:8501](http://localhost:8501).

---

## Dataset

**File:** `data/firstaidqa_v1-first-half.json`
**Size:** 2,775 first-aid Q&A pairs
**Format:** JSON array of `{"id": str, "question": str, "answer": str}`
**Source:** Included in the repository — no external download required.

The full dataset (`data/firstaidqa_v1.json`) and clinical evaluation set (`data/clinical_eval_set.csv`) are also included for notebook experiments.

---

## Notebooks

All experiment notebooks are in `notebooks/`. Each notebook adds the project root to `sys.path` via `import path_setup` so they work from the `notebooks/` directory without installation.

| Notebook | Purpose |
|---|---|
| `01-data-gen.ipynb` | Generate ground-truth Q&A pairs |
| `02-sqlite-text-search-evals.ipynb` | BM25 retrieval evaluation |
| `03-clinical-data-gen.ipynb` | Clinical dataset generation |
| `04-embedding-model-evals.ipynb` | Embedding model comparison |
| `05-sqlite-vector-search-mode-evals.ipynb` | Vector index mode comparison |
| `06-sqlite-vector-num-of-results-evals.ipynb` | Vector search top-k tuning |
| `07-sqlite-text-search-num-of-results-evals.ipynb` | Text search top-k tuning |
| `08-sqlite-hybrid-search-evals.ipynb` | Hybrid search (RRF) evaluation |
| `09-llm-as-judge.ipynb` | LLM-as-judge answer quality evaluation |
| `010-agent-evals.ipynb` | Agent evaluation |
| `010-monitoring.ipynb` | Monitoring dashboard from OTel traces |
| `pre-processing.ipynb` | Dataset pre-processing |

---

## Environment variables reference

| Variable | Required | Default | Description |
|---|---|---|---|
| `MISTRAL_API_KEY` | **Yes** | — | Mistral AI API key |
| `POSTGRES_HOST` | No | `localhost` | PostgreSQL host |
| `POSTGRES_DB` | No | `first_aid_qa_assistant` | Database name |
| `POSTGRES_USER` | No | `user` | Database user |
| `POSTGRES_PASSWORD` | No | `password` | Database password |

---

## Setup checklist

After cloning, run these steps in order:

```
[ ] uv sync
[ ] cp .env.example .env  (fill in MISTRAL_API_KEY)
[ ] uv run python scripts/download.py        # download ONNX model (~90 MB)
[ ] mkdir -p sqlite/text sqlite/vector sqlite/otel
[ ] docker compose up postgres -d            # or start your own Postgres
[ ] uv run python src/db/init.py             # create DB tables
[ ] uv run streamlit run src/app.py          # start the app
```