# WalkXR Game Jam: Adaptive Empathy Engine
## Phase 2.5 — Bridging the Gap Between "Working Walk" and "Emotional OS"

---

## 1. Project Overview

WalkXR Game Jam is an AI-powered empathy training simulator where users practice trauma-informed conversations with a psychologically realistic NPC. The first scenario features **James**, a military veteran with PTSD, at a bus stop.

Built with **LangGraph + GPT-4o-mini + Streamlit**, this prototype goes beyond the original WalkXR Design Document's Phase 2 (a working walk experience) by implementing core elements of Phase 3 (the Emotional OS) — specifically, **real-time adaptive NPC behavior, multi-dimensional emotional tracking, and data-driven coaching feedback**.

---

## 2. Where This Sits: Phase 2 → Phase 3

The WalkXR Master Design Document defines three phases:
- **Phase 1** (E01-E06): Build foundational infrastructure (RAG, BaseAgent, Orchestrator)
- **Phase 2** (E07-E10): Assemble and test a full walk experience
- **Phase 3** (E11-E12+): Generalize into a self-improving Emotional OS platform

**This Game Jam prototype operates at Phase 2.5** — a fully functional walk experience that already incorporates the adaptive intelligence planned for Phase 3.

---

## 3. Key Differentiators from the Original Design Document

### 3.1. Adaptive NPC State Machine (vs. Static Persona)

**Design Doc approach:** NPC agents follow pre-scripted persona descriptions. Behavior is defined once and remains consistent throughout.

**Game Jam approach:** James has a **live internal state** that evolves every turn:

| Metric | Range | What It Tracks |
|--------|-------|----------------|
| `npc_openness` | 0.0 – 1.0 | How willing James is to share personal details |
| `npc_mood` | 5 states | guarded → cautious → warming → open → withdrawn |
| `trust_shift` | -0.3 – +0.3 | Per-turn trust delta based on user's approach |

**Result:** James doesn't just "respond" — he **adapts**. Show genuine empathy and he gradually opens up. Push too hard or use clichés and he withdraws. This creates emergent, non-scripted emotional arcs unique to each session.

### 3.2. Multi-Dimensional Memory System (vs. Simple State Tracking)

**Design Doc approach:** Phase 2 uses basic turn counting and a simple memory dict. The sophisticated Emotional State Engine is deferred to Phase 3 (Epic E05).

**Game Jam approach:** Every turn updates **15 state dimensions simultaneously**:

- **Trust & Rapport:** `confidence_level`, `rapport_score` (composite: trust 40% + openness 30% + empathy ratio 30%)
- **Emotion Tracking:** `emotion_trajectory` (valence + arousal per turn), `dominant_emotion`
- **Behavioral Analysis:** `empathy_hits`, `empathy_misses`, `empathy_ratio` — the LLM classifies each user response as a "hit" (genuine empathy), "miss" (insensitive/cliché), or "neutral"
- **Engagement Detection:** `avg_response_length`, `silence_count` — detects user disengagement
- **NPC Internal State:** `npc_openness`, `npc_mood` — James's emotional evolution
- **Stage Progression:** 4-stage flow (opening → developing → deepening → closing) driven by rapport signals, not just turn count

### 3.3. Intelligent Conversation Routing (vs. Turn-Count Exit)

**Design Doc approach:** Conversations end when max turns are reached or the LLM says to stop.

**Game Jam approach:** 6 exit conditions create realistic, context-sensitive endings:

1. **Natural goodbye** — LLM determines the conversation reached a natural end
2. **Max turns** — hard safety limit (10 turns)
3. **NPC withdrawal** — James shuts down after repeated insensitivity (withdrawn mood + empathy ratio < 30%)
4. **User disengagement** — 3+ very short responses detected as loss of interest
5. **Deep connection achieved** — rapport ≥ 85% and openness ≥ 80% triggers a positive early ending
6. **Error recovery** — graceful handling of system errors

**Why this matters:** The conversation doesn't just "run out of turns." It ends for a *reason* — and that reason shapes the coaching feedback.

### 3.4. Feedback Loop with NPC Adaptation (vs. One-Way Prompt)

**Design Doc approach:** The system prompt is set once with scenario data. The NPC doesn't know how the conversation has been going.

**Game Jam approach:** Every turn, the system prompt is **rebuilt** with:
- James's current mood and openness level
- Stage-specific behavioral instructions (4 distinct instruction sets)
- The user's empathy score so far
- The last 3 behavioral observations from previous turns
- The user's dominant emotion

**This means James in turn 7 is fundamentally different from James in turn 1** — not because of a script, but because of accumulated interaction data.

### 3.5. Data-Driven Coaching Reflection (vs. Generic Feedback)

**Design Doc approach:** Evaluation (Epic E10) focuses on developer-facing tools like LangSmith tracing and red teaming. User-facing feedback is not specified.

**Game Jam approach:** End-of-session coaching is generated from **concrete session data**:
- Empathy scorecard (hits vs. misses with ratio)
- Stage progression path (e.g., "opening → developing → deepening → closing")
- Emotion trajectory across all turns
- NPC mood arc and final state
- **Automated end-reason analysis**: the system explains *why* the conversation ended (e.g., "James withdrew due to feeling pushed" vs. "A deep connection was achieved")

The coaching prompt includes a curated **Empathetic Reflection Lexicon** — 12 trauma-informed principles that ground the feedback in psychological best practices.

---

## 4. Technical Architecture

### 4.1. LangGraph Pipeline (5-Node State Machine)

```
User Input
  → retrieve_context    (scenario validation; future: RAG retrieval)
  → construct_prompt    (adaptive prompt with full memory injection)
  → call_llm            (GPT-4o-mini → structured JSON with 8 output fields)
  → update_memory       (15-dimension state update with composite scoring)
  → route decision:
      ├─ reflection     (6 exit conditions → data-driven coaching)
      └─ end            (return to user for next turn)
```

### 4.2. Structured LLM Output (8 Fields Per Turn)

Each LLM call produces not just dialogue, but a complete turn assessment:

```json
{
    "response": "James's dialogue with *body language*",
    "should_end_turn": false,
    "next_step": "continue",
    "emotion_assessment": {
        "valence": 0.2,
        "arousal": 0.4,
        "label": "cautiously hopeful"
    },
    "observation": "User asked permission before probing — showed respect for boundaries",
    "empathy_quality": "hit",
    "trust_shift": 0.15,
    "npc_internal_mood": "cautious"
}
```

### 4.3. Asymmetric Impact Design

A deliberate design choice: **negative interactions hurt more than positive ones help**.

- An empathy "miss" reduces NPC openness by 0.1; a "hit" only increases it by 0.05
- Once James enters "withdrawn" mood, recovery is capped at +0.02 per turn
- Consistent insensitivity (empathy ratio ≤ 30%) triggers a -0.03 confidence penalty per turn

This mirrors real trauma-informed relationships: trust is hard to build and easy to break.

### 4.4. Tech Stack

| Component | Technology |
|-----------|-----------|
| Orchestration | LangGraph (stateful graph with conditional routing) |
| LLM | GPT-4o-mini (JSON mode, temperature 0.7) |
| Frontend | Streamlit (real-time metrics sidebar, chat UI, emotion chart) |
| State Management | Python TypedDict (15-field UserMemory) |
| Framework | LangChain Core + LangChain OpenAI |

---

## 5. Live UI Features

The Streamlit interface provides real-time visibility into the simulation:

**Sidebar Metrics (updated every turn):**
- Trust level (%) and Rapport score (%)
- Current turn / max turns
- Empathy ratio (%)
- James's current mood (guarded/cautious/warming/open/withdrawn)
- James's openness level (%)
- Stage progression visualization (opening → developing → ...)

**End-of-Session Dashboard:**
- AI-generated coaching feedback (personalized based on session data)
- Emotion trajectory chart (valence + arousal over time)
- Session outcome narrative (why the conversation ended)

---

## 6. What This Proves for the WalkXR Platform

This Game Jam prototype validates several Phase 3 concepts ahead of schedule:

1. **Adaptive agents work.** A single LLM with dynamic prompt injection can simulate nuanced emotional evolution — no separate Emotional State Engine module needed at this stage.
2. **Multi-dimensional tracking is feasible.** 15 state dimensions updated per turn with no performance degradation.
3. **Asymmetric trust mechanics create realism.** The "trust is hard to build, easy to break" principle produces emergent, non-scripted conversations that feel authentic.
4. **Data-driven coaching is more valuable than generic feedback.** Concrete empathy scores and behavioral observations make the reflection actionable.
5. **Intelligent routing creates meaningful endings.** Conversations that end for a *reason* (withdrawal, connection, disengagement) are more impactful than arbitrary turn limits.

---

## 7. Future Roadmap (Toward Full Phase 3)

| Next Step | Description |
|-----------|-------------|
| RAG Integration | Replace the `retrieve_context` stub with LlamaIndex + ChromaDB for scenario knowledge retrieval |
| Multiple Scenarios | Extend beyond bus stop — new NPCs, settings, and therapeutic goals |
| Persistent Memory | Cross-session user growth tracking (currently resets on refresh) |
| BaseAgent Abstraction | Standardize agent class for multi-agent cohort development |
| Evaluation Pipeline | LangSmith integration for automated quality assessment |
| RLAIF Loop | Use session data to fine-tune agent responses over time |

---

## 8. How to Run

```bash
# Install dependencies
pip install -r requirements.txt

# Set OpenAI API key
echo "OPENAI_API_KEY=your-key-here" > .env

# Launch
streamlit run app.py
```

---

*Built for WalkXR Game Jam | LangGraph + GPT-4o-mini + Streamlit*
*Adaptive Empathy Engine — Where AI learns to listen.*