WalkXR-AI / Paul-Game-Jam-Phase-2.5
README.md

WalkXR Game Jam: Adaptive Empathy Engine

Phase 2.5 — Bridging the Gap Between "Working Walk" and "Emotional OS"

1. Project Overview

WalkXR Game Jam is an AI-powered empathy training simulator where users practice trauma-informed conversations with a psychologically realistic NPC. The first scenario features James, a military veteran with PTSD, at a bus stop.

Built with LangGraph + GPT-4o-mini + Streamlit, this prototype goes beyond the original WalkXR Design Document's Phase 2 (a working walk experience) by implementing core elements of Phase 3 (the Emotional OS) — specifically, real-time adaptive NPC behavior, multi-dimensional emotional tracking, and data-driven coaching feedback.

2. Where This Sits: Phase 2 → Phase 3

The WalkXR Master Design Document defines three phases:

  • Phase 1 (E01-E06): Build foundational infrastructure (RAG, BaseAgent, Orchestrator)
  • Phase 2 (E07-E10): Assemble and test a full walk experience
  • Phase 3 (E11-E12+): Generalize into a self-improving Emotional OS platform

This Game Jam prototype operates at Phase 2.5 — a fully functional walk experience that already incorporates the adaptive intelligence planned for Phase 3.

3. Key Differentiators from the Original Design Document

3.1. Adaptive NPC State Machine (vs. Static Persona)

Design Doc approach: NPC agents follow pre-scripted persona descriptions. Behavior is defined once and remains consistent throughout.

Game Jam approach: James has a live internal state that evolves every turn:

Metric Range What It Tracks
npc_openness 0.0 – 1.0 How willing James is to share personal details
npc_mood 5 states guarded → cautious → warming → open → withdrawn
trust_shift -0.3 – +0.3 Per-turn trust delta based on user's approach

Result: James doesn't just "respond" — he adapts. Show genuine empathy and he gradually opens up. Push too hard or use clichés and he withdraws. This creates emergent, non-scripted emotional arcs unique to each session.

3.2. Multi-Dimensional Memory System (vs. Simple State Tracking)

Design Doc approach: Phase 2 uses basic turn counting and a simple memory dict. The sophisticated Emotional State Engine is deferred to Phase 3 (Epic E05).

Game Jam approach: Every turn updates 15 state dimensions simultaneously:

  • Trust & Rapport: confidence_level, rapport_score (composite: trust 40% + openness 30% + empathy ratio 30%)
  • Emotion Tracking: emotion_trajectory (valence + arousal per turn), dominant_emotion
  • Behavioral Analysis: empathy_hits, empathy_misses, empathy_ratio — the LLM classifies each user response as a "hit" (genuine empathy), "miss" (insensitive/cliché), or "neutral"
  • Engagement Detection: avg_response_length, silence_count — detects user disengagement
  • NPC Internal State: npc_openness, npc_mood — James's emotional evolution
  • Stage Progression: 4-stage flow (opening → developing → deepening → closing) driven by rapport signals, not just turn count

3.3. Intelligent Conversation Routing (vs. Turn-Count Exit)

Design Doc approach: Conversations end when max turns are reached or the LLM says to stop.

Game Jam approach: 6 exit conditions create realistic, context-sensitive endings:

  1. Natural goodbye — LLM determines the conversation reached a natural end
  2. Max turns — hard safety limit (10 turns)
  3. NPC withdrawal — James shuts down after repeated insensitivity (withdrawn mood + empathy ratio < 30%)
  4. User disengagement — 3+ very short responses detected as loss of interest
  5. Deep connection achieved — rapport ≥ 85% and openness ≥ 80% triggers a positive early ending
  6. Error recovery — graceful handling of system errors

Why this matters: The conversation doesn't just "run out of turns." It ends for a reason — and that reason shapes the coaching feedback.

3.4. Feedback Loop with NPC Adaptation (vs. One-Way Prompt)

Design Doc approach: The system prompt is set once with scenario data. The NPC doesn't know how the conversation has been going.

Game Jam approach: Every turn, the system prompt is rebuilt with:

  • James's current mood and openness level
  • Stage-specific behavioral instructions (4 distinct instruction sets)
  • The user's empathy score so far
  • The last 3 behavioral observations from previous turns
  • The user's dominant emotion

This means James in turn 7 is fundamentally different from James in turn 1 — not because of a script, but because of accumulated interaction data.

3.5. Data-Driven Coaching Reflection (vs. Generic Feedback)

Design Doc approach: Evaluation (Epic E10) focuses on developer-facing tools like LangSmith tracing and red teaming. User-facing feedback is not specified.

Game Jam approach: End-of-session coaching is generated from concrete session data:

  • Empathy scorecard (hits vs. misses with ratio)
  • Stage progression path (e.g., "opening → developing → deepening → closing")
  • Emotion trajectory across all turns
  • NPC mood arc and final state
  • Automated end-reason analysis: the system explains why the conversation ended (e.g., "James withdrew due to feeling pushed" vs. "A deep connection was achieved")

The coaching prompt includes a curated Empathetic Reflection Lexicon — 12 trauma-informed principles that ground the feedback in psychological best practices.

4. Technical Architecture

4.1. LangGraph Pipeline (5-Node State Machine)

User Input
  → retrieve_context    (scenario validation; future: RAG retrieval)
  → construct_prompt    (adaptive prompt with full memory injection)
  → call_llm            (GPT-4o-mini → structured JSON with 8 output fields)
  → update_memory       (15-dimension state update with composite scoring)
  → route decision:
      ├─ reflection     (6 exit conditions → data-driven coaching)
      └─ end            (return to user for next turn)

4.2. Structured LLM Output (8 Fields Per Turn)

Each LLM call produces not just dialogue, but a complete turn assessment:

{
    "response": "James's dialogue with *body language*",
    "should_end_turn": false,
    "next_step": "continue",
    "emotion_assessment": {
        "valence": 0.2,
        "arousal": 0.4,
        "label": "cautiously hopeful"
    },
    "observation": "User asked permission before probing — showed respect for boundaries",
    "empathy_quality": "hit",
    "trust_shift": 0.15,
    "npc_internal_mood": "cautious"
}

4.3. Asymmetric Impact Design

A deliberate design choice: negative interactions hurt more than positive ones help.

  • An empathy "miss" reduces NPC openness by 0.1; a "hit" only increases it by 0.05
  • Once James enters "withdrawn" mood, recovery is capped at +0.02 per turn
  • Consistent insensitivity (empathy ratio ≤ 30%) triggers a -0.03 confidence penalty per turn

This mirrors real trauma-informed relationships: trust is hard to build and easy to break.

4.4. Tech Stack

Component Technology
Orchestration LangGraph (stateful graph with conditional routing)
LLM GPT-4o-mini (JSON mode, temperature 0.7)
Frontend Streamlit (real-time metrics sidebar, chat UI, emotion chart)
State Management Python TypedDict (15-field UserMemory)
Framework LangChain Core + LangChain OpenAI

5. Live UI Features

The Streamlit interface provides real-time visibility into the simulation:

Sidebar Metrics (updated every turn):

  • Trust level (%) and Rapport score (%)
  • Current turn / max turns
  • Empathy ratio (%)
  • James's current mood (guarded/cautious/warming/open/withdrawn)
  • James's openness level (%)
  • Stage progression visualization (opening → developing → ...)

End-of-Session Dashboard:

  • AI-generated coaching feedback (personalized based on session data)
  • Emotion trajectory chart (valence + arousal over time)
  • Session outcome narrative (why the conversation ended)

6. What This Proves for the WalkXR Platform

This Game Jam prototype validates several Phase 3 concepts ahead of schedule:

  1. Adaptive agents work. A single LLM with dynamic prompt injection can simulate nuanced emotional evolution — no separate Emotional State Engine module needed at this stage.
  2. Multi-dimensional tracking is feasible. 15 state dimensions updated per turn with no performance degradation.
  3. Asymmetric trust mechanics create realism. The "trust is hard to build, easy to break" principle produces emergent, non-scripted conversations that feel authentic.
  4. Data-driven coaching is more valuable than generic feedback. Concrete empathy scores and behavioral observations make the reflection actionable.
  5. Intelligent routing creates meaningful endings. Conversations that end for a reason (withdrawal, connection, disengagement) are more impactful than arbitrary turn limits.

7. Future Roadmap (Toward Full Phase 3)

Next Step Description
RAG Integration Replace the retrieve_context stub with LlamaIndex + ChromaDB for scenario knowledge retrieval
Multiple Scenarios Extend beyond bus stop — new NPCs, settings, and therapeutic goals
Persistent Memory Cross-session user growth tracking (currently resets on refresh)
BaseAgent Abstraction Standardize agent class for multi-agent cohort development
Evaluation Pipeline LangSmith integration for automated quality assessment
RLAIF Loop Use session data to fine-tune agent responses over time

8. How to Run

# Install dependencies
pip install -r requirements.txt

# Set OpenAI API key
echo "OPENAI_API_KEY=your-key-here" > .env

# Launch
streamlit run app.py

Built for WalkXR Game Jam | LangGraph + GPT-4o-mini + Streamlit Adaptive Empathy Engine — Where AI learns to listen.

Powered by GitFront