Cognitive-rag / 05_generation
README.md

05_generation — Answer Generation (AWS Bedrock)

Overview

This module implements the Generation stage of a Cognitive Retrieval-Augmented Generation (CognitiveRAG) pipeline.

Its responsibility is to take a user query and a set of retrieved document chunks, then generate a grounded, deterministic answer strictly based on the provided context.

The module uses Amazon Bedrock with Claude 3 Haiku to ensure reliability, traceability, and compliance with AWS-based hackathon requirements.

What This Module Does

Input

  • A user question
  • Retrieved document chunks (already selected by the retrieval stage)
  • Optional metadata about each chunk

Output

  • A concise answer generated only from the retrieved content
  • A list of sources used to produce the answer

This stage does not perform retrieval, embedding, or ranking. It assumes those steps have already been completed upstream.

Input Schema

{
  "query": "What is the parental leave policy?",
  "retrieved_chunks": [
    "Employees are entitled to parental leave of up to 12 weeks...",
    "Eligibility requirements include..."
  ],
  "chunk_metadata": [
    {
      "chunk_id": "mtsamples_0_chunk_17",
      "doc_id": "mtsamples_0",
      "section": "Employee Benefits",
      "page": null,
      "score": 0.92
    }
  ]
}

Output Schema

{
  "answer": "Employees are entitled to parental leave of up to 12 weeks.",
  "sources": [
    {
      "chunk_id": "mtsamples_0_chunk_17",
      "doc_id": "mtsamples_0",
      "section": "Employee Benefits",
      "page": null,
      "score": 0.92
    }
  ]
}

AWS Services Used (Design Rationale)

Amazon Bedrock (Claude 3 Haiku)

This module uses Amazon Bedrock to invoke Anthropic’s Claude 3 Haiku model via the Bedrock Runtime API.

Amazon Bedrock was selected to provide a fully managed, serverless interface for foundation models while maintaining strong governance, security, and cost controls.

Why Amazon Bedrock was chosen

  • Hackathon compliance
    Ensures the project meaningfully uses AWS-native services rather than external APIs.

  • Serverless inference
    No infrastructure provisioning, scaling logic, or deployment management required.

  • Security & governance
    IAM-based authentication (no hardcoded API keys), aligned with AWS best practices.

  • Cost efficiency
    Claude 3 Haiku is optimized for fast, low-latency, low-cost question-answering workloads.

  • Model flexibility
    Enables switching between Anthropic, Meta, or Mistral models without changing application logic.

Comparison to alternatives

Compared to alternatives such as OpenAI’s API, Amazon Bedrock provides:

  • Native integration with AWS IAM and billing controls
  • Centralized access to multiple model providers
  • Better alignment with enterprise and cloud-native architectures

Model Configuration

  • Model ID: anthropic.claude-3-haiku-20240307-v1:0
  • Inference type: On-demand
  • Temperature: 0
  • Maximum output tokens: 500

The system prompt explicitly enforces:

  • Use of retrieved context only
  • No hallucinations or external knowledge
  • A fallback response when information is missing

How It Works (High-Level Flow)

  1. Receives a user query and retrieved document chunks from the upstream retrieval stage
  2. Concatenates retrieved chunks into a single contextual prompt
  3. Sends the prompt to Claude 3 Haiku via Amazon Bedrock
  4. Parses the model response
  5. Returns the answer along with source metadata

How to Run Locally

Local Setup

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Create a .env file if you want to load AWS credentials from environment variables (otherwise rely on your standard AWS CLI config):

AWS_ACCESS_KEY_ID=your_key
AWS_SECRET_ACCESS_KEY=your_secret
AWS_DEFAULT_REGION=ca-central-1

Run

python generate_answer.py

Requirements

  • Python 3.9+
  • AWS credentials configured locally
  • IAM user with Amazon Bedrock permissions
  • boto3
  • python-dotenv

Environment variables are loaded from .env, which is excluded from version control.

Notes for Integration

  • This module is stateless
  • Can be invoked as a standalone script or imported as a function
  • Designed to plug directly into the retrieval stage output without modification
  • Output schema is stable and suitable for downstream UI or API layers

Author Notes (Sandy)

This generation stage prioritizes grounded, explainable answers over creativity.

The design focuses on determinism, traceability, and correctness, which are essential for enterprise-style use cases such as policy lookup, documentation QA, and internal knowledge systems.

Amazon Bedrock was selected to balance speed, cost, and governance, making this module suitable for both rapid prototyping and scalable production deployments.

Powered by GitFront