{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "35dbfedb", "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "3719afd0", "metadata": {}, "outputs": [], "source": [ "query = \"How does the agentic loop keep calling the model until it stops?\"" ] }, { "cell_type": "code", "execution_count": null, "id": "b95d0126", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"name\": \"search\",\n", " \"context\": {\n", " \"trace_id\": \"0x4354400e5794c32eb61dd9644dfc28e2\",\n", " \"span_id\": \"0xb7362fd7ef5dfea9\",\n", " \"trace_state\": \"[]\"\n", " },\n", " \"kind\": \"SpanKind.INTERNAL\",\n", " \"parent_id\": \"0xbbfa223ece42605a\",\n", " \"start_time\": \"2026-07-14T18:00:30.319890Z\",\n", " \"end_time\": \"2026-07-14T18:00:30.321468Z\",\n", " \"status\": {\n", " \"status_code\": \"UNSET\"\n", " },\n", " \"attributes\": {},\n", " \"events\": [],\n", " \"links\": [],\n", " \"resource\": {\n", " \"attributes\": {\n", " \"telemetry.sdk.language\": \"python\",\n", " \"telemetry.sdk.name\": \"opentelemetry\",\n", " \"telemetry.sdk.version\": \"1.43.0\",\n", " \"service.instance.id\": \"4029fd78-9ca0-4236-ad8e-d811854a361b\",\n", " \"service.name\": \"unknown_service\"\n", " },\n", " \"schema_url\": \"\"\n", " }\n", "}\n", "{\n", " \"name\": \"llm\",\n", " \"context\": {\n", " \"trace_id\": \"0x4354400e5794c32eb61dd9644dfc28e2\",\n", " \"span_id\": \"0xf4cb5c745265ef16\",\n", " \"trace_state\": \"[]\"\n", " },\n", " \"kind\": \"SpanKind.INTERNAL\",\n", " \"parent_id\": \"0xbbfa223ece42605a\",\n", " \"start_time\": \"2026-07-14T18:00:30.322370Z\",\n", " \"end_time\": \"2026-07-14T18:00:35.079096Z\",\n", " \"status\": {\n", " \"status_code\": \"UNSET\"\n", " },\n", " \"attributes\": {},\n", " \"events\": [],\n", " \"links\": [],\n", " \"resource\": {\n", " \"attributes\": {\n", " \"telemetry.sdk.language\": \"python\",\n", " \"telemetry.sdk.name\": \"opentelemetry\",\n", " \"telemetry.sdk.version\": \"1.43.0\",\n", " \"service.instance.id\": \"4029fd78-9ca0-4236-ad8e-d811854a361b\",\n", " \"service.name\": \"unknown_service\"\n", " },\n", " \"schema_url\": \"\"\n", " }\n", "}\n", "{\n", " \"name\": \"rag\",\n", " \"context\": {\n", " \"trace_id\": \"0x4354400e5794c32eb61dd9644dfc28e2\",\n", " \"span_id\": \"0xbbfa223ece42605a\",\n", " \"trace_state\": \"[]\"\n", " },\n", " \"kind\": \"SpanKind.INTERNAL\",\n", " \"parent_id\": null,\n", " \"start_time\": \"2026-07-14T18:00:30.319836Z\",\n", " \"end_time\": \"2026-07-14T18:00:35.079958Z\",\n", " \"status\": {\n", " \"status_code\": \"UNSET\"\n", " },\n", " \"attributes\": {},\n", " \"events\": [],\n", " \"links\": [],\n", " \"resource\": {\n", " \"attributes\": {\n", " \"telemetry.sdk.language\": \"python\",\n", " \"telemetry.sdk.name\": \"opentelemetry\",\n", " \"telemetry.sdk.version\": \"1.43.0\",\n", " \"service.instance.id\": \"4029fd78-9ca0-4236-ad8e-d811854a361b\",\n", " \"service.name\": \"unknown_service\"\n", " },\n", " \"schema_url\": \"\"\n", " }\n", "}\n", "The agentic loop keeps calling the model until it stops by using a **`while True` loop** that repeatedly checks for function calls. Here’s how it works based on the provided context:\n", "\n", "1. **Loop Condition**:\n", " - The loop runs indefinitely (`while True`) until the model stops generating function calls.\n", " - The loop breaks when `has_function_calls` is `False`, meaning the model has provided a final answer without further tool calls.\n", "\n", "2. **Process Flow**:\n", " - The model generates a response, which may include function calls (e.g., `search`).\n", " - The code checks each item in the response:\n", " - If it’s a function call (e.g., `function_call`), it executes the function, appends the result to the conversation, and marks `has_function_calls` as `True`.\n", " - If it’s a message (e.g., the model’s final answer), it prints the answer and checks if `has_function_calls` is `False` to exit the loop.\n", "\n", "3. **Termination**:\n", " - The loop exits when the model’s response contains no function calls (`has_function_calls == False`), indicating the agent has completed its task.\n", "\n", "### Example Breakdown:\n", "```python\n", "while True:\n", " response = openai_client.responses.create(...)\n", " messages.extend(response.output) # Add model's output to history\n", "\n", " for item in response.output:\n", " if item.type == \"function_call\":\n", " # Execute the function, append result, and continue looping\n", " ...\n", " has_function_calls = True\n", " elif item.type == \"message\":\n", " print(\"Final answer:\", item.content[0].text)\n", " if not has_function_calls:\n", " break # Exit loop\n", "```\n", "\n", "### Key Points:\n", "- The loop **continues** as long as the model requests further tool calls.\n", "- The model decides when to stop (e.g., after a final answer or when it’s satisfied).\n", "- The loop relies on the `has_function_calls` flag to track whether the model needs more iterations.\n", "\n", "This design ensures the agent dynamically adapts to the task, making multiple calls until it achieves its goal.\n" ] } ], "source": [ "from starter import rag_traced\n", "\n", "\n", "answer = rag_traced.rag(query)\n", "print(answer)" ] }, { "cell_type": "code", "execution_count": 3, "id": "e577fdd0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"name\": \"search\",\n", " \"context\": {\n", " \"trace_id\": \"0x6ce8d1a8c28847f58ddadf1f5d944397\",\n", " \"span_id\": \"0x5841abb3d5566bd8\",\n", " \"trace_state\": \"[]\"\n", " },\n", " \"kind\": \"SpanKind.INTERNAL\",\n", " \"parent_id\": \"0xd4236941298ca9a3\",\n", " \"start_time\": \"2026-07-14T18:15:12.381087Z\",\n", " \"end_time\": \"2026-07-14T18:15:12.383154Z\",\n", " \"status\": {\n", " \"status_code\": \"UNSET\"\n", " },\n", " \"attributes\": {},\n", " \"events\": [],\n", " \"links\": [],\n", " \"resource\": {\n", " \"attributes\": {\n", " \"telemetry.sdk.language\": \"python\",\n", " \"telemetry.sdk.name\": \"opentelemetry\",\n", " \"telemetry.sdk.version\": \"1.43.0\",\n", " \"service.instance.id\": \"4029fd78-9ca0-4236-ad8e-d811854a361b\",\n", " \"service.name\": \"unknown_service\"\n", " },\n", " \"schema_url\": \"\"\n", " }\n", "}\n", "{\n", " \"name\": \"llm\",\n", " \"context\": {\n", " \"trace_id\": \"0x6ce8d1a8c28847f58ddadf1f5d944397\",\n", " \"span_id\": \"0x007d6bd2145b8b1a\",\n", " \"trace_state\": \"[]\"\n", " },\n", " \"kind\": \"SpanKind.INTERNAL\",\n", " \"parent_id\": \"0xd4236941298ca9a3\",\n", " \"start_time\": \"2026-07-14T18:15:12.384449Z\",\n", " \"end_time\": \"2026-07-14T18:15:17.670019Z\",\n", " \"status\": {\n", " \"status_code\": \"UNSET\"\n", " },\n", " \"attributes\": {\n", " \"input_tokens\": 7336,\n", " \"output_tokens\": 650\n", " },\n", " \"events\": [],\n", " \"links\": [],\n", " \"resource\": {\n", " \"attributes\": {\n", " \"telemetry.sdk.language\": \"python\",\n", " \"telemetry.sdk.name\": \"opentelemetry\",\n", " \"telemetry.sdk.version\": \"1.43.0\",\n", " \"service.instance.id\": \"4029fd78-9ca0-4236-ad8e-d811854a361b\",\n", " \"service.name\": \"unknown_service\"\n", " },\n", " \"schema_url\": \"\"\n", " }\n", "}\n", "{\n", " \"name\": \"rag\",\n", " \"context\": {\n", " \"trace_id\": \"0x6ce8d1a8c28847f58ddadf1f5d944397\",\n", " \"span_id\": \"0xd4236941298ca9a3\",\n", " \"trace_state\": \"[]\"\n", " },\n", " \"kind\": \"SpanKind.INTERNAL\",\n", " \"parent_id\": null,\n", " \"start_time\": \"2026-07-14T18:15:12.381042Z\",\n", " \"end_time\": \"2026-07-14T18:15:17.674248Z\",\n", " \"status\": {\n", " \"status_code\": \"UNSET\"\n", " },\n", " \"attributes\": {},\n", " \"events\": [],\n", " \"links\": [],\n", " \"resource\": {\n", " \"attributes\": {\n", " \"telemetry.sdk.language\": \"python\",\n", " \"telemetry.sdk.name\": \"opentelemetry\",\n", " \"telemetry.sdk.version\": \"1.43.0\",\n", " \"service.instance.id\": \"4029fd78-9ca0-4236-ad8e-d811854a361b\",\n", " \"service.name\": \"unknown_service\"\n", " },\n", " \"schema_url\": \"\"\n", " }\n", "}\n", "The agentic loop keeps calling the model until it stops by using a **`while True` loop** with a condition to check whether the model has requested further tool calls. Here’s how it works based on the provided context:\n", "\n", "### Key Mechanism:\n", "1. **Loop Continuation**:\n", " - The loop runs indefinitely (`while True`) until a stopping condition is met.\n", " - Inside the loop, the model’s response is processed to check if it contains a `function_call` (indicating the model wants to use a tool).\n", "\n", "2. **Condition to Stop**:\n", " - The loop breaks when the model’s response contains **no function calls** (`has_function_calls == False`).\n", " - This means the model has decided it has enough information to answer without invoking additional tools.\n", "\n", "3. **Iteration Tracking**:\n", " - An iteration counter (`it`) is incremented to track how many round-trips (model → tool → model) have occurred.\n", " - This helps debug or limit the loop (e.g., to prevent infinite loops).\n", "\n", "---\n", "\n", "### How It Works in Code:\n", "```python\n", "while True:\n", " print(f\"iteration #{it}...\")\n", " has_function_calls = False # Reset flag for this iteration\n", "\n", " # Step 1: Send the current conversation history to the model\n", " response = openai_client.responses.create(\n", " model=\"gpt-5.4-mini\",\n", " input=messages, # Current conversation history\n", " tools=[search_tool],\n", " )\n", "\n", " # Step 2: Process the model's response\n", " messages.extend(response.output) # Append model's output to history\n", "\n", " for item in response.output:\n", " if item.type == \"function_call\":\n", " print(\"function_call:\", item.name, item.arguments)\n", " call_output = make_call(item) # Execute the tool and get results\n", " messages.append(call_output) # Add tool output to history\n", " has_function_calls = True # Flag that the model wants more\n", "\n", " elif item.type == \"message\":\n", " print(\"ASSISTANT:\", item.content[0].text)\n", "\n", " it += 1 # Increment iteration counter\n", "\n", " # Step 3: Stop if no function calls were requested\n", " if not has_function_calls:\n", " break\n", "```\n", "\n", "---\n", "\n", "### Why It Stops:\n", "- The loop **only exits** when the model’s final response contains no `function_call` entries. This signals the model has answered the question or reached a conclusion without needing further tool calls.\n", "- The instructions (e.g., `instructions` variable) guide the model to explicitly request tools (e.g., `search`) when needed, but it remains flexible to decide when to stop.\n", "\n", "---\n", "\n", "### Example:\n", "If the model asks for a tool call (e.g., `search`), the loop:\n", "1. Executes the tool (e.g., searches for \"Ollama\").\n", "2. Appends the tool’s result to the conversation history.\n", "3. Sends the updated history back to the model.\n", "4. Repeats until the model’s next response has no function calls.\n", "\n", "This continues until the model provides a final answer without further tool requests.\n" ] } ], "source": [ "answer = rag_traced.rag(query)\n", "print(answer)" ] }, { "cell_type": "code", "execution_count": 4, "id": "02384d83", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "5.28557\n", "5285.57\n" ] } ], "source": [ "from datetime import datetime\n", "\n", "start = datetime.fromisoformat(\"2026-07-14T18:15:12.384449Z\")\n", "end = datetime.fromisoformat(\"2026-07-14T18:15:17.670019Z\")\n", "\n", "duration = (end - start).total_seconds()\n", "\n", "print(duration)\n", "print(duration * 1000)" ] }, { "cell_type": "code", "execution_count": 4, "id": "b1836b8d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The agentic loop keeps calling the model until it stops by using a **`while True` loop** with a condition to check whether the model has requested further tool calls. Here’s how it works based on the provided context:\n", "\n", "### Key Mechanism:\n", "1. **Loop Continuation**:\n", " - The loop (`while True`) runs indefinitely until a specific condition is met.\n", " - Inside the loop, the model’s response is processed to check if it contains any `function_call` entries.\n", "\n", "2. **Condition to Stop**:\n", " - The loop breaks when `has_function_calls` is `False`. This flag is set to `True` each time the model requests a tool call (e.g., `search`) and `False` when the model provides a final answer without further tool calls.\n", "\n", "3. **Iteration Tracking**:\n", " - The iteration counter (`it`) is incremented each time through the loop to show progress (e.g., `iteration #1`, `iteration #2`).\n", "\n", "### Example Breakdown:\n", "```python\n", "while True:\n", " print(f\"iteration #{it}...\")\n", " has_function_calls = False # Reset flag for this iteration\n", "\n", " response = openai_client.responses.create(...)\n", "\n", " for item in response.output:\n", " if item.type == \"function_call\":\n", " print(\"function_call:\", item.name, item.arguments)\n", " call_output = make_call(item)\n", " messages.append(call_output)\n", " has_function_calls = True # Model wants more tool calls\n", "\n", " elif item.type == \"message\":\n", " print(\"ASSISTANT:\", item.content[0].text)\n", "\n", " it += 1\n", " if not has_function_calls: # Exit condition: no more tool calls\n", " break\n", "```\n", "\n", "### Why It Stops:\n", "- The loop stops when the model’s response contains no `function_call` entries. This means the model has decided to provide a final answer without needing additional tool calls.\n", "- The instructions (e.g., `instructions` variable) guide the model to make multiple searches and only stop when it has sufficient information.\n", "\n", "### Summary:\n", "The loop persists until the model explicitly signals it’s done with tool calls, leveraging the `has_function_calls` flag to enforce this condition. This dynamic approach allows the agent to adapt to the model’s decisions, such as retrying searches for typos or expanding queries.\n" ] } ], "source": [ "from starter import rag_traced\n", "\n", "answer = rag_traced.rag(query)\n", "print(answer)" ] }, { "cell_type": "code", "execution_count": 5, "id": "e1759c57", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " name start_time end_time input_tokens \\\n", "0 search 1784056376194982017 1784056376197035791 NaN \n", "1 llm 1784056376204200104 1784056379820000468 7336.0 \n", "2 rag 1784056376194915754 1784056379825301546 NaN \n", "\n", " output_tokens cost \n", "0 NaN None \n", "1 484.0 None \n", "2 NaN None \n" ] } ], "source": [ "import sqlite3\n", "import pandas as pd\n", "\n", "conn = sqlite3.connect(\"traces.db\")\n", "\n", "df = pd.read_sql(\n", " \"SELECT * FROM spans\",\n", " conn\n", ")\n", "\n", "print(df)" ] }, { "cell_type": "code", "execution_count": 6, "id": "3e7da26a", "metadata": {}, "outputs": [], "source": [ "df[\"duration_ms\"] = (\n", " df[\"end_time\"] -\n", " df[\"start_time\"]\n", ") / 1e6" ] }, { "cell_type": "code", "execution_count": 7, "id": "bd74f0f1", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "name\n", "llm 3615.800364\n", "search 2.053774\n", "Name: duration_ms, dtype: float64" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "(\n", " df[df.name != \"rag\"]\n", " .groupby(\"name\")[\"duration_ms\"]\n", " .sum()\n", ")" ] }, { "cell_type": "code", "execution_count": 10, "id": "09a1e82f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The agentic loop keeps calling the model until it stops because of the **`while True` loop structure** combined with the **`has_function_calls` flag**.\n", "\n", "### How It Works:\n", "1. **Infinite Loop (`while True`)**:\n", " The loop runs indefinitely until explicitly broken out of.\n", " ```python\n", " while True:\n", " # Model call and processing\n", " if has_function_calls == False:\n", " break # Exit condition\n", " ```\n", "\n", "2. **`has_function_calls` Flag**:\n", " After each model response, the code checks if the model requested any function calls (e.g., `search`).\n", " - If **no function calls** are present, the loop breaks, and the final answer is returned.\n", " - If **function calls** exist, the loop continues, processing the tool results and re-sending them to the model.\n", "\n", "3. **Iteration Logic**:\n", " - The model decides whether to call tools (e.g., `search`) based on its reasoning.\n", " - The loop keeps iterating until the model provides a final answer without further tool requests.\n", "\n", "### Why It Stops:\n", "The loop stops when the model’s response contains **no function calls** (`has_function_calls == False`), indicating it has completed its task and provided a final answer.\n", "\n", "### Example Flow:\n", "- **User Query**: *\"How do I run Olama locally?\"*\n", "- **Model’s First Attempt**: Searches for \"Olama\" → Returns no results → Asks for a tool call.\n", "- **Tool Call**: Executes `search(\"Olama\")` → Returns empty results.\n", "- **Model’s Second Attempt**: Searches for \"Ollama\" → Returns useful results → Provides an answer.\n", "- **Final Response**: No further function calls → Loop breaks, and the answer is returned.\n", "\n", "This design ensures the agent dynamically adapts to incomplete or incorrect initial queries by repeatedly querying tools until it finds a satisfactory answer.\n" ] } ], "source": [ "\n", "answer = rag_traced.rag(query)\n", "print(answer)" ] }, { "cell_type": "code", "execution_count": 12, "id": "c39ebb51", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " name start_time end_time input_tokens \\\n", "0 search 1784056376194982017 1784056376197035791 NaN \n", "1 llm 1784056376204200104 1784056379820000468 7336.0 \n", "2 rag 1784056376194915754 1784056379825301546 NaN \n", "3 search 1784056605627375896 1784056605629557648 NaN \n", "4 llm 1784056605637611721 1784056608796712115 7336.0 \n", "5 rag 1784056605627301767 1784056608800542395 NaN \n", "6 search 1784056611544953865 1784056611546858029 NaN \n", "7 llm 1784056611553761475 1784056615061661678 7336.0 \n", "8 rag 1784056611544911917 1784056615065781962 NaN \n", "9 search 1784056616258031330 1784056616259944932 NaN \n", "10 llm 1784056616265641879 1784056618620603018 7336.0 \n", "11 rag 1784056616257988741 1784056618629422079 NaN \n", "\n", " output_tokens cost \n", "0 NaN None \n", "1 484.0 None \n", "2 NaN None \n", "3 NaN None \n", "4 508.0 None \n", "5 NaN None \n", "6 NaN None \n", "7 512.0 None \n", "8 NaN None \n", "9 NaN None \n", "10 398.0 None \n", "11 NaN None \n" ] } ], "source": [ "import sqlite3\n", "import pandas as pd\n", "\n", "conn = sqlite3.connect(\"traces.db\")\n", "\n", "df = pd.read_sql(\n", " \"SELECT * FROM spans\",\n", " conn\n", ")\n", "\n", "print(df)" ] }, { "cell_type": "code", "execution_count": 13, "id": "e141e399", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1 7336.0\n", "4 7336.0\n", "7 7336.0\n", "10 7336.0\n", "Name: input_tokens, dtype: float64\n" ] } ], "source": [ "llm_df = df[df.name == \"llm\"]\n", "\n", "print(\n", " llm_df[\"input_tokens\"]\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "rag-monitoring", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.1" } }, "nbformat": 4, "nbformat_minor": 5 }