> ## Documentation Index
> Fetch the complete documentation index at: https://ago.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Simulation API

> Run agent quality simulations, manage test datasets, and validate agent behavior programmatically

Run agent quality simulations, manage test datasets, and validate agent behavior through the API. Also available as MCP tools for AI agent workflows.

## Authentication

All endpoints require an API key with `agents_config:read` or `agents_config:write` scope.

```bash theme={null}
curl -H "X-API-Key: ago_v1_sk_your_key_here" \
  https://your-instance.ago.ai/api/v1/simulation/datasets
```

## Quick Test

Run a single ad-hoc test against an agent. Returns the response synchronously (may take 10-30 seconds).

```bash theme={null}
POST /api/v1/simulation/quick-test
```

```json theme={null}
{
  "agent_id": "agent-uuid",
  "question": "What are your business hours?"
}
```

**Response:**

```json theme={null}
{
  "object": "simulation_quick_test",
  "response": "We're open Monday through Friday, 9 AM to 5 PM EST.",
  "tool_calls": null,
  "execution_time_ms": 2450,
  "error": null,
  "routed_agent_id": null,
  "routed_agent_name": null
}
```

**Scope:** `agents_config:write`

***

## Datasets

Datasets group related test cases for batch simulation runs.

### List Datasets

```bash theme={null}
GET /api/v1/simulation/datasets?page=1&page_size=50
```

**Response:**

```json theme={null}
{
  "object": "list",
  "data": [
    {
      "object": "simulation_dataset",
      "id": "dataset-uuid",
      "name": "Support Agent Tests",
      "description": "Core functionality tests",
      "agent_id": "agent-uuid",
      "agent_name": "Support Agent",
      "default_persona_id": null,
      "test_case_count": 25,
      "created_at": "2026-01-15T10:30:00+00:00",
      "updated_at": "2026-01-15T10:30:00+00:00"
    }
  ],
  "has_more": false,
  "total": 1
}
```

**Scope:** `agents_config:read`

### Create Dataset

```bash theme={null}
POST /api/v1/simulation/datasets
```

```json theme={null}
{
  "name": "Support Agent Tests",
  "description": "Core functionality tests for the support agent",
  "agent_id": "agent-uuid",
  "default_persona_id": "persona-uuid"
}
```

| Field                | Type   | Required | Description                    |
| -------------------- | ------ | -------- | ------------------------------ |
| name                 | string | Yes      | Dataset name                   |
| description          | string | No       | Purpose description            |
| agent\_id            | string | No       | Default agent for test runs    |
| default\_persona\_id | string | No       | Default persona for test cases |

**Scope:** `agents_config:write`

### Get, Update, Delete Dataset

```bash theme={null}
GET    /api/v1/simulation/datasets/{dataset_id}
PUT    /api/v1/simulation/datasets/{dataset_id}
DELETE /api/v1/simulation/datasets/{dataset_id}
```

Deleting a dataset removes all its test cases and runs.

***

## Test Cases

Each test case defines a question and the expected agent behavior.

### Expected Types

| Type            | What It Tests                     | Required Fields                                       |
| --------------- | --------------------------------- | ----------------------------------------------------- |
| `answer`        | Agent gives correct text response | `expected_answer`                                     |
| `tool_call`     | Agent calls the right tool        | `expected_tool_name`, optionally `expected_tool_args` |
| `agent_routing` | Router selects the right agent    | `expected_agent_id`                                   |

### Create Test Case

```bash theme={null}
POST /api/v1/simulation/datasets/{dataset_id}/test-cases
```

**Answer test:**

```json theme={null}
{
  "question": "What are your business hours?",
  "expected_type": "answer",
  "expected_answer": "Monday to Friday, 9 AM to 5 PM EST."
}
```

**Tool call test:**

```json theme={null}
{
  "question": "I need to speak with a human",
  "expected_type": "tool_call",
  "expected_tool_name": "ago_ticketing",
  "expected_tool_args": { "priority": "high" }
}
```

**Multi-turn test** (conversation with multiple exchanges):

```json theme={null}
{
  "question": "Multi-turn: collect address",
  "expected_type": "answer",
  "steps": [
    {
      "step_order": 1,
      "user_message": "I want to update my address",
      "expected_type": "answer",
      "expected_answer": "What is your new address?"
    },
    {
      "step_order": 2,
      "user_message": "123 Main St, Springfield",
      "expected_type": "tool_call",
      "expected_tool_name": "update_address"
    }
  ]
}
```

Steps run in sequence within the same conversation thread, so the agent retains context.

**Scope:** `agents_config:write`

### List Test Cases

```bash theme={null}
GET /api/v1/simulation/datasets/{dataset_id}/test-cases?page=1&page_size=50
```

**Scope:** `agents_config:read`

### Bulk Create

```bash theme={null}
POST /api/v1/simulation/datasets/{dataset_id}/test-cases/bulk
```

```json theme={null}
{
  "test_cases": [
    { "question": "Question 1", "expected_type": "answer", "expected_answer": "Answer 1" },
    { "question": "Question 2", "expected_type": "answer", "expected_answer": "Answer 2" }
  ]
}
```

### Import from Conversation

Extract a test case from an existing conversation thread:

```bash theme={null}
POST /api/v1/simulation/datasets/{dataset_id}/import/conversation/{thread_id}
```

Uses the last user message and assistant response from the thread. **Scope:** `agents_config:write`

***

## Personas

Personas simulate different user profiles during testing. They can override the agent, LLM temperature, available tools, and user attributes.

### Create Persona

```bash theme={null}
POST /api/v1/simulation/personas
```

```json theme={null}
{
  "name": "Premium Customer",
  "description": "Long-time customer with premium subscription",
  "role": "Premium",
  "behavior_notes": "Expects fast, personalized responses",
  "agent_id": "agent-uuid",
  "custom_llm_temperature": 0.3,
  "enabled_tool_ids": ["tool-uuid-1", "tool-uuid-2"],
  "user_overrides": {
    "email": "premium@example.com",
    "additional_info": { "plan": "premium" }
  }
}
```

| Field                    | Type   | Required | Description                                                                   |
| ------------------------ | ------ | -------- | ----------------------------------------------------------------------------- |
| name                     | string | Yes      | Persona name                                                                  |
| description              | string | No       | Description                                                                   |
| role                     | string | No       | Short role label                                                              |
| behavior\_notes          | string | No       | Instructions for evaluation                                                   |
| agent\_id                | string | No       | Override agent for this persona                                               |
| custom\_llm\_temperature | number | No       | Override LLM temperature                                                      |
| enabled\_tool\_ids       | array  | No       | Restrict available tools                                                      |
| user\_overrides          | object | No       | Mock user properties (email, additional\_info, business\_context, jwt\_token) |

**Scope:** `agents_config:write`

### List, Get, Update, Delete Personas

```bash theme={null}
GET    /api/v1/simulation/personas
GET    /api/v1/simulation/personas/{persona_id}
PUT    /api/v1/simulation/personas/{persona_id}
DELETE /api/v1/simulation/personas/{persona_id}
```

***

## Simulation Runs

### Start a Run

```bash theme={null}
POST /api/v1/simulation/datasets/{dataset_id}/runs
```

```json theme={null}
{
  "agent_id": "agent-uuid"
}
```

Runs execute asynchronously. The response returns immediately with `status: "pending"`.

**Scope:** `agents_config:write`

### Poll Run Status

```bash theme={null}
GET /api/v1/simulation/runs/{run_id}/status
```

```json theme={null}
{
  "object": "simulation_run_status",
  "status": "running",
  "total_tests": 25,
  "completed_tests": 12,
  "progress_percentage": 48.0
}
```

Poll this endpoint until `status` is `"completed"` or `"failed"`.

| Status    | Description               |
| --------- | ------------------------- |
| pending   | Created, waiting to start |
| running   | Tests in progress         |
| completed | All tests finished        |
| failed    | Run encountered an error  |

**Scope:** `agents_config:read`

### Get Run Details

```bash theme={null}
GET /api/v1/simulation/runs/{run_id}
```

```json theme={null}
{
  "object": "simulation_run",
  "id": "run-uuid",
  "dataset_id": "dataset-uuid",
  "dataset_name": "Support Agent Tests",
  "agent_id": "agent-uuid",
  "agent_name": "Support Agent",
  "status": "completed",
  "total_tests": 25,
  "passed_tests": 23,
  "failed_tests": 2,
  "success_rate": 92.0,
  "error_message": null,
  "started_at": "2026-01-15T10:30:00+00:00",
  "completed_at": "2026-01-15T10:45:00+00:00"
}
```

### Get Run Results

```bash theme={null}
GET /api/v1/simulation/runs/{run_id}/results?page=1&page_size=50
```

```json theme={null}
{
  "object": "list",
  "data": [
    {
      "object": "simulation_result",
      "id": "result-uuid",
      "test_case_id": "test-case-uuid",
      "conversation_step_id": null,
      "thread_id": "thread-uuid",
      "actual_response": "We're open Monday through Friday, 9 AM to 5 PM EST.",
      "actual_tool_calls": null,
      "actual_routed_agent_id": null,
      "actual_routed_agent_name": null,
      "passed": true,
      "llm_evaluation": {
        "passed": true,
        "score": 0.95,
        "reasoning": "Response accurately conveys business hours.",
        "criteria_scores": { "accuracy": 1.0, "completeness": 0.9, "relevance": 1.0 }
      },
      "evaluation_score": 0.95,
      "executed_at": "2026-01-15T10:32:00+00:00",
      "execution_time_ms": 1250
    }
  ],
  "has_more": true,
  "total": 25
}
```

### List Runs, Delete Run

```bash theme={null}
GET    /api/v1/simulation/datasets/{dataset_id}/runs
DELETE /api/v1/simulation/runs/{run_id}
```

***

## MCP Tools

All simulation endpoints are available as MCP tools. Connect to the MCP server at `/api/v1/mcp` with an API key that has `agents_config` scopes.

### Workflow: Run a Simulation via MCP

1. **Start the run:** Call `start_simulation_run` with `dataset_id` and `agent_id`
2. **Poll for completion:** Call `get_simulation_run_status` with `run_id` until `status` is `"completed"` or `"failed"`
3. **Get results:** Call `get_simulation_run_results` with `run_id` to see individual test results

### Available MCP Tools

| Tool                                  | Description                        | Scope |
| ------------------------------------- | ---------------------------------- | ----- |
| `list_simulation_datasets`            | List all datasets                  | read  |
| `get_simulation_dataset`              | Get dataset by ID                  | read  |
| `create_simulation_dataset`           | Create a dataset                   | write |
| `update_simulation_dataset`           | Update a dataset                   | write |
| `delete_simulation_dataset`           | Delete a dataset                   | write |
| `list_simulation_test_cases`          | List test cases in a dataset       | read  |
| `get_simulation_test_case`            | Get test case by ID                | read  |
| `create_simulation_test_case`         | Create a test case                 | write |
| `update_simulation_test_case`         | Update a test case                 | write |
| `delete_simulation_test_case`         | Delete a test case                 | write |
| `bulk_create_simulation_test_cases`   | Create multiple test cases         | write |
| `import_simulation_from_conversation` | Create test case from conversation | write |
| `list_simulation_personas`            | List personas                      | read  |
| `get_simulation_persona`              | Get persona by ID                  | read  |
| `create_simulation_persona`           | Create a persona                   | write |
| `update_simulation_persona`           | Update a persona                   | write |
| `delete_simulation_persona`           | Delete a persona                   | write |
| `start_simulation_run`                | Start a simulation run             | write |
| `list_simulation_runs`                | List runs for a dataset            | read  |
| `get_simulation_run`                  | Get run details                    | read  |
| `get_simulation_run_status`           | Poll run progress                  | read  |
| `get_simulation_run_results`          | Get run results                    | read  |
| `delete_simulation_run`               | Delete a run                       | write |
| `run_simulation_quick_test`           | Run a single test                  | write |

***

## Error Responses

Errors follow the standard API v1 format:

```json theme={null}
{
  "error": {
    "type": "invalid_request_error",
    "code": "not_found",
    "message": "Dataset not found with ID 'abc'.",
    "param": "dataset_id",
    "doc_url": "https://ago.mintlify.app/api/errors#not_found"
  }
}
```

***

## Related

* [Simulation Testing Overview](../features/simulation-testing) - Testing concepts and UI guide
* [MCP Integration](../guides/mcp-integration-howto) - How to set up MCP clients
* [Agents API](./agents-api) - Agent management endpoints