Knowledge Coverage

Measure how well your knowledge base answers a set of reference business questions, broken down by domain

Knowledge Coverage tells you whether your documents can answer the business questions your company cares about. You provide a list of reference questions grouped by domain (legal, HR, finance, and so on), and AGO checks each one against your knowledge base. The result is a percentage per domain, such as “84% on legal questions”.

How it works

A coverage referential is a list of questions organized by domain. For each question, AGO searches your documents and a judge decides whether the retrieved documents actually answer it. A domain’s coverage is the share of its questions that are answered.

Reference questions are plain text, so the same referential works for any company. You can keep a referential as a JSON file outside AGO and import it whenever you want to measure coverage.

Access

Go to Knowledge > Knowledge Coverage.

Import a referential

1. Click Import referential (JSON).
2. Paste your referential JSON and click Import.

The JSON has a name and a list of domains, each with its questions:

{
  "name": "Legal baseline",
  "version": "1.0",
  "domains": [
    {
      "name": "Legal",
      "questions": [
        "Can a customer cancel without notice?",
        "What is the withdrawal period?"
      ]
    },
    {
      "name": "HR",
      "questions": ["How many days of paid leave per year?"]
    }
  ]
}

Manage referentials

The Referentials table lists everything you have imported or built, with each referential’s question count and its last coverage percentage. Click a row to make it the active one — the coverage bars and gaps below reflect the selected referential. From the same row you can run its evaluation, explore its questions, export it back to JSON, or delete it (deleting also removes its questions and past results). Each referential is scored on its own, so you get a focused percentage per referential rather than one number for everything.

Explore and organize questions

Click the Explore questions icon on a referential to open its question list. The questions are shown in a paginated table so large referentials stay easy to browse.

On this page you can reorganize the questions by category (domain):

• Filter by category to focus on one category at a time.
• Move a question to another category using the category dropdown on its row.
• Rename a category — select a category in the filter, then click Rename category. Every question in it moves to the new name.
• Add a category to create a new name, then move questions into it. A category with no questions is not kept, so move at least one question in for it to stick.

Build a referential from your conversations

Instead of writing questions by hand, you can build referentials from the questions your users actually ask:

1. Click Create questions from last 7 days to mine recent conversations. AGO extracts the business questions and merges near-identical phrasings into one. A progress bar shows how many conversations have been analyzed, and you can click Stop to end the run early — the conversations already analyzed are kept.
2. Turn on Daily automatic analysis to keep doing this every day for new conversations.
3. Click Update referentials from conversations. AGO groups the mined questions by domain — using each question’s topic, or a domain it assigns when the question has none — and builds one referential per domain.

Running it again feeds the existing referentials with any new questions and creates a referential for any new domain, without duplicating questions. You then run each referential’s evaluation from its row.

The mined questions themselves are browsable via View extracted questions on the Coverage page, with how many times each was asked and its domains. You can filter them by domain, by the agent that handled the conversation, and by the date the question was asked. Click a question to see the conversations it came from, each with a link to open the conversation.

Run an evaluation

Click the Run evaluation button on a referential’s row. Scoring runs in the background because it checks every question against your documents. While it runs, the row shows a live progress bar (questions checked so far). When it finishes, the row shows its coverage percentage, and selecting it shows a coverage bar for each domain below.

Read the results

• Per-domain bars show the coverage percentage and how many questions are answered (for example, 8/10).
• Coverage gaps lists the questions your knowledge base does not answer yet, with the reason. Use this list to decide which documents to add or improve.

After you add the missing documents, run the evaluation again to see the coverage rise.

Access from external agents

Coverage data is also available through the API and the MCP server, so an external agent can read the per-domain coverage, the reference questions, and the gaps. This requires an API key with the coverage:read scope.