Build Your First Support Agent

Step-by-step tutorial to create a working support agent from scratch

This tutorial walks you through creating a support agent from scratch. By the end, you’ll have a working agent that answers questions from your knowledge base, creates tickets for unresolved issues, and can be tested with automated simulations.

What You’ll Build

A customer support agent that:

• Answers questions using your documentation
• Creates support tickets when it can’t resolve an issue
• Responds in a friendly, professional tone
• Can be tested and debugged before going live

Prerequisites

• Admin access to your AGO platform
• At least one knowledge source connected and synced (see Knowledge Setup Tutorial)
• An LLM model configured in your platform settings

---

1. Create a Prompt Template

   The prompt template defines how your agent behaves — its personality, rules, and response style.

   1. Navigate to AI Studio > Prompt Templates
   2. Click Create Template
   3. Enter a Name (e.g., support-agent-v1)
   4. In the prompt editor, paste this starting template:

   You are a friendly customer support assistant for our company.
   
   ## Your Role
   Help customers with product questions, troubleshooting, and account inquiries.
   
   ## Guidelines
   - Be warm, empathetic, and patient
   - Acknowledge the customer's concern before solving it
   - Provide step-by-step instructions when needed
   - If you don't know the answer, say so honestly
   
   ## Knowledge Usage
   Search the knowledge base for relevant documentation.
   Always cite the source article when referencing documentation.
   If information isn't available, offer to create a support ticket.
   
   ## Escalation
   Create a ticket when:
   - The issue requires account access you don't have
   - The customer explicitly asks for human help
   - You've tried troubleshooting without success
   
   ## Response Format
   - Keep responses concise — aim for 2-3 paragraphs maximum
   - Use bullet points for lists
   - Include links to relevant documentation when available
   

   5. Click Save

   Start simple. You can always refine the prompt after testing.
2. Create the Agent

   1. Navigate to AI Studio > Agents
   2. Click Create Agent
   3. In the General tab, configure:
      • Name: support-agent (internal identifier)
      • Display Name: Support Assistant (what users see)
      • Icon: Pick an emoji (e.g., 💬)
      • Prompt Template: Select the support-agent-v1 template you just created

   Configure Knowledge (Context Tab)

   4. Switch to the Context tab
   5. Under Knowledge Sources, select the sources containing your support documentation
   6. Leave Documents to Retrieve at the default (8) — you can tune this later

   Configure Tools (Capabilities Tab)

   7. Switch to the Capabilities tab
   8. The ticketing tool is added automatically to all agents by default. If you’ve disabled this global setting, add the Ticketing Tool manually from the tools list.

   Set Permissions (Access Control Tab)

   9. Switch to the Access Control tab

   10. Assign the agent to the appropriate user groups (e.g., Default to make it available to all users)

   11. Click Save

3. Test Manually

   Before running automated tests, verify the agent works with a few manual conversations.

   1. Open the chat widget or portal
   2. Select your Support Assistant agent
   3. Try these test messages:

   Message	What to check
   A question your docs cover	Does it find and cite the right article?
   A question your docs don’t cover	Does it admit it doesn’t know?
   ”I need to talk to someone”	Does it create a ticket?
   A vague question	Does it ask for clarification?

   If the agent can’t find relevant content, check that:

   • The knowledge source is synced (green status)
   • The source is assigned to this agent in the Context tab
   • Your documents actually cover the topic
4. Debug with Search Lab

   Use the AI Debugger to understand why the agent retrieves certain documents.

   1. Navigate to AI Studio > Search Lab
   2. Select your agent
   3. Enter a test query (e.g., “How do I reset my password?”)
   4. Click Run Search
   5. Review the results:
      • Similarity score: How closely each document matches (0–1)
      • Reranked score: Score after reranking (if enabled)
      • Chunk: The text segment retrieved

   If relevant documents rank low:

   • Check your document titles and headings — they help retrieval
   • Consider enabling the Reranker in the agent’s Context tab
   • Try rephrasing the query to match how users actually ask
5. Create a Simulation Test

   Simulations let you test your agent automatically against a dataset of expected interactions.

   1. Navigate to AI Studio > Agent Lab
   2. Click Create Dataset
   3. Add test cases — each with a user message and what you expect:

   User Message	Expected Behavior
   ”How do I change my password?”	Answers from knowledge base
   ”My account was charged twice”	Creates a support ticket
   ”What’s the weather today?”	Says it can’t help with that
   ”I’m really frustrated, nothing works”	Acknowledges frustration, offers ticket

   4. Save the dataset
   5. Click Run Simulation
   6. Review the results — check that each response matches your expectations
6. Iterate and Improve

   Based on your testing:

   If the agent gives wrong answers

   • Refine the prompt template — add specific instructions for problem topics
   • Check if the knowledge base has accurate, up-to-date content
   • Use Search Lab to debug retrieval

   If the agent creates tickets too often

   • Adjust the prompt: “Only create tickets when you’ve exhausted all knowledge base answers”
   • Review the ticketing tool’s prompt to make trigger conditions clearer

   If responses are too long

   • Add to the prompt: “Keep responses under 3 paragraphs”
   • Lower the Temperature setting in the agent’s General tab (try 0.1–0.3)

   If responses vary too much

   • Lower the Temperature to reduce randomness
   • Add pin documents with authoritative answers for critical topics

What’s Next

You now have a working support agent. Here are ways to extend it:

• Add more tools — Connect external APIs, send emails, or display rich UI components
• Set up multi-agent routing — Create specialized agents for different domains
• Write better prompts — Advanced prompt engineering techniques
• Monitor quality — Run simulations regularly to catch regressions
• Enable satisfaction feedback — Collect user feedback after responses

---

Related

• Agents — Full agent configuration reference
• Prompt Templates — Template management and versioning
• Agent Best Practices — Design patterns and common pitfalls
• AI Debugger — Troubleshoot retrieval and responses