Mintlify

Import documentation from Mintlify-powered sites into AGO

Import documentation from Mintlify-powered documentation sites into AGO. This connector uses web scraping to extract content, so no API key is required.

Overview

Feature	Details
Authentication	None required
Content Type	Developer documentation
Method	Web scraping
Hierarchy	From navigation config or URL structure

Prerequisites

• A publicly accessible Mintlify documentation site
• The base URL of the documentation (e.g., docs.example.com)

Configuration

Required Fields

Field	Description	Example
domain	Base URL of the Mintlify docs site	docs.example.com

Optional Fields

Field	Description	Example
include_patterns	URL patterns to include (fnmatch)	["/api/*", "/guides/*"]
exclude_patterns	URL patterns to exclude (fnmatch)	["/changelog/*", "/blog/*"]

Setup Steps

1. Navigate to Knowledge → Sources
2. Click Create Source
3. Select Mintlify as the source type
4. Enter the documentation domain
5. (Optional) Configure include/exclude patterns
6. Click Save
7. Click Sync to start importing

URL Pattern Filtering

Use fnmatch-style patterns to control what gets imported:

Include Patterns

Only import pages matching these patterns:

{
  "include_patterns": [
    "/api/*",
    "/guides/*",
    "/tutorials/*"
  ]
}

Exclude Patterns

Skip pages matching these patterns:

{
  "exclude_patterns": [
    "/changelog/*",
    "/blog/*",
    "/internal/*"
  ]
}

Pattern Examples

Pattern	Matches
/api/*	/api/authentication, /api/endpoints
/guides/getting-started	Exact match only
*/v2/*	Any path containing /v2/

How It Works

1. Discovery: Fetches the sitemap to find all pages
2. Filtering: Applies include/exclude patterns
3. Hierarchy: Uses the site’s navigation config (docs.json or mint.json) to build the sidebar structure. If no config is available, infers hierarchy from URL paths
4. Extraction: Fetches markdown content from each page
5. Storage: Saves documents with metadata and links to the original pages

Limitations

• Only works with publicly accessible sites
• May be affected by rate limiting
• Complex JavaScript-rendered content may not extract fully
• Respects robots.txt directives

Troubleshooting

Pages Not Found

• Verify the domain is correct and accessible
• Check if the site requires authentication
• Ensure pages aren’t blocked by robots.txt

Missing Content

• Some dynamic content may not be captured
• Check if content is loaded via JavaScript
• Review exclude patterns aren’t too broad

Sync Takes Too Long

• Large sites may take time to crawl
• Consider using include patterns to limit scope
• Check for redirect loops

Related Documentation

• Knowledge Connectors - All available connectors
• Knowledge Source Management - Source configuration