services / site-map
weboperational · 15 ms
Site Map (URL discovery)
List the URLs of a website without reading them: robots.txt, sitemaps and sitemap indexes plus homepage links, up to 1,000 same-site URLs with lastmod and source. A cheap first step to decide which pages to read with web-read-batch or site-crawl. Returns no page content. Params: url (site or domain, required), max_urls (1 to 1000, default 200).
Run free trial ↗1 free call per day with the example input. Paid: $0.003 USDC, no limit.
Call it
Input
| Field | Type | Description |
|---|---|---|
| url * | string | Site URL or domain |
| max_urls | integer = 200 |
Output
| Field | Type | Description |
|---|---|---|
| site | string | |
| count | integer | |
| sitemaps | array | |
| from_sitemap | integer | |
| urls | array | |
| robots_txt_found | boolean | |
| checked_at | string |
Example response (data)
{
"site": "https://agentsvc.io",
"count": 3,
"sitemaps": [
"https://agentsvc.io/sitemap.xml"
],
"from_sitemap": 3,
"urls": [
{
"url": "https://agentsvc.io/",
"lastmod": "2026-10-06T14:17:50.554Z",
"source": "sitemap"
},
{
"url": "https://agentsvc.io/services",
"lastmod": "2026-10-06T14:17:50.554Z",
"source": "sitemap"
}
],
"robots_txt_found": true,
"checked_at": "2026-10-06T14:17:50.557Z"
}