Web Scrape Batch (up to 20 URLs to Markdown)

Bulk web scraper: read up to 20 web pages in ONE paid call, each as clean, LLM-ready Markdown with title and meta description. One payment instead of 20, pages fetched in parallel, duplicate URLs read once. Use when your agent works through a list of URLs (search results, product pages, a sitemap, a link list). A failed page is reported with its error while the others still return; charged only if at least one page was read. For a single URL use web-scrape, to find the URLs of one site first use site-map. Params: urls (1 to 20, required), max_chars per page (500 to 30000, default 8000), include_links.

Call it

import { wrapFetchWithPayment, x402Client } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";

const client = new x402Client();
registerExactEvmScheme(client, { signer: privateKeyToAccount(process.env.EVM_PRIVATE_KEY) });
const payFetch = wrapFetchWithPayment(fetch, client);

const res = await payFetch("https://agentsvc.io/api/v1/proxy/web-scrape-batch", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({"urls":["https://example.com/","https://agentsvc.io/","https://agentsvc.io/docs"],"max_chars":3000}),
});
const { data, payment } = await res.json();  // payment.status "settling"; header X-Payment-Settle: sync returns payment.transaction

Input

Output

Example response (data)

{
  "pages": [
    {
      "url": "https://example.com/",
      "ok": true,
      "final_url": "https://example.com/",
      "status": 200,
      "title": "Example Domain",
      "description": "",
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples in documents.",
      "word_count": 21,
      "truncated": false
    },
    {
      "url": "https://agentsvc.io/docs",
      "ok": true,
      "final_url": "https://agentsvc.io/docs",
      "status": 200,
      "title": "Docs | agentsvc.io",
      "description": "How to call agentsvc.io services",
      "markdown": "# Docs\n\n…",
      "word_count": 1840,
      "truncated": true
    },
    {
      "url": "https://nonexistent.invalid/",
      "ok": false,
      "error": "fetch failed"
    }
  ],
  "ok_count": 2,
  "failed_count": 1,
  "fetched_at": "2026-10-20T12:00:00.000Z"
}