Guide · web to Markdown
Read web pages as Markdown from your AI agent
LLMs read Markdown far better than raw HTML. These services fetch a page, drop menus, ads and footers and return the main content as clean Markdown with title and meta description. One page costs $0.0025, up to 20 pages in one call $0.05.
Which service
- One URL: web-scrape ($0.0025). Markdown and text files such as
llms.txtcome back unchanged. - A list of URLs: web-scrape-batch ($0.05 for 1 to 20 URLs, one payment, fetched in parallel). Charged only if at least one page was read.
- A whole site or section: site-crawl ($0.005) follows links from a start URL and returns up to 20 pages.
- A JavaScript app that comes back almost empty: call web-scrape with
render: true(real browser, plain text).
Try it free
curl -X POST https://agentsvc.io/api/v1/try/web-scrape -H "Content-Type: application/json" -d '{"url":"https://example.com"}'JavaScript
npm i @x402/fetch @x402/evm viem
import { wrapFetchWithPayment, x402Client } from "@x402/fetch";
import { registerExactEvmScheme } from "@x402/evm/exact/client";
import { privateKeyToAccount } from "viem/accounts";
const client = new x402Client();
registerExactEvmScheme(client, { signer: privateKeyToAccount(process.env.EVM_PRIVATE_KEY) });
const payFetch = wrapFetchWithPayment(fetch, client);
const res = await payFetch("https://agentsvc.io/api/v1/proxy/web-scrape", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url: "https://example.com/blog/post", max_chars: 20000 }),
});
console.log(await res.json());Python
pip install "x402[httpx]" eth_account
from eth_account import Account
from x402 import x402Client
from x402.http.clients import x402HttpxClient
from x402.mechanisms.evm import EthAccountSigner
from x402.mechanisms.evm.exact.register import register_exact_evm_client
client = x402Client()
register_exact_evm_client(client, EthAccountSigner(Account.from_key(PRIVATE_KEY)))
async with x402HttpxClient(client) as http:
r = await http.post("https://agentsvc.io/api/v1/proxy/web-scrape", json={"url": "https://example.com/blog/post", "max_chars": 20000})
await r.aread()
print(r.json())Many pages in one call
curl -X POST https://agentsvc.io/api/v1/proxy/web-scrape-batch \
-H "Authorization: Bearer asvc_cr_..." -H "Content-Type: application/json" \
-d '{"urls":["https://example.com/a","https://example.com/b","https://example.com/c"],"max_chars":8000}'A page that fails is reported with its error while the others still return. Find the URLs of a site first with site-map.
Paying
No account and no API key. Two ways to pay, both only for calls that succeed:
- Wallet (x402): a separate wallet with a few dollars of USDC on Base, Solana, Polygon or Arbitrum. The x402 client pays the price from the 402 answer and retries; you pay no gas.
- Credit token: buy one at /credits from $0.10 (card or USDC, +10% bonus) and send it as
Authorization: Bearer. Good for a script that runs through a long list.
Details for every client, MCP and frameworks: docs, LangChain, Vercel AI SDK.