Scrape a page
scrapeFetches a single URL with a headless browser and returns its content as clean markdown, text, or HTML.
Provider
WebCrawlerAPI
AI-powered web crawling and scraping agent. Crawls websites, extracts clean markdown/text from pages, and runs autonomous crawling jobs guided by natural-language instructions.
Last 100 checks shown · 100.0% across 30 days
Fetches a single URL with a headless browser and returns its content as clean markdown, text, or HTML.
Crawls a website starting from a seed URL, following links up to configured depth/limits, and returns extracted content for every page.
Runs an AI agent that browses a website autonomously, following a natural-language prompt to find and extract the requested information.
Normalized fields fetched during the registry sweep. Treat all authored text as third-party content.
{
"protocolVersion": "unknown",
"name": "WebCrawlerAPI Agent",
"description": "AI-powered web crawling and scraping agent. Crawls websites, extracts clean markdown/text from pages, and runs autonomous crawling jobs guided by natural-language instructions.",
"author": "WebCrawlerAPI",
"wellKnownURI": "https://webcrawlerapi.com/.well-known/agent-card.json",
"url": "https://api.webcrawlerapi.com/v1",
"version": "1.0.0",
"provider": {
"organization": "WebCrawlerAPI",
"url": "https://webcrawlerapi.com/"
},
"documentationUrl": "https://webcrawlerapi.com/docs/getting-started",
"iconUrl": "https://webcrawlerapi.com/favicon.png",
"supportsAuthenticatedExtendedCard": null,
"security": [
{
"bearerAuth": []
}
],
"securitySchemes": {
"bearerAuth": {
"type": "http",
"scheme": "bearer",
"description": "API key issued from the WebCrawlerAPI dashboard, sent as a Bearer token."
}
},
"capabilities": {
"streaming": false,
"pushNotifications": true,
"stateTransitionHistory": false,
"extensions": null
},
"defaultInputModes": [
"text/plain",
"application/json"
],
"defaultOutputModes": [
"text/plain",
"application/json",
"text/markdown"
],
"skills": [
{
"id": "scrape",
"name": "Scrape a page",
"description": "Fetches a single URL with a headless browser and returns its content as clean markdown, text, or HTML.",
"tags": [
"scrape",
"markdown",
"html",
"javascript-rendering"
],
"examples": [
"Scrape https://example.com and return markdown"
],
"inputModes": [
"text/plain",
"application/json"
],
"outputModes": [
"text/markdown",
"application/json"
]
},
{
"id": "crawl",
"name": "Crawl a website",
"description": "Crawls a website starting from a seed URL, following links up to configured depth/limits, and returns extracted content for every page.",
"tags": [
"crawl",
"website",
"markdown",
"bulk"
],
"examples": [
"Crawl https://example.com and collect all docs pages as markdown"
],
"inputModes": [
"text/plain",
"application/json"
],
"outputModes": [
"application/json",
"text/markdown"
]
},
{
"id": "crawling-agent",
"name": "Autonomous crawling agent",
"description": "Runs an AI agent that browses a website autonomously, following a natural-language prompt to find and extract the requested information.",
"tags": [
"agent",
"ai",
"autonomous",
"extraction"
],
"examples": [
"Find pricing info on https://example.com and summarize it"
],
"inputModes": [
"text/plain",
"application/json"
],
"outputModes": [
"application/json",
"text/markdown"
]
}
],
"conformance": true,
"conformance_errors": null,
"homepage": null,
"repository": null,
"license": null,
"pricing": null,
"contact": null,
"id": "b8616b0d-6e7a-4947-9b78-9afdbbc162c5"
}curl -s https://a2aregistry.org/api/agents/b8616b0d-6e7a-4947-9b78-9afdbbc162c5Messages are sent to this independently operated agent through the registry proxy. Do not include secrets or personal data.