Skip to content
Scalekit Docs
Talk to an Engineer Dashboard

Connect AI agents to ScrapingAnt

Scalekit connector
Open markdown

The ScrapingAnt connector lets your AI agent act in each user's ScrapingAnt account. Each user connects their own ScrapingAnt API key once, and Scalekit sends it with every call, so your agent never handles credentials. It comes with 5 tools.

Tools
5
What they doRead · write · destructive
5 · 0 · 05 read0 write0 destructive
Users sign in with

Setup

  1. Install the SDK

    Terminal window
    npm install @scalekit-sdk/node dotenv
  2. Set your credentials

    Add your Scalekit credentials to your .env file. Find values in app.scalekit.com > Developers > API Credentials.

    .env
    SCALEKIT_ENVIRONMENT_URL=<your-environment-url>
    SCALEKIT_CLIENT_ID=<your-client-id>
    SCALEKIT_CLIENT_SECRET=<your-client-secret>
  3. Create the ScrapingAnt connection

    In AgentKit > Connections, create a ScrapingAnt connection. The name you give it is the connection_name your code passes. See Configure connections.

  4. Authorize a user and make your first call

    quickstart.mts
    import { ScalekitClient } from '@scalekit-sdk/node'
    import 'dotenv/config'
    import { createInterface } from 'node:readline/promises'
    const scalekit = new ScalekitClient(
    process.env.SCALEKIT_ENVIRONMENT_URL,
    process.env.SCALEKIT_CLIENT_ID,
    process.env.SCALEKIT_CLIENT_SECRET,
    )
    const actions = scalekit.actions
    const connector = 'scrapingant'
    const identifier = 'user_123'
    // Generate an authorization link for the user
    const { link } = await actions.getAuthorizationLink({ connectionName: connector, identifier })
    console.log('Authorize ScrapingAnt:', link)
    const rl = createInterface({ input: process.stdin, output: process.stdout })
    await rl.question('Press Enter after authorizing...')
    rl.close()
    // Make your first call
    const result = await actions.executeTool({
    connector,
    identifier,
    toolName: 'scrapingant_usage_get',
    toolInput: {},
    })
    console.log(result)
    Terminal window
    npx tsx quickstart.mts

    Each user opens the link once and enters their ScrapingAnt credentials there. See Authorize a user for the full flow and statuses.

Tools

Pass the exact name to execute_tool
Try in PlaygroundRequest a tool
  • scrapingant_extractScrape a web page through ScrapingAnt and extract the data you describe into JSON with AI.Read-only

    Extract Data with AI

    Scrape a web page through ScrapingAnt and extract the data you describe into JSON with AI. Returns a JSON object whose camelCase properties match the fields you asked for. Use this for structured fields such as product title and price. Use scrapingant_scrape_markdown for the whole page text.

    Inputs

    extract_propertiesstringrequired
    Free-text list of the data to extract, usually comma-separated. Nested lists can be described in brackets. Example: product title, price(number), full description, reviews(list: review title, review content)
    urlstringrequired
    Full URL of the page to scrape, including https://. Example: https://example.com.
    block_resourcestring
    A resource type the browser should not load, to speed up scraping. Works only with browser true. Example: image.one of documentstylesheetimagemediafontscripttexttrackxhrfetcheventsourcewebsocketmanifestother
    browserboolean
    Render the page in a headless browser (true, the default) or fetch it without one (false, faster and cheaper). Example: true.
    cookiesstring
    Cookies to send to the target site, as name=value pairs separated by semicolons. Example: session_id=abc123;lang=en.
    js_snippetstring
    Base64-encoded JavaScript to run after the page loads. Works only with browser true. Example: ZG9jdW1lbnQuYm9keS5zdHlsZS5iYWNrZ3JvdW5kQ29sb3IgPSAnd2hpdGUnOw==.
    proxy_countrystring
    Two-letter country code to send the request from. A random country is used when empty. Example: US.
    proxy_typestring
    Proxy pool to use: datacenter (default) or residential (more credits, harder to block). Example: datacenter.one of datacenterresidential
    return_page_sourceboolean
    Return the HTML as the server sent it, before JavaScript runs. Works only with browser true. Defaults to false. Example: false.
    timeoutinteger
    Maximum seconds the scrape may run, from 5 to 60. Defaults to 60. Example: 60.
    wait_for_selectorstring
    CSS selector to wait for before returning the page. Works only with browser true. Example: div.product-list.
  • scrapingant_scrape_extendedScrape a web page through ScrapingAnt and return the page with its response details.Read-only

    Scrape Page (Extended JSON)

    Scrape a web page through ScrapingAnt and return the page with its response details. Returns JSON with html, text, cookies, status_code, headers, the XHR requests the page made and iframes. Use this when you need the target's status code, cookies or background requests. Use scrapingant_scrape_html for HTML only.

    Inputs

    urlstringrequired
    Full URL of the page to scrape, including https://. Example: https://example.com.
    block_resourcestring
    A resource type the browser should not load, to speed up scraping. Works only with browser true. Example: image.one of documentstylesheetimagemediafontscripttexttrackxhrfetcheventsourcewebsocketmanifestother
    browserboolean
    Render the page in a headless browser (true, the default) or fetch it without one (false, faster and cheaper). Example: true.
    cookiesstring
    Cookies to send to the target site, as name=value pairs separated by semicolons. Example: session_id=abc123;lang=en.
    js_snippetstring
    Base64-encoded JavaScript to run after the page loads. Works only with browser true. Example: ZG9jdW1lbnQuYm9keS5zdHlsZS5iYWNrZ3JvdW5kQ29sb3IgPSAnd2hpdGUnOw==.
    proxy_countrystring
    Two-letter country code to send the request from. A random country is used when empty. Example: US.
    proxy_typestring
    Proxy pool to use: datacenter (default) or residential (more credits, harder to block). Example: datacenter.one of datacenterresidential
    return_page_sourceboolean
    Return the HTML as the server sent it, before JavaScript runs. Works only with browser true. Defaults to false. Example: false.
    timeoutinteger
    Maximum seconds the scrape may run, from 5 to 60. Defaults to 60. Example: 60.
    wait_for_selectorstring
    CSS selector to wait for before returning the page. Works only with browser true. Example: div.product-list.
  • scrapingant_scrape_htmlScrape a web page through ScrapingAnt's headless browsers and rotating proxies and return its HTML.Read-only

    Scrape Page HTML

    Scrape a web page through ScrapingAnt's headless browsers and rotating proxies and return its HTML. Returns the page's HTML as text. Use this for raw HTML. Use scrapingant_scrape_markdown for LLM-ready text, scrapingant_scrape_extended for HTML plus status, cookies and headers, or scrapingant_extract for structured fields.

    Inputs

    urlstringrequired
    Full URL of the page to scrape, including https://. Example: https://example.com.
    block_resourcestring
    A resource type the browser should not load, to speed up scraping. Works only with browser true. Example: image.one of documentstylesheetimagemediafontscripttexttrackxhrfetcheventsourcewebsocketmanifestother
    browserboolean
    Render the page in a headless browser (true, the default) or fetch it without one (false, faster and cheaper). Example: true.
    cookiesstring
    Cookies to send to the target site, as name=value pairs separated by semicolons. Example: session_id=abc123;lang=en.
    js_snippetstring
    Base64-encoded JavaScript to run after the page loads. Works only with browser true. Example: ZG9jdW1lbnQuYm9keS5zdHlsZS5iYWNrZ3JvdW5kQ29sb3IgPSAnd2hpdGUnOw==.
    proxy_countrystring
    Two-letter country code to send the request from. A random country is used when empty. Example: US.
    proxy_typestring
    Proxy pool to use: datacenter (default) or residential (more credits, harder to block). Example: datacenter.one of datacenterresidential
    return_page_sourceboolean
    Return the HTML as the server sent it, before JavaScript runs. Works only with browser true. Defaults to false. Example: false.
    timeoutinteger
    Maximum seconds the scrape may run, from 5 to 60. Defaults to 60. Example: 60.
    wait_for_selectorstring
    CSS selector to wait for before returning the page. Works only with browser true. Example: div.product-list.
  • scrapingant_scrape_markdownScrape a web page through ScrapingAnt and convert it into clean Markdown for LLMs and retrieval.Read-only

    Scrape Page as Markdown

    Scrape a web page through ScrapingAnt and convert it into clean Markdown for LLMs and retrieval. Returns JSON with url and markdown. Use this to read a page's content. Use scrapingant_extract to pull specific fields as JSON.

    Inputs

    urlstringrequired
    Full URL of the page to scrape, including https://. Example: https://example.com.
    block_resourcestring
    A resource type the browser should not load, to speed up scraping. Works only with browser true. Example: image.one of documentstylesheetimagemediafontscripttexttrackxhrfetcheventsourcewebsocketmanifestother
    browserboolean
    Render the page in a headless browser (true, the default) or fetch it without one (false, faster and cheaper). Example: true.
    cookiesstring
    Cookies to send to the target site, as name=value pairs separated by semicolons. Example: session_id=abc123;lang=en.
    js_snippetstring
    Base64-encoded JavaScript to run after the page loads. Works only with browser true. Example: ZG9jdW1lbnQuYm9keS5zdHlsZS5iYWNrZ3JvdW5kQ29sb3IgPSAnd2hpdGUnOw==.
    proxy_countrystring
    Two-letter country code to send the request from. A random country is used when empty. Example: US.
    proxy_typestring
    Proxy pool to use: datacenter (default) or residential (more credits, harder to block). Example: datacenter.one of datacenterresidential
    return_page_sourceboolean
    Return the HTML as the server sent it, before JavaScript runs. Works only with browser true. Defaults to false. Example: false.
    timeoutinteger
    Maximum seconds the scrape may run, from 5 to 60. Defaults to 60. Example: 60.
    wait_for_selectorstring
    CSS selector to wait for before returning the page. Works only with browser true. Example: div.product-list.
  • scrapingant_usage_getGet your ScrapingAnt subscription and remaining API credits.Read-only

    Get API Credit Usage

    Get your ScrapingAnt subscription and remaining API credits. Returns plan_name, start_date, end_date, plan_total_credits and remained_credits. Use this before large scraping jobs.

    Inputs

    This tool takes no inputs.