Skip to content
Scalekit Docs
Talk to an Engineer Dashboard

Connect AI agents to Diffbot

Scalekit connector
Open markdown

The Diffbot connector lets your AI agent act in each user's Diffbot account. Each user connects their own Diffbot API key once, and Scalekit sends it with every call, so your agent never handles credentials. It comes with 39 tools.

Tools
39
What they doRead · write · destructive
25 · 9 · 525 read9 write5 destructive
Users sign in with

Setup

  1. Install the SDK

    Terminal window
    npm install @scalekit-sdk/node dotenv
  2. Set your credentials

    Add your Scalekit credentials to your .env file. Find values in app.scalekit.com > Developers > API Credentials.

    .env
    SCALEKIT_ENVIRONMENT_URL=<your-environment-url>
    SCALEKIT_CLIENT_ID=<your-client-id>
    SCALEKIT_CLIENT_SECRET=<your-client-secret>
  3. Create the Diffbot connection

    In AgentKit > Connections, create a Diffbot connection. The name you give it is the connection_name your code passes. See Configure connections.

  4. Authorize a user and make your first call

    quickstart.mts
    import { ScalekitClient } from '@scalekit-sdk/node'
    import 'dotenv/config'
    import { createInterface } from 'node:readline/promises'
    const scalekit = new ScalekitClient(
    process.env.SCALEKIT_ENVIRONMENT_URL,
    process.env.SCALEKIT_CLIENT_ID,
    process.env.SCALEKIT_CLIENT_SECRET,
    )
    const actions = scalekit.actions
    const connector = 'diffbot'
    const identifier = 'user_123'
    // Generate an authorization link for the user
    const { link } = await actions.getAuthorizationLink({ connectionName: connector, identifier })
    console.log('Authorize Diffbot:', link)
    const rl = createInterface({ input: process.stdin, output: process.stdout })
    await rl.question('Press Enter after authorizing...')
    rl.close()
    // Make your first call
    const result = await actions.executeTool({
    connector,
    identifier,
    toolName: 'diffbot_account_get',
    toolInput: {},
    })
    console.log(result)
    Terminal window
    npx tsx quickstart.mts

    Each user opens the link once and enters their Diffbot credentials there. See Authorize a user for the full flow and statuses.

Tools

Pass the exact name to execute_tool
Try in PlaygroundRequest a tool
  • diffbot_account_getGet the Diffbot token's account details and API usage.Read-only

    Get Account

    Get the Diffbot token's account details and API usage. Returns the plan, token status, credit usage and daily API call volumes. Use it to check the token works and how many credits are left.

    Inputs

    daysinteger
    Number of days of API call volume to return. Defaults to 31. Example: 31.
  • diffbot_bulk_data_getDownload a bulk job's extracted data, or its URL report.Read-only

    Get Bulk Data

    Download a bulk job's extracted data, or its URL report. Returns the extracted objects as JSON (or CSV). Use List Bulk Jobs to check the job is done.

    Inputs

    namestringrequired
    Name of the job, as given when it was created. Example: acme-blog-crawl.
    formatstring
    csv for CSV with top-level fields only. Defaults to JSON. Example: csv.one of csv
    numinteger
    Only the most recently processed N URLs. Example: 100.
    typestring
    urls returns the URL report (CSV) instead of extracted data. Example: urls.one of urls
  • diffbot_bulk_jobs_listList the token's bulk jobs, or one job by name, with their status.Read-only

    List Bulk Jobs

    List the token's bulk jobs, or one job by name, with their status. Returns jobs with name, status, URLs processed and download URLs. Use Get Bulk Data to download a job's results.

    Inputs

    namestring
    Only return this job. Example: acme-products-batch.
  • diffbot_crawl_data_getDownload a crawl job's extracted data, or its URL report.Read-only

    Get Crawl Data

    Download a crawl job's extracted data, or its URL report. Returns the extracted objects as JSON (or CSV). Use Search with DQL (type crawl) to query a subset across crawl jobs.

    Inputs

    namestringrequired
    Name of the job, as given when it was created. Example: acme-blog-crawl.
    formatstring
    csv for CSV with top-level fields only. Defaults to JSON. Example: csv.one of csv
    numinteger
    Only the most recently processed N URLs. Example: 100.
    typestring
    urls returns the URL report (CSV) instead of extracted data. Example: urls.one of urls
  • diffbot_crawl_jobs_listList the token's crawl jobs, or one job by name, with their status.Read-only

    List Crawl Jobs

    List the token's crawl jobs, or one job by name, with their status. Returns jobs with name, status, pages crawled and processed, limits and download URLs. Use Get Crawl Data to download a job's results.

    Inputs

    namestring
    Only return this job. Example: acme-blog-crawl.
  • diffbot_custom_api_extractExtract a page with one of your Custom APIs, which apply your own CSS-selector rules.Read-only

    Extract with Custom API

    Extract a page with one of your Custom APIs, which apply your own CSS-selector rules. Returns an objects array shaped by your rules. Use List Custom API Rules to see your Custom APIs.

    Inputs

    apistringrequired
    Name of your Custom API (the name you gave its ruleset). Example: my-store-products.
    urlstringrequired
    URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
    fieldsstring
    Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
    proxystring
    IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
    proxy_authstring
    Credentials for your own proxy, as username:password. Example: user:pass.
    timeoutinteger
    Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
    use_proxystring
    default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default.one of defaultnone
  • diffbot_custom_api_rules_listList the custom extraction rulesets defined on the token.Read-only

    List Custom API Rules

    List the custom extraction rulesets defined on the token. Returns each ruleset with its API, URL pattern, test URL, rules and prefilters. Use Save Custom API Rules to add or change one.

    Inputs

    This tool takes no inputs.

  • diffbot_dql_report_getGet a DQL coverage report by its id.Read-only

    Get DQL Coverage Report

    Get a DQL coverage report by its id. Returns per-field coverage for the query. Use Create DQL Coverage Report to make one.

    Inputs

    report_idstringrequired
    Report id, from Create DQL Coverage Report. Example: rpt_9f8e7d6c.