> **Building with AI coding agents?** Install the authstack plugin with one command. This equips your agent with accurate Scalekit implementation patterns.
>
> **Recommended** (the `-y` flags skip prompts, so the command runs without an interactive terminal):
> ```bash
> npx -y @scalekit-inc/cli setup -y
> ```
>
> Global:
> ```bash
> npm install -g @scalekit-inc/cli
> scalekit setup -y
> ```
>
> Supports Claude Code, Cursor, GitHub Copilot, Codex + skills for other Agent Skills-compatible agents in `~/.agents/skills`.
> Skills: integrate-agentkit, implement-saaskit, add-mcp-oauth, implement-sso, implement-scim.
> [Full setup guide](https://docs.scalekit.com/dev-kit/build-with-ai/)

---

# Connect AI agents to Diffbot

The Diffbot connector lets your AI agent act in each user's Diffbot account. Each user connects their own Diffbot API key once, and Scalekit sends it with every call, so your agent never handles credentials. It comes with 39 tools.

**Authentication:** API Key
**Categories:** AI, Search
**Tools:** 39: 25 read, 9 write, 5 destructive
**Users sign in with:** API key
**Built by:** Scalekit connector
**Try it:** [Playground in the Scalekit dashboard](https://app.scalekit.com/ws/signup?sk_intent=playground&provider=DIFFBOT)

## Setup

1. ### Install the SDK

   **Node.js**

   ```bash
   npm install @scalekit-sdk/node dotenv
   ```

   **Python**

   ```bash
   pip install scalekit-sdk-python python-dotenv
   ```

2. ### Set your credentials

   Add your Scalekit credentials to your `.env` file. Find values in **[app.scalekit.com](https://app.scalekit.com)** > **Developers** > **API Credentials**.

   ```sh title=".env"
   SCALEKIT_ENVIRONMENT_URL=<your-environment-url>
   SCALEKIT_CLIENT_ID=<your-client-id>
   SCALEKIT_CLIENT_SECRET=<your-client-secret>
   ```

3. ### Create the Diffbot connection

   In **AgentKit > Connections**, create a Diffbot connection. The name you give it is the `connection_name` your code passes. See [Configure connections](/agentkit/connections/).

4. ### Authorize a user and make your first call

   **Node.js** (`quickstart.mts`)

   ```typescript
   import { ScalekitClient } from '@scalekit-sdk/node'
   import 'dotenv/config'
   import { createInterface } from 'node:readline/promises'

   const scalekit = new ScalekitClient(
     process.env.SCALEKIT_ENVIRONMENT_URL,
     process.env.SCALEKIT_CLIENT_ID,
     process.env.SCALEKIT_CLIENT_SECRET,
   )
   const actions = scalekit.actions

   const connector = 'diffbot'
   const identifier = 'user_123'

   // Generate an authorization link for the user
   const { link } = await actions.getAuthorizationLink({ connectionName: connector, identifier })
   console.log('Authorize Diffbot:', link)
   const rl = createInterface({ input: process.stdin, output: process.stdout })
   await rl.question('Press Enter after authorizing...')
   rl.close()

   // Make your first call
   const result = await actions.executeTool({
     connector,
     identifier,
     toolName: 'diffbot_account_get',
     toolInput: {},
   })
   console.log(result)
   ```

   ```bash
   npx tsx quickstart.mts
   ```

   **Python** (`quickstart.py`)

   ```python
   import os
   from scalekit import ScalekitClient
   from dotenv import load_dotenv
   load_dotenv()

   scalekit_client = ScalekitClient(
       env_url=os.getenv("SCALEKIT_ENVIRONMENT_URL"),
       client_id=os.getenv("SCALEKIT_CLIENT_ID"),
       client_secret=os.getenv("SCALEKIT_CLIENT_SECRET"),
   )
   actions = scalekit_client.actions

   connection_name = "diffbot"
   identifier = "user_123"

   # Generate an authorization link for the user
   link_response = actions.get_authorization_link(
       connection_name=connection_name,
       identifier=identifier,
   )
   print("Authorize Diffbot:", link_response.link)
   input("Press Enter after authorizing...")

   # Make your first call
   result = actions.execute_tool(
       tool_input={},
       tool_name="diffbot_account_get",
       connection_name=connection_name,
       identifier=identifier,
   )
   print(result)
   ```

   ```bash
   python quickstart.py
   ```

   Each user opens the link once and enters their Diffbot credentials there. See [Authorize a user](/agentkit/tools/authorize/) for the full flow and statuses.

## Tools

Pass the exact name to `execute_tool`, with an input like each tool's example. To ask for a tool that's missing, use the [request form](https://scalekitsupport.portal.usepylon.com/forms/request-a-connector-tool).

**Node.js**

```typescript
const result = await actions.executeTool({
  toolName: 'diffbot_account_get',
  toolInput: {},
  connector: 'diffbot',
  identifier: 'user_123',
})
```

**Python**

```python
result = actions.execute_tool(
    tool_name="diffbot_account_get",
    tool_input={},
    connection_name="diffbot",
    identifier="user_123",
)
```

`result.data` is the app's response as JSON, and `result.execution_id` (`executionId` in Node.js) is the ID of the call.

### `diffbot_account_get`

Get Account · Read-only

Get the Diffbot token's account details and API usage.
Returns the plan, token status, credit usage and daily API call volumes.
Use it to check the token works and how many credits are left.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `days` (`integer`, optional): Number of days of API call volume to return. Defaults to 31. Example: 31.

Example input: `{}`

### `diffbot_bulk_data_get`

Get Bulk Data · Read-only

Download a bulk job's extracted data, or its URL report.
Returns the extracted objects as JSON (or CSV).
Use List Bulk Jobs to check the job is done.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `format` (`string`, optional): csv for CSV with top-level fields only. Defaults to JSON. Example: csv. One of: `csv`.
- `num` (`integer`, optional): Only the most recently processed N URLs. Example: 100.
- `type` (`string`, optional): urls returns the URL report (CSV) instead of extracted data. Example: urls. One of: `urls`.

Example input: `{"name":"<name>"}`

### `diffbot_bulk_jobs_list`

List Bulk Jobs · Read-only

List the token's bulk jobs, or one job by name, with their status.
Returns jobs with name, status, URLs processed and download URLs.
Use Get Bulk Data to download a job's results.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `name` (`string`, optional): Only return this job. Example: acme-products-batch.

Example input: `{}`

### `diffbot_crawl_data_get`

Get Crawl Data · Read-only

Download a crawl job's extracted data, or its URL report.
Returns the extracted objects as JSON (or CSV).
Use Search with DQL (type crawl) to query a subset across crawl jobs.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `format` (`string`, optional): csv for CSV with top-level fields only. Defaults to JSON. Example: csv. One of: `csv`.
- `num` (`integer`, optional): Only the most recently processed N URLs. Example: 100.
- `type` (`string`, optional): urls returns the URL report (CSV) instead of extracted data. Example: urls. One of: `urls`.

Example input: `{"name":"<name>"}`

### `diffbot_crawl_jobs_list`

List Crawl Jobs · Read-only

List the token's crawl jobs, or one job by name, with their status.
Returns jobs with name, status, pages crawled and processed, limits and download URLs.
Use Get Crawl Data to download a job's results.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `name` (`string`, optional): Only return this job. Example: acme-blog-crawl.

Example input: `{}`

### `diffbot_custom_api_extract`

Extract with Custom API · Read-only

Extract a page with one of your Custom APIs, which apply your own CSS-selector rules.
Returns an objects array shaped by your rules.
Use List Custom API Rules to see your Custom APIs.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `api` (`string`, required): Name of your Custom API (the name you gave its ruleset). Example: my-store-products.
- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"api":"<api>","url":"<url>"}`

### `diffbot_custom_api_rules_list`

List Custom API Rules · Read-only

List the custom extraction rulesets defined on the token.
Returns each ruleset with its API, URL pattern, test URL, rules and prefilters.
Use Save Custom API Rules to add or change one.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs: none.

Example input: `{}`

### `diffbot_dql_report_get`

Get DQL Coverage Report · Read-only

Get a DQL coverage report by its id.
Returns per-field coverage for the query.
Use Create DQL Coverage Report to make one.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `report_id` (`string`, required): Report id, from Create DQL Coverage Report. Example: rpt_9f8e7d6c.

Example input: `{"report_id":"<report_id>"}`

### `diffbot_dql_search`

Search with DQL · Read-only

Search the Diffbot Knowledge Graph (organizations, people, articles, products and more) or your crawl collections with a DQL query.
Returns hits and a data array of matching entities with their facts.
Use Enhance Entity to match one known person or organization.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `query` (`string`, required): DQL query, for example type:Organization locations.city.name:"San Francisco" nbEmployees>5000. Example: type:Organization name:"Diffbot".
- `cluster` (`string`, optional): Articles only: all tags each article with a clusterId, best keeps one per cluster, dedupe removes near-duplicates. Example: best. One of: `all`, `best`, `dedupe`.
- `col` (`string`, optional): With type crawl: comma-separated crawl job names to search, or all. Example: acme-blog-crawl.
- `filter` (`string`, optional): Semicolon-separated fields to keep in the response, as dot paths or JsonPath, for example $.name;$.homepageUri. Example: $.name;$.homepageUri;$.nbEmployees.
- `filter_exclude` (`string`, optional): Semicolon-separated fields to drop from the response, as dot paths or JsonPath. Example: $.descriptors.
- `from` (`integer`, optional): Index of the first result (not for Article queries). Example: 0.
- `jsonmode` (`string`, optional): extended adds fact origins; id returns only Diffbot IDs and origins. Example: extended. One of: `extended`, `id`.
- `no_dedup_articles` (`boolean`, optional): Return duplicate articles instead of deduplicating them. Example: false.
- `non_canonical_facts` (`boolean`, optional): Also return non-canonical facts. Example: false.
- `size` (`integer`, optional): Maximum results; -1 returns all. Defaults to 50. Example: 10.
- `type` (`string`, optional): query (default) searches the Knowledge Graph; crawl searches your crawl job collections. Example: query. One of: `query`, `crawl`.

Example input: `{"query":"<query>"}`

### `diffbot_enhance`

Enhance Entity · Read-only

Match a person or organization from partial details (name, URL, email, employer and so on) to the Knowledge Graph.
Returns the best matching entities with score and full facts.
Use Search with DQL for open-ended queries. Use Create Enhance Bulk Job for many records.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `type` (`string`, required): Entity type: Person or Organization. Example: Organization. One of: `Person`, `Organization`.
- `custom_id` (`string`, optional): Your own ID, echoed back for correlation. Example: crm-00123.
- `description` (`string`, optional): Free-text description of the entity. Example: Knowledge graph and web extraction company.
- `email` (`array`, optional): Email address(es). Person only. Example: ["jane@example.com"].
- `employer` (`string`, optional): Employer name. Person only. Example: Diffbot.
- `filter` (`string`, optional): Semicolon-separated fields to keep, as dot paths or JsonPath. Example: $.name;$.homepageUri.
- `filter_exclude` (`string`, optional): Semicolon-separated fields to drop. Example: $.descriptors.
- `id` (`string`, optional): Diffbot ID of a known entity. Example: EYX1i02YVPsuT7fPLUYgRhQ.
- `ip` (`string`, optional): IP address of the entity. Example: 203.0.113.10.
- `jsonmode` (`string`, optional): extended adds fact origins. Example: extended. One of: `extended`.
- `location` (`string`, optional): Location of the entity. Example: Menlo Park, California.
- `name` (`array`, optional): Name(s) of the person or organization. Example: ["Diffbot"].
- `non_canonical_facts` (`boolean`, optional): Also return non-canonical facts. Example: false.
- `phone` (`string`, optional): Phone number of the entity. Example: +1 650 555 0100.
- `refresh` (`boolean`, optional): Recrawl the entity's sources and rebuild it before returning. Example: false.
- `school` (`string`, optional): School attended. Person only. Example: Stanford University.
- `search` (`boolean`, optional): Also search the web for sources and merge them in. Example: false.
- `size` (`integer`, optional): Maximum matches to return. Defaults to 1. Example: 1.
- `threshold` (`number`, optional): Minimum similarity score for a match. Example: 0.5.
- `title` (`string`, optional): Job title. Person only. Example: VP Engineering.
- `url` (`array`, optional): Homepage or profile URL(s) of the entity. Example: ["diffbot.com"].

Example input: `{"type":"Person"}`

### `diffbot_enhance_bulk_coverage_report_get`

Get Enhance Coverage Report · Read-only

Get a coverage report for an Enhance bulk job.
Returns per-field coverage of the matched entities.
Use Get Enhance Bulk Results for the records themselves.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `bulkjob_id` (`string`, required): Enhance bulk job ID, returned by Create Enhance Bulk Job. Example: a1b2c3d4e5.
- `report_id` (`string`, required): Enhance coverage report ID. Example: rpt_1a2b3c.

Example input: `{"bulkjob_id":"<bulkjob_id>","report_id":"<report_id>"}`

### `diffbot_enhance_bulk_jobs_list`

List Enhance Bulk Jobs · Read-only

List the token's Enhance bulk jobs with their status.
Returns each job's ID, name, status and record counts.
Use Get Enhance Bulk Job Status for one job.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs: none.

Example input: `{}`

### `diffbot_enhance_bulk_result_get`

Get Enhance Bulk Result · Read-only

Get the result for one record of an Enhance bulk job.
Returns that record's matches with score and entity.
Use Get Enhance Bulk Results for all records.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `bulkjob_id` (`string`, required): Enhance bulk job ID, returned by Create Enhance Bulk Job. Example: a1b2c3d4e5.
- `job_index` (`integer`, required): Index of the record within the bulk job. Example: 0.

Example input: `{"bulkjob_id":"<bulkjob_id>","job_index":0}`

### `diffbot_enhance_bulk_results_get`

Get Enhance Bulk Results · Read-only

Download the results of a completed Enhance bulk job.
Returns one result per record, each with score and matched entity (JSON lines by default).
Use Get Enhance Bulk Result for one record.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `bulkjob_id` (`string`, required): Enhance bulk job ID, returned by Create Enhance Bulk Job. Example: a1b2c3d4e5.
- `filter` (`string`, optional): Semicolon-separated fields to keep in the response, as dot paths or JsonPath, for example $.name;$.homepageUri. Example: $.name;$.homepageUri;$.nbEmployees.
- `filter_exclude` (`string`, optional): Semicolon-separated fields to drop from the response, as dot paths or JsonPath. Example: $.descriptors.
- `format` (`string`, optional): Output format: json, jsonl (default), csv, xls or xlsx. Example: json. One of: `json`, `jsonl`, `csv`, `xls`, `xlsx`.
- `from` (`integer`, optional): 1-based index of the first result. Example: 1.
- `only_matches` (`string`, optional): true returns only records with a match. Example: true. One of: `true`, `false`.
- `size` (`integer`, optional): Number of results from that position; -1 for all. Example: 100.

Example input: `{"bulkjob_id":"<bulkjob_id>"}`

### `diffbot_enhance_bulk_status_get`

Get Enhance Bulk Job Status · Read-only

Get one Enhance bulk job's status and progress.
Returns its status and counts of processed and matched records.
Use Get Enhance Bulk Results once it completes.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `bulkjob_id` (`string`, required): Enhance bulk job ID, returned by Create Enhance Bulk Job. Example: a1b2c3d4e5.

Example input: `{"bulkjob_id":"<bulkjob_id>"}`

### `diffbot_extract_analyze`

Analyze Page · Read-only

Classify a web page's type (article, product, discussion and so on) and extract it with the matching Extract API.
Returns the detected type and an objects array of structured data.
Use a specific Extract tool (for example Extract Article) when you know the page type.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `discussion` (`boolean`, optional): false skips extracting comments or reviews. Example: false.
- `fallback` (`string`, optional): Extract API to use when the page type is other. Example: article. One of: `article`, `product`, `discussion`, `image`, `video`, `list`.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `mode` (`string`, optional): Force a specific Extract API instead of auto-detection: article, product, discussion, image, video, list, event or job. Example: article. One of: `article`, `product`, `discussion`, `image`, `video`, `list`, `event`, `job`.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_article`

Extract Article · Read-only

Extract the article (title, author, date, text, images, tags) from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with title, author, date, text, html, images, tags, categories and sentiment.
Use Analyze Page when unsure of the page type.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `category_confidence` (`number`, optional): Minimum category score, 0.0 to 1.0. Defaults to 0.5. Example: 0.5.
- `discussion` (`boolean`, optional): false skips extracting comments. Example: false.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring, breadcrumb or quotes. Example: meta,querystring.
- `max_tags` (`integer`, optional): Maximum tags to return. Defaults to 10. Example: 10.
- `natural_language` (`string`, optional): Run the extracted text through the Natural Language API: comma-separated entities, sentiment, summary, facts, openFacts, records, categories, sentences or language. Example: entities,sentiment.
- `paging` (`boolean`, optional): true joins multi-page articles into one result. Example: true.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `summary_num_sentences` (`integer`, optional): Sentences in the summary when natural_language includes summary. Defaults to 3. Example: 3.
- `tag_confidence` (`number`, optional): Minimum tag score, 0.0 to 1.0. Defaults to 0.5. Example: 0.5.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_discussion`

Extract Discussion · Read-only

Extract a forum thread, comments or reviews from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with the thread title and posts, each with author, date, text and votes.
Use Extract Article for the article a comment thread belongs to.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `max_pages` (`string`, optional): Pages of a thread to join, or all. Defaults to 1. Example: all.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_event`

Extract Event · Read-only

Extract event details from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with event title, start and end dates, venue, location, organizer and description.
Use Analyze Page when unsure of the page type.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_image`

Extract Images · Read-only

Extract the main images from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with image URLs, sizes, titles and captions.
Use Extract Video for videos.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_job`

Extract Job Posting · Read-only

Extract a job posting from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with job title, employer, location, salary, employment type, skills and description.
Use Analyze Page when unsure of the page type.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_list`

Extract List · Read-only

Extract the items of a list page (search results, product listings, article indexes) from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with the list items, each with title, link, summary and image.
Use Crawl tools to follow every link on a site.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_product`

Extract Product · Read-only

Extract product details (name, price, availability, specs) from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with title, brand, offer and regular price, availability, SKU, specifications, images and reviews.
Use Analyze Page when unsure of the page type.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `discussion` (`boolean`, optional): false skips extracting reviews. Example: false.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_extract_video`

Extract Video · Read-only

Extract video details from a web page URL with Diffbot's AI, with no rules needed.
Returns an objects array with title, video URL, duration, author, date, view count and thumbnail.
Use Extract Images for images.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `url` (`string`, required): URL of the web page to extract. Example: https://www.example.com/blog/launch-announcement.
- `fields` (`string`, optional): Comma-separated optional fields to add, such as links, meta, querystring or breadcrumb. Example: meta,querystring.
- `proxy` (`string`, optional): IP address (and port) of your own proxy to fetch the page through. Example: 203.0.113.10:8080.
- `proxy_auth` (`string`, optional): Credentials for your own proxy, as username:password. Example: user:pass.
- `timeout` (`integer`, optional): Milliseconds to wait for the page to be fetched. Defaults to 30000. Example: 30000.
- `use_proxy` (`string`, optional): default uses Diffbot's datacenter proxy for this request; none disables proxies even if enabled globally. Example: default. One of: `default`, `none`.

Example input: `{"url":"<url>"}`

### `diffbot_natural_language_process`

Process Text · Read-only

Analyze raw text with Diffbot Natural Language: entities, sentiment, facts, relationships and more.
Returns entities (with Knowledge Graph links), sentiment, facts and sentences for the text.
Use Extract Article with natural_language to analyze a web page instead.

Read only: yes. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `content` (`string`, required): Text to analyze, 1 to 100,000 characters. Example: Diffbot is a startup based in Menlo Park..
- `document_type` (`string`, optional): Optional hint about the kind of document. Example: news.
- `fields` (`string`, optional): Comma-separated results to return: entities, sentiment, facts, records, sentences, categories, summary, openFacts or language. Defaults to entities,sentiment,facts,records,sentences. Example: entities,sentiment,facts.
- `format` (`string`, optional): plain text (default) or plain text with title (title, two line breaks, then body). Example: plain text. One of: `plain text`, `plain text with title`.
- `lang` (`string`, optional): ISO 639-1 language code. Defaults to auto-detect. Example: en.
- `summary_sentences` (`integer`, optional): Maximum sentences in the summary, 1 to 10. Defaults to 3. Example: 3.

Example input: `{"content":"<content>"}`

### `diffbot_bulk_job_create`

Create Bulk Job · Write

Start a bulk job that extracts a list of URLs with an Extract API, without crawling.
Returns the job's settings and status.
Use List Bulk Jobs to follow progress and Get Bulk Data to download results.

Read only: no. Destructive: no. Idempotent (safe to retry): no.

Inputs:

- `api_url` (`string`, required): Extract or Custom API URL to process each URL with, optionally with parameters. Example: https://api.diffbot.com/v3/product.
- `name` (`string`, required): Unique job name, used to manage the job and get its data. Example: acme-products-batch.
- `urls` (`string`, required): Space-separated URLs to process. Startup plans need at least 50 URLs. Example: https://www.example.com/p/1 https://www.example.com/p/2.
- `notify_email` (`string`, optional): Email notified when the job completes. Example: ops@example.com.
- `notify_webhook` (`string`, optional): URL that receives a POST when the job completes. Example: https://example.com/hooks/diffbot.
- `obey_robots` (`string`, optional): 0 ignores robots.txt. Example: 1. One of: `0`, `1`.
- `page_process_pattern` (`string`, optional): ||-separated strings the page HTML must contain to be processed. Example: <article.
- `repeat` (`number`, optional): Repeat the job every this many days. Example: 7.0.

Example input: `{"api_url":"<api_url>","name":"<name>","urls":"<urls>"}`

### `diffbot_bulk_job_pause`

Pause or Resume Bulk Job · Write

Pause a bulk job, or resume a paused one.
Returns the job's updated status.
Use List Bulk Jobs to check the status.

Read only: no. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `pause` (`integer`, required): 1 pauses the job; 0 resumes it. Example: 1. One of: `0`, `1`.

Example input: `{"name":"<name>","pause":1}`

### `diffbot_crawl_create`

Create Crawl · Write

Start a crawl that spiders a site from seed URLs and extracts matching pages with an Extract API.
Returns the job's settings and status.
Use List Crawl Jobs to follow progress and Get Crawl Data to download results.

Read only: no. Destructive: no. Idempotent (safe to retry): no.

Inputs:

- `api_url` (`string`, required): Extract API URL to process pages with, optionally with parameters. Example: https://api.diffbot.com/v3/analyze.
- `name` (`string`, required): Unique job name, used to manage the crawl and get its data. Example: acme-blog-crawl.
- `seeds` (`string`, required): Seed URL(s), separated by spaces. A non-www subdomain seed limits the crawl to that subdomain. Example: https://blog.example.com.
- `crawl_delay` (`number`, optional): Seconds to wait between URLs from one IP. Example: 0.25.
- `max_hops` (`integer`, optional): Link depth from the seeds; 0 processes only seeds, -1 (default) is unlimited. Example: 2.
- `max_rounds` (`integer`, optional): Maximum repeat rounds; 0 (default) repeats indefinitely. Example: 4.
- `max_to_crawl` (`integer`, optional): Maximum pages to spider. Defaults to 100000. Example: 1000.
- `max_to_process` (`integer`, optional): Maximum pages to process. Defaults to 100000. Example: 500.
- `notify_email` (`string`, optional): Email notified when the crawl completes or hits a limit. Example: ops@example.com.
- `notify_webhook` (`string`, optional): URL that receives a POST when the crawl completes or hits a limit. Example: https://example.com/hooks/diffbot.
- `obey_robots` (`integer`, optional): 1 (default) obeys robots.txt; 0 ignores it. Example: 1. One of: `0`, `1`.
- `only_process_if_new` (`integer`, optional): 1 (default) processes only new pages on repeat rounds; 0 processes all. Example: 1. One of: `0`, `1`.
- `page_process_pattern` (`string`, optional): ||-separated strings the page HTML must contain to be processed. Example: <article.
- `repeat` (`number`, optional): Repeat the crawl every this many days (for example 7.0). Example: 7.0.
- `restrict_domain` (`integer`, optional): 1 (default) keeps the crawl on the seed domains; 0 allows limited cross-domain crawling. Example: 1. One of: `0`, `1`.
- `url_crawl_pattern` (`string`, optional): ||-separated strings a URL must contain to be crawled; ! negates, ^ and $ anchor. Example: /blog/||!/tag/.
- `url_crawl_regex` (`string`, optional): Regex a URL must match to be crawled. Overrides url_crawl_pattern. Example: ^https://blog\.example\.com/20\d\d/.
- `url_process_pattern` (`string`, optional): ||-separated strings a URL must contain to be processed. Example: /blog/20.
- `url_process_regex` (`string`, optional): Regex a URL must match to be processed. Overrides url_process_pattern. Example: /blog/\d{4}/.
- `use_proxies` (`integer`, optional): 1 forces proxy IPs for crawling and processing. Example: 0. One of: `0`, `1`.

Example input: `{"api_url":"<api_url>","name":"<name>","seeds":"<seeds>"}`

### `diffbot_crawl_job_pause`

Pause or Resume Crawl · Write

Pause a crawl job, or resume a paused one.
Returns the job's updated status.
Use List Crawl Jobs to check the status.

Read only: no. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `pause` (`integer`, required): 1 pauses the job; 0 resumes it. Example: 1. One of: `0`, `1`.

Example input: `{"name":"<name>","pause":1}`

### `diffbot_crawl_job_round_start`

Start Crawl Round · Write

Start a new round of a crawl job now (repeat the crawl manually).
Returns the job's updated status.
By default only new pages are processed in the new round.

Read only: no. Destructive: no. Idempotent (safe to retry): no.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `round_start` (`integer`, required): Must be 1. Example: 1. One of: `1`.

Example input: `{"name":"<name>","round_start":1}`

### `diffbot_custom_api_rules_save`

Save Custom API Rules · Write

Create or update a Custom API ruleset that extracts fields with CSS selectors for URLs matching a pattern.
Returns the saved ruleset.
Use List Custom API Rules to see existing rulesets.

Read only: no. Destructive: no. Idempotent (safe to retry): no.

Inputs:

- `api` (`string`, required): API the rules apply to, prefixed with /api/ (for example /api/article), or all. Example: /api/product.
- `rules` (`array`, required): Rules, each with a field name and a CSS selector, plus optional type, args and filters. Example: [{"name": "price", "selector": ".price"}].
- `url_pattern` (`string`, required): Regex of the URLs the ruleset applies to. Example: ^https?://(www\.)?example\.com/products/.*.
- `notes` (`array`, optional): Free-text notes on the ruleset. Example: ["Added for the product catalog"].
- `prefilters` (`array`, optional): CSS selectors removed from the page before extraction. Example: [".cookie-banner"].
- `render_options` (`string`, optional): Rendering options. Example: wait=2000.
- `test_url` (`string`, optional): URL used to check the rules still work. Example: https://www.example.com/products/widget.
- `use_proxy` (`string`, optional): none disables proxies for these URLs when proxies are set globally. Example: none.

Example input: `{"api":"<api>","rules":[{"name":"price","selector":".price"}],"url_pattern":"<url_pattern>"}`

### `diffbot_dql_report_create`

Create DQL Coverage Report · Write

Generate a coverage report for a DQL query, showing how fully each field is filled.
Returns the report with its id and per-field coverage.
Use Get DQL Coverage Report to fetch it again later.

Read only: no. Destructive: no. Idempotent (safe to retry): no.

Inputs:

- `query` (`string`, required): DQL query to report on. Example: type:Organization industries:"Software Companies".
- `exportspec` (`string`, optional): Columns for CSV-style export, for example name;summary. Example: name;homepageUri.
- `filter` (`string`, optional): Semicolon-separated fields to keep in the response, as dot paths or JsonPath, for example $.name;$.homepageUri. Example: $.name;$.homepageUri;$.nbEmployees.
- `filter_exclude` (`string`, optional): Semicolon-separated fields to drop from the response, as dot paths or JsonPath. Example: $.descriptors.
- `from` (`integer`, optional): Index of the first result. Example: 0.
- `size` (`integer`, optional): Maximum results to cover; -1 for all. Defaults to 50. Example: 100.

Example input: `{"query":"<query>"}`

### `diffbot_enhance_bulk_create`

Create Enhance Bulk Job · Write

Start an asynchronous job that enhances many person or organization records at once.
Returns the bulk job ID (202).
Use Get Enhance Bulk Job Status, then Get Enhance Bulk Results.

Read only: no. Destructive: no. Idempotent (safe to retry): no.

Inputs:

- `records` (`array`, required): Records to enhance. Each has a type (Person or Organization) and any of the Enhance inputs: name, url, location, email, employer, title, school, phone, description, ip, id. Example: [{"type": "Organization", "name": "Diffbot"}, {"type": "Organization", "name": "Readwise"}].
- `name` (`string`, optional): Name for the bulk job. Example: crm-enrichment-nov.
- `refresh` (`boolean`, optional): Recrawl sources before returning. Example: false.
- `search` (`boolean`, optional): Also search the web for sources. Example: false.
- `size` (`integer`, optional): Matches per record. Defaults to 1. Example: 1.
- `threshold` (`number`, optional): Minimum similarity score for a match. Example: 0.5.
- `webhook_url` (`string`, optional): URL notified when the job finishes. Example: https://example.com/hooks/diffbot.

Example input: `{"records":[{"type":"Organization","name":"Diffbot"},{"type":"Organization","name":"Readwise"}]}`

### `diffbot_enhance_bulk_stop`

Stop Enhance Bulk Job · Write

Stop a running Enhance bulk job.
Returns the job's updated status.
Use Get Enhance Bulk Job Status to check it.

Read only: no. Destructive: no. Idempotent (safe to retry): yes.

Inputs:

- `bulkjob_id` (`string`, required): Enhance bulk job ID, returned by Create Enhance Bulk Job. Example: a1b2c3d4e5.

Example input: `{"bulkjob_id":"<bulkjob_id>"}`

### `diffbot_bulk_job_delete`

Delete Bulk Job · Destructive

Delete a bulk job and all its data.
Returns a confirmation.
Use Get Bulk Data first to keep the results.

Read only: no. Destructive: yes. Idempotent (safe to retry): yes.

Inputs:

- `delete` (`integer`, required): Must be 1. Example: 1. One of: `1`.
- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.

Example input: `{"delete":1,"name":"<name>"}`

### `diffbot_bulk_job_restart`

Restart Bulk Job · Destructive

Restart a bulk job, erasing its processed data and reprocessing all URLs.
Returns the job's updated status.
Use Get Bulk Data first to keep the current results.

Read only: no. Destructive: yes. Idempotent (safe to retry): no.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `restart` (`integer`, required): Must be 1. Example: 1. One of: `1`.

Example input: `{"name":"<name>","restart":1}`

### `diffbot_crawl_job_delete`

Delete Crawl · Destructive

Delete a crawl job and all its data.
Returns a confirmation.
Use Get Crawl Data first to keep the results.

Read only: no. Destructive: yes. Idempotent (safe to retry): yes.

Inputs:

- `delete` (`integer`, required): Must be 1. Example: 1. One of: `1`.
- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.

Example input: `{"delete":1,"name":"<name>"}`

### `diffbot_crawl_job_restart`

Restart Crawl · Destructive

Restart a crawl job from scratch, erasing its processed data and reprocessing all URLs.
Returns the job's updated status.
Use Start Crawl Round to repeat a crawl without erasing data.

Read only: no. Destructive: yes. Idempotent (safe to retry): no.

Inputs:

- `name` (`string`, required): Name of the job, as given when it was created. Example: acme-blog-crawl.
- `restart` (`integer`, required): Must be 1. Example: 1. One of: `1`.

Example input: `{"name":"<name>","restart":1}`

### `diffbot_enhance_bulk_delete`

Delete Enhance Bulk Job · Destructive

Delete an Enhance bulk job and its results.
Returns a confirmation.
Use Get Enhance Bulk Results first to keep the results.

Read only: no. Destructive: yes. Idempotent (safe to retry): yes.

Inputs:

- `bulkjob_id` (`string`, required): Enhance bulk job ID, returned by Create Enhance Bulk Job. Example: a1b2c3d4e5.

Example input: `{"bulkjob_id":"<bulkjob_id>"}`

## Related

Other AI connectors ([all 216](/agentkit/connectors/?category=ai)):

- [Chorus](/agentkit/connectors/chorus/): Scalekit connector, Username and password, 9 tools
- [Gong](/agentkit/connectors/gong/): Scalekit connector, OAuth, 69 tools
- [Jiminny](/agentkit/connectors/jiminny/): Scalekit connector, Access token, 27 tools


---

## More Scalekit documentation

| Resource | What it contains | When to use it |
|----------|-----------------|----------------|
| [/llms.txt](/llms.txt) | Structured index with routing hints per product area | Start here — find which documentation set covers your topic before loading full content |
| [/llms-full.txt](/llms-full.txt) | Complete documentation for all Scalekit products in one file | Use when you need exhaustive context across multiple products or when the topic spans several areas |
| [sitemap-0.xml](https://docs.scalekit.com/sitemap-0.xml) | Full URL list of every documentation page | Use to discover specific page URLs you can fetch for targeted, page-level answers |
