The ScrapingBee connector lets your AI agent act in each user's ScrapingBee account. Each user connects their own ScrapingBee API key once, and Scalekit sends it with every call, so your agent never handles credentials. It comes with 19 tools.
- Tools
- 19
- What they doRead · write · destructive
- 1 · 18 · 01 read18 write0 destructive
- Users sign in with
- API key
Setup
Install the SDK
Terminal window npm install @scalekit-sdk/node dotenvTerminal window pip install scalekit-sdk-python python-dotenvSet your credentials
Add your Scalekit credentials to your
.envfile. Find values in app.scalekit.com > Developers > API Credentials..env SCALEKIT_ENVIRONMENT_URL=<your-environment-url>SCALEKIT_CLIENT_ID=<your-client-id>SCALEKIT_CLIENT_SECRET=<your-client-secret>Create the ScrapingBee connection
In AgentKit > Connections, create a ScrapingBee connection. The name you give it is the
connection_nameyour code passes. See Configure connections.Authorize a user and make your first call
quickstart.mts import { ScalekitClient } from '@scalekit-sdk/node'import 'dotenv/config'import { createInterface } from 'node:readline/promises'const scalekit = new ScalekitClient(process.env.SCALEKIT_ENVIRONMENT_URL,process.env.SCALEKIT_CLIENT_ID,process.env.SCALEKIT_CLIENT_SECRET,)const actions = scalekit.actionsconst connector = 'scrapingbee'const identifier = 'user_123'// Generate an authorization link for the userconst { link } = await actions.getAuthorizationLink({ connectionName: connector, identifier })console.log('Authorize ScrapingBee:', link)const rl = createInterface({ input: process.stdin, output: process.stdout })await rl.question('Press Enter after authorizing...')rl.close()// Make your first callconst result = await actions.executeTool({connector,identifier,toolName: 'scrapingbee_usage_get',toolInput: {},})console.log(result)Terminal window npx tsx quickstart.mtsquickstart.py import osfrom scalekit import ScalekitClientfrom dotenv import load_dotenvload_dotenv()scalekit_client = ScalekitClient(env_url=os.getenv("SCALEKIT_ENVIRONMENT_URL"),client_id=os.getenv("SCALEKIT_CLIENT_ID"),client_secret=os.getenv("SCALEKIT_CLIENT_SECRET"),)actions = scalekit_client.actionsconnection_name = "scrapingbee"identifier = "user_123"# Generate an authorization link for the userlink_response = actions.get_authorization_link(connection_name=connection_name,identifier=identifier,)print("Authorize ScrapingBee:", link_response.link)input("Press Enter after authorizing...")# Make your first callresult = actions.execute_tool(tool_input={},tool_name="scrapingbee_usage_get",connection_name=connection_name,identifier=identifier,)print(result)Terminal window python quickstart.pyEach user opens the link once and enters their ScrapingBee credentials there. See Authorize a user for the full flow and statuses.
Tools
Pass the exact name toexecute_toolscrapingbee_usage_getGet the ScrapingBee account's API credit and concurrency usage for the current billing period, in real time.Read-onlyGet Account Usage
Get the ScrapingBee account's API credit and concurrency usage for the current billing period, in real time. Returns max_api_credit, used_api_credit, max_concurrency, current_concurrency and renewal_subscription_date. Use scrapingbee_usage_get before a large job to check remaining credits. It spends no credits but is limited to 6 calls per minute.
Inputs
This tool takes no inputs.
scrapingbee_amazon_pricing_getGet the seller offers and prices for one Amazon product by ASIN as structured JSON.WriteGet Amazon Product Offers
Get the seller offers and prices for one Amazon product by ASIN as structured JSON. Returns each offer's price, currency, condition, seller, shipping and delivery details. Use scrapingbee_amazon_pricing_get to compare sellers. Use scrapingbee_amazon_product_get for the product's description, images and reviews. Costs 5 credits for a light request, 15 otherwise.
- Idempotent
Inputs
asinstringrequired- The product's 10-character ASIN, from an Amazon product URL or scrapingbee_amazon_search. Example: B0DPDRNSXV.
countrystring- Two-letter country code for the location of the request. Example: us.
currencystring- ISO 4217 currency code to show prices in. Example: USD.
devicestring- Device to emulate: desktop (default), mobile or tablet. Example: mobile.one of
desktopmobiletablet domainstring- Amazon top-level domain to use, such as com, co.uk, de or fr (default com). Example: co.uk.
languagestring- Language code for the page, in ISO format. Example: en_US.
light_requestboolean- Leave empty or true for a fast light request (5 credits); set false to force a JavaScript-rendered request (15 credits), for example to get reviews or other dynamic content. Example: false.
zip_codestring- ZIP or postal code to localize prices and delivery. Example: 10001.
scrapingbee_amazon_product_getGet one Amazon product's details by ASIN as structured JSON.WriteGet Amazon Product
Get one Amazon product's details by ASIN as structured JSON. Returns title, brand, price, currency, rating, review count, availability, description, bullet points, images, variants, categories and top reviews. Use scrapingbee_amazon_product_get after scrapingbee_amazon_search. Use scrapingbee_amazon_pricing_get to compare the product's offers from different sellers. Costs 5 credits for a light request, 15 otherwise.
- Idempotent
Inputs
querystringrequired- The product's 10-character ASIN, from an Amazon product URL or scrapingbee_amazon_search. Example: B0DPDRNSXV.
autoselect_variantboolean- Set true to auto-select the product's default variant. Example: true.
countrystring- Two-letter country code for the location of the request. Example: us.
currencystring- ISO 4217 currency code to show prices in. Example: USD.
devicestring- Device to emulate: desktop (default), mobile or tablet. Example: mobile.one of
desktopmobiletablet domainstring- Amazon top-level domain to use, such as com, co.uk, de or fr (default com). Example: co.uk.
languagestring- Language code for the page, in ISO format. Example: en_US.
light_requestboolean- Leave empty or true for a fast light request (5 credits); set false to force a JavaScript-rendered request (15 credits), for example to get reviews or other dynamic content. Example: false.
zip_codestring- ZIP or postal code to localize prices and delivery. Example: 10001.
scrapingbee_amazon_searchSearch Amazon products by keyword and return the results page as structured JSON.WriteSearch Amazon
Search Amazon products by keyword and return the results page as structured JSON. Returns products with asin, title, price, currency, rating, review count, url and flags such as is_prime, is_sponsored and best_seller, plus the page number. Use scrapingbee_amazon_search to find products and their ASINs. Use scrapingbee_amazon_product_get for one product's full details and scrapingbee_amazon_pricing_get for its seller offers. Costs 5 credits per page for a light request, 15 otherwise.
- Idempotent
Inputs
querystringrequired- Amazon search term. Example: wireless earbuds.
autoselect_variantboolean- Set true to auto-select a product variant in results. Example: true.
category_idstring- Amazon category (node) ID to limit results to. Example: 565108.
countrystring- Two-letter country code for the location of the request. Example: us.
currencystring- ISO 4217 currency code to show prices in. Example: USD.
devicestring- Device to emulate: desktop (default), mobile or tablet. Example: mobile.one of
desktopmobiletablet domainstring- Amazon top-level domain to use, such as com, co.uk, de or fr (default com). Example: co.uk.
languagestring- Language code for the page, in ISO format. Example: en_US.
light_requestboolean- Leave empty or true for a fast light request (5 credits); set false to force a JavaScript-rendered request (15 credits), for example to get reviews or other dynamic content. Example: false.
merchant_idstring- Amazon seller (merchant) ID to limit results to. Example: A2EUQ1WTGCTBG2.
pagesinteger- Number of result pages to fetch from start_page (default 1). Each page is billed. Example: 2.
sort_bystring- Result order: featured, most_recent, price_low_to_high, price_high_to_low, average_review or bestsellers. Example: price_low_to_high.one of
featuredmost_recentprice_low_to_highprice_high_to_lowaverage_reviewbestsellers start_pageinteger- First result page to fetch (default 1). Example: 2.
zip_codestring- ZIP or postal code to localize prices and delivery. Example: 10001.
scrapingbee_chatgpt_promptSend a prompt to ChatGPT through ScrapingBee, optionally with web search, and return its answer.WriteAsk ChatGPT
Send a prompt to ChatGPT through ScrapingBee, optionally with web search, and return its answer. Returns results_markdown, results_text and results_json (the answer as Markdown, plain text and parsed blocks), llm_model and the prompt. Use scrapingbee_chatgpt_prompt to see how ChatGPT answers a question. Use scrapingbee_google_search or scrapingbee_page_ai_query to get facts from search results or a specific page. Costs 15 credits per call.
- Idempotent
Inputs
promptstringrequired- The prompt to send to ChatGPT. Example: Explain the benefits of renewable energy in 100 words.
country_codestring- Two-letter country code to send the request from. Example: us.
searchboolean- Set true to let ChatGPT search the web while answering. Example: true.
scrapingbee_fast_searchRun a lightweight web search that answers in under a second.WriteFast Web Search
Run a lightweight web search that answers in under a second. Returns organic results with title, link and description. Use scrapingbee_fast_search for quick lookups. Use scrapingbee_google_search for Google-specific result types, location targeting and filters. Costs 10 credits per call.
- Idempotent
Inputs
searchstringrequired- The query to search for. Example: pizza in new york.
country_codestring- Two-letter ISO 3166-1 country code to localize results (default us). Example: fr.
pageinteger- Result page number, starting at 1. Example: 2.
scrapingbee_google_searchRun a Google search (web, news, maps, images, Lens, shopping, AI Mode or ads) and return the results page as structured JSON.WriteGoogle Search
Run a Google search (web, news, maps, images, Lens, shopping, AI Mode or ads) and return the results page as structured JSON. Returns organic_results with position, title, url, domain and description, plus sections Google shows such as ai_overviews, top_stories, knowledge_graph, local_results, related_queries and ads. Use scrapingbee_google_search for full Google results and filters. Use scrapingbee_fast_search for a quicker, cheaper list of links. Costs 10 credits for a light request, 15 otherwise.
- Idempotent
Inputs
searchstringrequired- The query, as you would type it into Google. Example: best pizza in new york.
country_codestring- Two-letter country code to search from (default us). Example: gb.
date_rangestring- Only return results from this period: past_hour, past_day, past_week, past_month or past_year. Example: past_week.one of
past_hourpast_daypast_weekpast_monthpast_year devicestring- Device to search from: desktop (default) or mobile. Example: mobile.one of
desktopmobile expand_aioboolean- Set true to expand AI Overviews that Google shows collapsed (classic and ads searches; 15 credits). Example: true.
extra_paramsstring- Extra Google URL parameters, URL-encoded as one string. Only filter, fpstate, locale, nfpr, safe, safe_search, tbm, tbs and udm are accepted, each once. Example: locale%3Dde-DE%26filter%3D0.
latitudenumber- Latitude in decimal degrees of the place to search from, used with longitude. Example: 40.7128.
light_requestboolean- Leave empty or true for a fast light request (10 credits); set false for a full JavaScript-rendered request (15 credits) when content is missing. Example: false.
longitudenumber- Longitude in decimal degrees of the place to search from, used with latitude. Example: -74.0060.
max_pricenumber- Highest price for Google Shopping results (search_type shopping only), in the marketplace currency. Example: 100.
min_pricenumber- Lowest price for Google Shopping results (search_type shopping only), in the marketplace currency. Example: 20.
nfprboolean- Set true to stop Google from auto-correcting a misspelled query. Example: true.
pageinteger- Result page number to return, starting at 1. Example: 2.
pagesinteger- Number of consecutive result pages to fetch and merge into one response (default 1). Example: 3.
radiusinteger- Radius in meters around latitude and longitude to search within. Example: 5000.
search_typestring- Kind of Google search: classic (default web results), news, maps, images, lens, shopping, ai_mode or ads. Example: news.one of
classicnewsmapsimageslensshoppingai_modeads sort_bystring- Order of Google Shopping results (search_type shopping only): relevance, reviews, price_asc or price_desc. Example: price_asc.one of
relevancereviewsprice_ascprice_desc
scrapingbee_page_ai_extractFetch a web page and have AI extract fields you describe in plain language.WriteExtract Page Data with AI
Fetch a web page and have AI extract fields you describe in plain language. Returns a JSON object with one key per rule, typed as the rule asks (string, number, boolean, list or nested item). Use scrapingbee_page_ai_extract when selectors are unknown or change between pages. Use scrapingbee_page_data_extract when you have stable selectors, and scrapingbee_page_ai_query for a single free-form question. Adds 5 credits to the normal page cost (10 credits by default).
- Idempotent
Inputs
ai_extract_rulesstringrequired- AI extraction rules as a JSON string. Each key is an output field name and each value describes in plain words what to extract, or is an object with description, type (string, number, boolean, list or item), optional enum and, for item, an output object of sub-fields. Example: {"name": "the product name", "price": {"description": "the product price in dollars", "type": "number"}}.
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
ai_selectorstring- CSS selector of the part of the page the AI should read, to improve accuracy and speed. Example: #product-details.
block_adsboolean- Set true to block ads while the page loads. Example: true.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
render_jsboolean- Set false to fetch the page without a headless browser (1 credit instead of 5, faster, but no JavaScript runs). Leave empty for the default, true. Example: false.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
scrapingbee_page_ai_queryFetch a web page and answer one plain-language question about its content with AI.WriteAsk AI About a Page
Fetch a web page and answer one plain-language question about its content with AI. Returns JSON whose body field holds the AI's answer, plus resolved-url and cost in credits. Use scrapingbee_page_ai_query for one free-form answer. Use scrapingbee_page_ai_extract for several named, typed fields. Adds 5 credits to the normal page cost (10 credits by default).
- Idempotent
Inputs
ai_querystringrequired- What you want from the page, in plain language. Example: price of the product.
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
ai_selectorstring- CSS selector of the part of the page the AI should read, to improve accuracy and speed. Example: #product-details.
block_adsboolean- Set true to block ads while the page loads. Example: true.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
render_jsboolean- Set false to fetch the page without a headless browser (1 credit instead of 5, faster, but no JavaScript runs). Leave empty for the default, true. Example: false.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
scrapingbee_page_data_extractFetch a web page and extract named fields from it with CSS or XPath selectors.WriteExtract Page Data with Selectors
Fetch a web page and extract named fields from it with CSS or XPath selectors. Returns a JSON object with one key per rule, holding the matched text, attribute or list. Use scrapingbee_page_data_extract when you know the page's selectors; it is exact and adds no AI cost. Use scrapingbee_page_ai_extract when you can only describe the fields in words. Costs the same credits as scrapingbee_page_scrape (5 by default).
- Idempotent
Inputs
extract_rulesstringrequired- Extraction rules as a JSON string. Each key is a name you choose for an output field and each value is a CSS selector, or an XPath starting with /. A value can also be an object with selector, type (item or list) and output for repeated or nested data. See https://www.scrapingbee.com/documentation/data-extraction/. Example: {"title": "h1", "links": {"selector": "a", "type": "list", "output": "@href"}}.
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
block_adsboolean- Set true to block ads while the page loads. Example: true.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
render_jsboolean- Set false to fetch the page without a headless browser (1 credit instead of 5, faster, but no JavaScript runs). Leave empty for the default, true. Example: false.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
scrapingbee_page_markdown_getFetch a web page and return its main content converted to Markdown.WriteGet Page as Markdown
Fetch a web page and return its main content converted to Markdown. Returns JSON whose body field holds the Markdown, plus resolved-url, initial-status-code and cost in credits. Use scrapingbee_page_markdown_get to read an article or document page with headings and links kept. Use scrapingbee_page_text_get for plain text, or scrapingbee_page_scrape for the full HTML. Costs the same credits as scrapingbee_page_scrape (5 by default).
- Idempotent
Inputs
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
block_adsboolean- Set true to block ads while the page loads. Example: true.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
render_jsboolean- Set false to fetch the page without a headless browser (1 credit instead of 5, faster, but no JavaScript runs). Leave empty for the default, true. Example: false.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
scrapingbee_page_scrapeFetch a web page through ScrapingBee's proxies and headless browser and return its HTML wrapped in JSON.WriteScrape Web Page
Fetch a web page through ScrapingBee's proxies and headless browser and return its HTML wrapped in JSON. Returns body (the page HTML), the final resolved-url, initial-status-code, response headers, cookies, cost in credits and any XHR calls the page made. Use scrapingbee_page_scrape when you need the raw HTML or a browser scenario. Use scrapingbee_page_markdown_get or scrapingbee_page_text_get for readable content, scrapingbee_page_data_extract or scrapingbee_page_ai_extract for structured fields, and scrapingbee_page_screenshot_capture for an image. Costs 5 credits by default, 1 without JavaScript rendering, 25 with premium proxies and 75 with stealth proxies.
- Idempotent
Inputs
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
block_adsboolean- Set true to block ads while the page loads. Example: true.
block_resourcesboolean- Set false to load images and CSS (they are blocked by default to save time). Example: false.
cookiesstring- Cookies to send to the target page, as name=value pairs separated by semicolons. Example: session=abc123;lang=en.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile js_scenariostring- A JavaScript scenario as a JSON string: an object with an instructions list of browser steps such as click, wait, wait_for, scroll_y, fill and evaluate, run in order after the page loads. Requires JavaScript rendering. See https://www.scrapingbee.com/documentation/js-scenario/. Example: {"instructions": [{"click": "#load-more"}, {"wait": 1000}]}.
max_costinteger- Highest credit cost Auto-Mode may reach, for example 10 to stop before premium (25) and stealth (75) proxies. Only used with mode=auto. Example: 10.
modestring- Set to auto to let ScrapingBee try JavaScript rendering and proxy tiers from cheapest to most expensive and charge only for the first that succeeds (0 credits if all fail). Example: auto.one of
auto premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
render_jsboolean- Set false to fetch the page without a headless browser (1 credit instead of 5, faster, but no JavaScript runs). Leave empty for the default, true. Example: false.
return_page_sourceboolean- Set true to return the HTML the server sent, before the browser ran any JavaScript. Example: true.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
transparent_status_codeboolean- Set true to report the target page's own HTTP status code instead of ScrapingBee's. Example: true.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
scrapingbee_page_screenshot_captureRender a web page in a headless browser and capture a PNG screenshot of the viewport, the full page or one element.WriteCapture Page Screenshot
Render a web page in a headless browser and capture a PNG screenshot of the viewport, the full page or one element. Returns JSON with screenshot (the PNG as a base64 string) and body (the page HTML), plus resolved-url and cost in credits. Use scrapingbee_page_screenshot_capture when you need to see the page. Use scrapingbee_page_scrape or scrapingbee_page_markdown_get when you only need its content. Always renders JavaScript, so it costs at least 5 credits.
- Idempotent
Inputs
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
block_adsboolean- Set true to block ads while the page loads. Example: true.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
screenshot_full_pageboolean- Set true to capture the whole scrollable page instead of only the visible viewport. Example: true.
screenshot_selectorstring- CSS selector of one element to capture instead of the viewport. Example: #main-chart.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
window_heightinteger- Browser viewport height in pixels (default 1080). Example: 800.
window_widthinteger- Browser viewport width in pixels (default 1920). Example: 1280.
scrapingbee_page_text_getFetch a web page and return its main content as plain text.WriteGet Page as Text
Fetch a web page and return its main content as plain text. Returns JSON whose body field holds the text, plus resolved-url, initial-status-code and cost in credits. Use scrapingbee_page_text_get for the shortest readable version of a page. Use scrapingbee_page_markdown_get to keep headings and links. Costs the same credits as scrapingbee_page_scrape (5 by default).
- Idempotent
Inputs
urlstringrequired- Full URL of the page to fetch, including http:// or https://. Example: https://www.scrapingbee.com/blog/.
block_adsboolean- Set true to block ads while the page loads. Example: true.
country_codestring- Two-letter ISO 3166-1 country code of the proxy to send the request from, such as us, de or gb. Works with classic, premium and stealth proxies; each has its own supported list (https://www.scrapingbee.com/documentation/country_codes/). Example: de.
devicestring- Device type to emulate: desktop (default) or mobile. Example: mobile.one of
desktopmobile premium_proxyboolean- Set true to route the request through residential premium proxies for sites that block the classic pool. Costs 25 credits with JavaScript rendering, 10 without. Example: true.
render_jsboolean- Set false to fetch the page without a headless browser (1 credit instead of 5, faster, but no JavaScript runs). Leave empty for the default, true. Example: false.
session_idinteger- Any integer from 0 to 10000000. Requests that share a session_id go out through the same IP address for 5 minutes. Example: 4521.
stealth_proxyboolean- Set true to use the stealth proxy pool for the hardest sites, when premium_proxy is not enough. Needs JavaScript rendering, costs 75 credits, and ignores the timeout input. Example: true.
timeoutinteger- Maximum time in milliseconds, 1000 to 140000 (default 140000), that ScrapingBee may spend on the request. Keep it under 110000 so the result arrives before the tool call times out. Example: 60000.
waitinteger- Extra time in milliseconds, 0 to 35000, to let the page's JavaScript finish before the result is returned. Example: 3000.
wait_browserstring- Browser event to wait for: domcontentloaded (default), load, networkidle0 (no open connections for 500 ms) or networkidle2 (at most 2 for 500 ms). Example: load.one of
domcontentloadedloadnetworkidle0networkidle2 wait_forstring- CSS selector, or XPath starting with /, of an element to wait for before the result is returned. Example: .product-list.
scrapingbee_walmart_product_getGet one Walmart product's details by product ID as structured JSON.WriteGet Walmart Product
Get one Walmart product's details by product ID as structured JSON. Returns title, description, price (current, original, currency, discount), rating, reviews, availability, seller, images and specifications. Use scrapingbee_walmart_product_get after scrapingbee_walmart_search. Costs 10 credits for a light request, 15 otherwise.
- Idempotent
Inputs
product_idstringrequired- Walmart product ID, from a walmart.com/ip/... URL or scrapingbee_walmart_search. Example: 5491199371.
delivery_zipstring- ZIP code to localize prices and delivery. Example: 10001.
devicestring- Device to emulate: desktop (default), mobile or tablet. Example: mobile.one of
desktopmobiletablet domainstring- Walmart domain for a regional site, for example com or ca. Leave empty for walmart.com. Example: ca.
light_requestboolean- Leave empty or true for a fast light request (10 credits); set false to force a JavaScript-rendered request (15 credits). Example: false.
store_idstring- Walmart store ID to localize availability and prices. Example: 3081.
scrapingbee_walmart_searchSearch Walmart products by keyword and return the results page as structured JSON.WriteSearch Walmart
Search Walmart products by keyword and return the results page as structured JSON. Returns meta_data (result count, page, total_pages) and products with position, product_id, title, price, rating, review count, seller and url. Use scrapingbee_walmart_search to find products and their IDs. Use scrapingbee_walmart_product_get for one product's full details. Costs 10 credits for a light request, 15 otherwise.
- Idempotent
Inputs
querystringrequired- Walmart search term. Example: iphone.
delivery_zipstring- ZIP code to localize prices and delivery. Example: 10001.
devicestring- Device to emulate: desktop (default), mobile or tablet. Example: mobile.one of
desktopmobiletablet domainstring- Walmart domain for a regional site, for example com or ca. Leave empty for walmart.com. Example: ca.
fulfillment_speedstring- Only show items deliverable this fast: today, tomorrow, 2_days or anytime. Example: tomorrow.one of
todaytomorrow2_daysanytime fulfillment_typestring- Set to in_store to only show items available in store. Example: in_store.one of
in_store light_requestboolean- Leave empty or true for a fast light request (10 credits); set false to force a JavaScript-rendered request (15 credits). Example: false.
max_priceinteger- Highest price to include, in whole currency units. Example: 500.
min_priceinteger- Lowest price to include, in whole currency units. Example: 50.
sort_bystring- Result order: best_match (default), price_low, price_high or best_seller. Example: price_low.one of
best_matchprice_lowprice_highbest_seller start_pageinteger- Result page number to fetch (default 1). Example: 2.
store_idstring- Walmart store ID to localize availability and prices. Example: 3081.
scrapingbee_youtube_metadata_getGet one YouTube video's metadata by video ID.WriteGet YouTube Video Metadata
Get one YouTube video's metadata by video ID. Returns title, description, channel name, ID and URL, duration, view, like and comment counts, upload date, categories, tags, age limit and thumbnails. Use scrapingbee_youtube_metadata_get for a video's details. Use scrapingbee_youtube_subtitles_get for what is said in it. Costs 5 credits per call.
- Idempotent
Inputs
video_idstringrequired- The YouTube video ID, the v value in a youtube.com/watch?v=... URL or from scrapingbee_youtube_search. Example: kqtD5dpn9C8.
scrapingbee_youtube_searchSearch YouTube for videos, channels, playlists or movies with optional filters.WriteSearch YouTube
Search YouTube for videos, channels, playlists or movies with optional filters. Returns results, a list of YouTube's own result objects; video items carry the video ID, title, channel, view count, length, publish time and thumbnails. Use scrapingbee_youtube_search to find videos and their IDs. Use scrapingbee_youtube_metadata_get for one video's full details and scrapingbee_youtube_subtitles_get for its transcript. Costs 5 credits per call.
- Idempotent
Inputs
searchstringrequired- YouTube search terms. Example: python tutorial.
creative_commonsboolean- Set true to return only Creative Commons licensed videos. Example: true.
durationstring- Video length in minutes: <4, 4-20 or >20. Example: 4-20.one of
<44-20>20 hdboolean- Set true to return only HD videos. Example: true.
liveboolean- Set true to return only live streams. Example: true.
sort_bystring- Result order: relevance (default), rating, view_count or upload_date. Example: view_count.one of
relevanceratingview_countupload_date subtitlesboolean- Set true to return only videos with subtitles or closed captions. Example: true.
typestring- Kind of result to return: video, channel, playlist or movie. Example: video.one of
videochannelplaylistmovie upload_datestring- Only videos uploaded in this period: last_hour, today, this_week, this_month or this_year. Example: this_week.one of
last_hourtodaythis_weekthis_monththis_year
scrapingbee_youtube_subtitles_getGet the subtitles of one YouTube video by video ID.WriteGet YouTube Video Subtitles
Get the subtitles of one YouTube video by video ID. Returns subtitles grouped by origin (auto_generated, uploader_provided) and language, each a list of timed snippets with start_ms, duration and text. Use scrapingbee_youtube_subtitles_get to read or summarize what a video says. Use scrapingbee_youtube_metadata_get for its title, counts and description. Costs 5 credits per call.
- Idempotent
Inputs
video_idstringrequired- The YouTube video ID, the v value in a youtube.com/watch?v=... URL or from scrapingbee_youtube_search. Example: kqtD5dpn9C8.
languagestring- ISO language code of the subtitles to return. Example: en.
subtitle_originstring- Which subtitles to return: auto_generated or uploader_provided. Leave empty for both. Example: uploader_provided.one of
auto_generateduploader_provided
No tools match.