The AssemblyAI connector lets your AI agent act in each user's AssemblyAI account. Each user connects their own AssemblyAI API key once, and Scalekit sends it with every call, so your agent never handles credentials. It comes with 12 tools.
- Tools
- 12
- What they doRead · write · destructive
- 9 · 2 · 19 read2 write1 destructive
- Users sign in with
- API key
Setup
Install the SDK
Terminal window npm install @scalekit-sdk/node dotenvTerminal window pip install scalekit-sdk-python python-dotenvSet your credentials
Add your Scalekit credentials to your
.envfile. Find values in app.scalekit.com > Developers > API Credentials..env SCALEKIT_ENVIRONMENT_URL=<your-environment-url>SCALEKIT_CLIENT_ID=<your-client-id>SCALEKIT_CLIENT_SECRET=<your-client-secret>Create the AssemblyAI connection
In AgentKit > Connections, create an AssemblyAI connection. The name you give it is the
connection_nameyour code passes. See Configure connections.Authorize a user and make your first call
quickstart.mts import { ScalekitClient } from '@scalekit-sdk/node'import 'dotenv/config'import { createInterface } from 'node:readline/promises'const scalekit = new ScalekitClient(process.env.SCALEKIT_ENVIRONMENT_URL,process.env.SCALEKIT_CLIENT_ID,process.env.SCALEKIT_CLIENT_SECRET,)const actions = scalekit.actionsconst connector = 'assemblyai'const identifier = 'user_123'// Generate an authorization link for the userconst { link } = await actions.getAuthorizationLink({ connectionName: connector, identifier })console.log('Authorize AssemblyAI:', link)const rl = createInterface({ input: process.stdin, output: process.stdout })await rl.question('Press Enter after authorizing...')rl.close()// Make your first callconst result = await actions.executeTool({connector,identifier,toolName: 'assemblyai_list_llm_models',toolInput: {},})console.log(result)Terminal window npx tsx quickstart.mtsquickstart.py import osfrom scalekit import ScalekitClientfrom dotenv import load_dotenvload_dotenv()scalekit_client = ScalekitClient(env_url=os.getenv("SCALEKIT_ENVIRONMENT_URL"),client_id=os.getenv("SCALEKIT_CLIENT_ID"),client_secret=os.getenv("SCALEKIT_CLIENT_SECRET"),)actions = scalekit_client.actionsconnection_name = "assemblyai"identifier = "user_123"# Generate an authorization link for the userlink_response = actions.get_authorization_link(connection_name=connection_name,identifier=identifier,)print("Authorize AssemblyAI:", link_response.link)input("Press Enter after authorizing...")# Make your first callresult = actions.execute_tool(tool_input={},tool_name="assemblyai_list_llm_models",connection_name=connection_name,identifier=identifier,)print(result)Terminal window python quickstart.pyEach user opens the link once and enters their AssemblyAI credentials there. See Authorize a user for the full flow and statuses.
Tools
Pass the exact name toexecute_toolassemblyai_create_chat_completionSend a prompt or chat messages to an LLM through AssemblyAI's LLM Gateway, optionally with a transcript's text inserted, to summarize, answer questions about or extract data from audio.Read-onlyCreate Chat Completion
Send a prompt or chat messages to an LLM through AssemblyAI's LLM Gateway, optionally with a transcript's text inserted, to summarize, answer questions about or extract data from audio. Returns choices with the model's message (content and any tool_calls), finish_reason, usage token counts and request_id. Use this for free-form prompts over transcripts (the replacement for LeMUR). Use assemblyai_run_speech_understanding for built-in translation, speaker identification, custom formatting, summaries or action items. Get a valid model ID from assemblyai_list_llm_models. To use a transcript, it must be completed; reference it with transcript_id and the {{ transcript }} tag.
Inputs
modelstringrequired- ID of the LLM to use, from assemblyai_list_llm_models, for example claude-sonnet-4-5-20250929, gpt-4.1 or gemini-2.5-flash-lite. Never guess it; check the model list. Example: gemini-2.5-flash-lite
fallback_configobject- Fallback behaviour: retry (default true) retries once after 500 ms, and depth (1 or 2, default 1) caps how many fallbacks are tried. Example: {"retry": true, "depth": 2}
fallbacksarray- Fallback models to try in order when the primary model fails. Each item needs a model and can override other request fields. Example: [{"model": "claude-sonnet-4-6"}]
max_tokensinteger- Maximum number of tokens to generate. Defaults to 1000. Example: 1500
messagesarray- Conversation so far, as objects with role (system, user, assistant or tool) and content. Provide messages or prompt. To use a transcript, put the tag {{ transcript }} in a message and set transcript_id. Example: [{"role": "system", "content": "You summarize calls."}, {"role": "user", "content": "Summarize this call: {{ transcript }}"}]
model_regionstring- Set to global to route to the provider's global (non-regional) endpoints for lower cost. Currently live for Anthropic Claude models. Leave empty for in-region processing.one of
global post_processing_stepsarray- Steps applied to the model's answer. The only step today is json-repair, which fixes malformed JSON. Example: [{"type": "json-repair"}]
promptstring- A single prompt, sent as one user message. Use instead of messages for one-shot requests. Can contain the {{ transcript }} tag. Example: List the action items in this meeting: {{ transcript }}
response_formatobject- Constrain the answer to JSON that matches a schema: {"type": "json_schema", "json_schema": {"name": ..., "schema": {...}, "strict": true}}. Not supported by gpt-oss models. Example: {"type": "json_schema", "json_schema": {"name": "summary", "schema": {"type": "object", "properties": {"summary": {"type": "string"}}, "required": ["summary"]}, "strict": true}}
temperaturenumber- Randomness from 0 to 2; lower is more deterministic. Leave empty for the model's default. Example: 0.2
tool_choicestring- Which tool the model may call: none, auto, or {"type": "function", "function": {"name": "get_weather"}} to force one. Example: auto
toolsarray- Functions the model may call, each {"type": "function", "function": {"name": ..., "description": ..., "parameters": <JSON Schema>}}. Tool calls come back in choices[].message.tool_calls; send results back as role tool messages. Example: [{"type": "function", "function": {"name": "get_weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}]
transcript_idstring- ID of a completed AssemblyAI transcript whose text replaces the first {{ transcript }} tag in the messages or prompt. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
assemblyai_get_redacted_audioGet the PII-redacted copy of a transcript's audio.Read-onlyGet Redacted Audio
Get the PII-redacted copy of a transcript's audio. Returns status (redacted_audio_ready) and redacted_audio_url, a download link that expires after 24 hours. Use this after assemblyai_create_transcript ran with redact_pii and redact_pii_audio set to true and the transcript completed. Use assemblyai_get_transcript for the redacted text.
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
assemblyai_get_subtitlesExport a completed AssemblyAI transcript as SRT or WebVTT subtitles.Read-onlyGet Subtitles
Export a completed AssemblyAI transcript as SRT or WebVTT subtitles. Returns the caption file as plain text, not JSON. Use this to produce captions for a video player. Use assemblyai_get_transcript_sentences for structured, timestamped segments. The transcript must have status completed (check with assemblyai_get_transcript).
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
chars_per_captioninteger- Maximum number of characters in each caption. Leave empty for AssemblyAI's default line length. Example: 32
subtitle_formatstring- Caption format: srt (SubRip, the default) or vtt (WebVTT).one of
srtvttdefaultsrt
assemblyai_get_transcriptGet an AssemblyAI transcript by ID, including its processing status.Read-onlyGet Transcript
Get an AssemblyAI transcript by ID, including its processing status. Returns status (queued, processing, completed or error), the full text, word-level timestamps, speaker utterances, detected language, error when it failed, and the results of any enabled features (entities, sentiment, PII redaction, speech understanding). Use this to poll a transcript started with assemblyai_create_transcript and to read the whole result. Use assemblyai_get_transcript_sentences or assemblyai_get_transcript_paragraphs for readable segments, and assemblyai_list_transcripts to find IDs.
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
assemblyai_get_transcript_paragraphsGet a completed AssemblyAI transcript split into paragraphs.Read-onlyGet Transcript Paragraphs
Get a completed AssemblyAI transcript split into paragraphs. Returns the transcript id, confidence, audio_duration and a paragraphs list, each with text, start and end (milliseconds), confidence and words. Use this for a readable, paragraph-formatted transcript. Use assemblyai_get_transcript_sentences for sentence-level segments and assemblyai_get_transcript for the full text. The transcript must have status completed (check with assemblyai_get_transcript).
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
assemblyai_get_transcript_sentencesGet a completed AssemblyAI transcript split into sentences.Read-onlyGet Transcript Sentences
Get a completed AssemblyAI transcript split into sentences. Returns the transcript id, confidence, audio_duration and a sentences list, each with text, start and end (milliseconds), confidence, speaker and words. Use this for sentence-level timestamps or captions. Use assemblyai_get_transcript_paragraphs for longer reading blocks and assemblyai_get_transcript for the full text. The transcript must have status completed (check with assemblyai_get_transcript).
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
assemblyai_list_llm_modelsList the LLMs available through AssemblyAI's LLM Gateway.Read-onlyList LLM Models
List the LLMs available through AssemblyAI's LLM Gateway. Returns data, one entry per model with id, name, creator, context_length, supported_parameters, pricing, available_regions, retirement_date and benchmark performance. Use this to pick a valid model ID for assemblyai_create_chat_completion.
Inputs
This tool takes no inputs.
assemblyai_list_transcriptsList transcripts created with this API key's project in the last 30 days, newest first, optionally filtered by status or creation date.Read-onlyList Transcripts
List transcripts created with this API key's project in the last 30 days, newest first, optionally filtered by status or creation date. Returns transcripts (id, status, audio_url, created, completed, error) and page_details with prev_url and next_url links. Use this to find transcript IDs. Use assemblyai_get_transcript to read one transcript. To page back, pass the last transcript's id as before_id.
Inputs
after_idstring- Only return transcripts created after the transcript with this ID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
before_idstring- Only return transcripts created before the transcript with this ID. Use the oldest ID from the previous page to get the next (older) page. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
created_onstring- Only return transcripts created on this date (YYYY-MM-DD). Example: 2026-10-01
limitinteger- Maximum number of transcripts to return, 1 to 200. Defaults to 10. Example: 50
statusstring- Only return transcripts with this status: queued, processing, completed or error.one of
queuedprocessingcompletederror throttled_onlyboolean- Set to true to return only throttled transcripts. Overrides status.
assemblyai_search_transcript_wordsSearch a completed AssemblyAI transcript for words, numbers or short phrases (up to five words each).Read-onlySearch Transcript Words
Search a completed AssemblyAI transcript for words, numbers or short phrases (up to five words each). Returns total_count and, for each search term, the matched text, count, timestamps ([start, end] in milliseconds) and word indexes. Use this to find where something was said in one transcript. Use assemblyai_get_transcript to read the whole text. The transcript must have status completed (check with assemblyai_get_transcript).
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
wordsstringrequired- Comma-separated list of words, numbers or phrases (up to five words each) to search for. Matching is case-insensitive. Example: refund,cancel my account,42
assemblyai_create_transcriptSubmit an audio or video file at a URL to AssemblyAI for asynchronous transcription, with optional speaker labels, language detection, PII redaction and audio intelligence.WriteCreate Transcript
Submit an audio or video file at a URL to AssemblyAI for asynchronous transcription, with optional speaker labels, language detection, PII redaction and audio intelligence. Returns the new transcript object with its id and status queued. Use this to start a transcription. Poll assemblyai_get_transcript until status is completed or error, then read it with assemblyai_get_transcript_sentences, assemblyai_get_transcript_paragraphs or assemblyai_get_subtitles. The file must be reachable at a URL; uploading local files is not supported.
Inputs
audio_urlstringrequired- Public or pre-signed URL of the audio or video file to transcribe (MP3, WAV, MP4, M4A, FLAC, WebM and other common formats). AssemblyAI downloads the file itself, so the URL must be reachable without login. Local file uploads are not supported by this tool. Example: https://assembly.ai/wildfires.mp3
audio_end_atinteger- Stop transcribing at this point in the file, in milliseconds. Example: 60000
audio_start_frominteger- Start transcribing at this point in the file, in milliseconds. Example: 5000
auto_highlightsboolean- Set to true to extract key phrases. Results appear in auto_highlights_result.
content_safetyboolean- Set to true to flag sensitive content (hate speech, violence, profanity and similar). Results appear in content_safety_labels.
content_safety_confidenceinteger- Confidence threshold (25 to 100, default 50) for content moderation labels. Only used when content_safety is true. Example: 60
custom_spellingarray- Spelling replacements applied to the transcript. Each item maps one or more spoken forms (from, each up to 5 words) to a single replacement word (to). Example: [{"from": ["dicarlo"], "to": "DeCarlo"}]
disfluenciesboolean- Set to true to keep filler words such as um and uh in the transcript.
domainstring- Set to medical-v1 to enable Medical Mode, which improves accuracy for medications, procedures, conditions and dosages (English, Spanish, German and French). Leave empty for general audio.one of
medical-v1 entity_detectionboolean- Set to true to detect named entities (people, organizations, locations, dates and so on). Results appear in entities.
filter_profanityboolean- Set to true to mask profanity in the transcript text.
format_textboolean- Apply casing and text formatting (numbers, dates and so on). Defaults to true. PII redaction needs it on.
iab_categoriesboolean- Set to true to classify the content into IAB topic categories. Results appear in iab_categories_result.
keyterms_promptarray- Domain words or phrases (up to 6 words each) the model should recognize, such as product names, people or technical terms. Up to 1,000 terms for universal-3-5-pro or 200 for universal-2. Example: ["AssemblyAI", "Universal-3", "diarization"]
language_codestring- Language of the audio, as an AssemblyAI language code such as en, en_us, en_uk, es, fr, de, hi, ja or zh. Leave empty to detect the language automatically. Do not combine with language_detection true. Supported codes: https://www.assemblyai.com/docs/pre-recorded-audio/supported-languages. Example: en_us
language_codesarray- Languages spoken in a code-switched file (speakers mixing languages). One of the codes must be en. Example: ["en", "es"]
language_confidence_thresholdnumber- Minimum confidence (0 to 1) for the automatically detected language. The transcript fails with an error when detection confidence is lower. Only valid with language detection. Example: 0.7
language_detectionboolean- Automatic language detection. Applied automatically when language_code is empty. Set to false only together with a language_code.
language_detection_optionsobject- Options for automatic language detection. expected_languages limits detection to the listed codes, fallback_language is used when the detected language is not in that list (auto lets the model pick), code_switching true detects speakers switching languages mid-file, and on_no_speech_detected chooses between failing (error, the default) or completing with empty text (fallback). Example: {"expected_languages": ["en", "es"], "fallback_language": "en"}
multichannelboolean- Set to true to transcribe each audio channel separately (for example a call recorded with one speaker per channel). Words and utterances then carry a channel number.
promptstring- Free-text context of up to 1,500 words to guide the universal-3-5-pro model, such as the topic, names, jargon or the desired style. Ignored by other models. Example: A customer support call about a billing dispute for Acme Cloud.
punctuateboolean- Add punctuation to the transcript. Defaults to true. Speaker labels and sentiment analysis need it on.
redact_piiboolean- Set to true to redact personal information (names, phone numbers, card numbers and so on) from the transcript text. Requires format_text true. Choose the categories with redact_pii_policies.
redact_pii_audioboolean- Set to true to also produce a copy of the audio with the redacted PII beeped out. Requires redact_pii true. Fetch it with assemblyai_get_redacted_audio once the transcript completes.
redact_pii_audio_optionsobject- Options for redacted audio: return_redacted_no_speech_audio true returns a file even when the audio has no speech, and override_audio_redaction_method silence replaces PII with silence instead of a beep. Example: {"override_audio_redaction_method": "silence"}
redact_pii_audio_qualitystring- File type of the redacted audio: mp3 (default) or wav.one of
mp3wav redact_pii_policiesarray- Categories of personal information to redact when redact_pii is true, such as person_name, phone_number, email_address, credit_card_number, us_social_security_number or location. Example: ["person_name", "phone_number", "email_address"]
redact_pii_return_unredactedboolean- Set to true to also return the original text as unredacted_text, unredacted_words and unredacted_utterances. Requires redact_pii true.
redact_pii_substring- How redacted text is replaced: hash (default, replaces each character with #) or entity_name (replaces it with the category, such as [PERSON_NAME]).one of
entity_namehash redact_static_entitiesobject- Extra exact terms to redact, as a map from a label to the list of terms. Redacted with the redact_pii_sub style, on top of PII redaction. Example: {"INTERNAL_TOOL": ["Bearclaw", "Cubclaw"]}
remove_audio_tagsstring- Universal-3.5 Pro can add inline annotations such as audio event markers and speaker cues. all (the default) removes them all, speaker removes only speaker cues and keeps the rest.one of
allspeaker sentiment_analysisboolean- Set to true to score the sentiment (positive, neutral, negative) of each sentence. Results appear in sentiment_analysis_results.
speaker_labelsboolean- Set to true to identify who spoke when (speaker diarization). The transcript then includes utterances with speaker labels A, B, C and so on. Required by speaker identification.
speaker_optionsobject- Range of possible speakers, as min_speakers_expected and max_speakers_expected (min must not exceed max); include_speaker_confidence true adds a per-word speaker confidence. Requires speaker_labels true. Do not combine with speakers_expected. Example: {"min_speakers_expected": 2, "max_speakers_expected": 4}
speakers_expectedinteger- Exact number of speakers to identify. Requires speaker_labels true. Do not combine with speaker_options. Example: 2
speech_modelsarray- Speech models to use, in priority order: universal-3-5-pro (most accurate, supports prompt and temperature) and universal-2 (broadest language coverage). When the first model does not support the audio's language the next one is used. Omit for the default ["universal-3-5-pro", "universal-2"].
speech_thresholdnumber- Reject files where less than this fraction (0 to 1) of the audio is speech. Example: 0.5
speech_understanding_requestobject- Optional speech understanding task to run on the finished transcript, with results returned on the transcript. The speech understanding task to run, as an object with exactly one task key. Translation: {"translation": {"target_languages": ["es", "de"], "formal": true}} (add "match_original_utterance": true with speaker labels to get per-utterance translations). Speaker identification: {"speaker_identification": {"speaker_type": "role", "known_values": ["interviewer", "candidate"]}}, or speaker_type "name" with a speakers list of objects that each have a name (or role) and an optional description. Needs a transcript made with speaker_labels true. Custom formatting: {"custom_formatting": {"date": "mm/dd/yyyy", "phone_number": "(xxx)xxx-xxxx", "email": "username@domain.com"}}. Summarization: {"summarization": {"summary_type": "bullets"}} (bullets or paragraph, optional effort low or medium). Action items: {"action_items": {"include_decisions": true}}.
temperaturenumber- Randomness of the universal-3-5-pro model's output, from 0 (default, most deterministic) to 1. Ignored by other models. Example: 0
assemblyai_run_speech_understandingRun a built-in speech understanding task on a completed AssemblyAI transcript: translation, speaker identification, custom formatting, summarization or action items.WriteRun Speech Understanding
Run a built-in speech understanding task on a completed AssemblyAI transcript: translation, speaker identification, custom formatting, summarization or action items. Returns the task request and its response: translated_texts by language, a speaker label to name or role mapping, formatted text and utterances, or a summary, plus the updated utterances. Use this for those fixed tasks on an existing transcript. Use assemblyai_create_chat_completion for free-form prompts, or pass speech_understanding_request to assemblyai_create_transcript to run the task during transcription. The transcript must be completed; speaker identification also needs it created with speaker_labels true.
Inputs
requestobjectrequired- The speech understanding task to run, as an object with exactly one task key. Translation: {"translation": {"target_languages": ["es", "de"], "formal": true}} (add "match_original_utterance": true with speaker labels to get per-utterance translations). Speaker identification: {"speaker_identification": {"speaker_type": "role", "known_values": ["interviewer", "candidate"]}}, or speaker_type "name" with a speakers list of objects that each have a name (or role) and an optional description. Needs a transcript made with speaker_labels true. Custom formatting: {"custom_formatting": {"date": "mm/dd/yyyy", "phone_number": "(xxx)xxx-xxxx", "email": "username@domain.com"}}. Summarization: {"summarization": {"summary_type": "bullets"}} (bullets or paragraph, optional effort low or medium). Action items: {"action_items": {"include_decisions": true}}.
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
assemblyai_delete_transcriptDelete an AssemblyAI transcript's data permanently.DestructiveDelete Transcript
Delete an AssemblyAI transcript's data permanently. Any file uploaded for it is deleted too. Returns the transcript object with its text and results removed and status kept for reference. Use this to remove a transcript you no longer need, for example for data retention. This cannot be undone.
- Idempotent
Inputs
transcript_idstringrequired- ID of the transcript, as returned by assemblyai_create_transcript or assemblyai_list_transcripts. A UUID. Example: 0072a82b-aa22-4962-add2-6121c36c17c6
No tools match.
