Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.cyberdesk.io/llms.txt

Use this file to discover all available pages before exploring further.

​
Writing Effective Prompts

When creating workflows, clarity and specificity are paramount. Your prompts should be:
  • Unambiguous: Leave no room for interpretation
  • Specific: Include exact details like file paths, button names, or expected values
  • Actionable: Use clear, directive language

​
Example of Good vs Poor Prompting

"Download the report"

​
Understanding Variables

Cyberdesk supports three types of variables in your workflows. All variable types support nested access for structured data.

​
Input Variables: {variable}

  • When defined: Before the workflow starts
  • Purpose: Pass dynamic data like patient names, dates, or IDs
  • Example: {patient_name}, {report_date}, {patient.demographics.dob}, {items[0].id}

​
Runtime Variables: {{variable}}

  • When defined: During workflow execution
  • Purpose: Store values discovered during the workflow for later use
  • Example: {{extracted_invoice_id}}, {{invoice.line_items[0].amount}}
  • Important: Can be set by focused_action, copy_to_clipboard, and extract_prompt flows that use upsert_runtime_values
Key Difference: Input variables are replaced before the agent sees them, runtime variables are set during execution and used by subsequent steps, and sensitive variables are passed verbatim (as aliases) to tools but their plaintext values are never exposed to the agent; they are resolved only during actual computer actions.

​
Sensitive Variables: {$variable}

  • When defined: At run creation (via sensitive_input_values)
  • Purpose: Handle secrets like passwords, SSNs, tokens without exposing plaintext to the agent or logs
  • Example: {$password}, {$credentials.api_key}
  • Behavior:
    • Kept as {$variable} in prompts and passed verbatim to tools
    • Stored securely in a third‑party vault (Basis Theory) only for the duration of the run
    • Never logged, never shown on the dashboard, and never sent to LLMs
    • Resolved to plaintext only at the last mile (e.g., when typing), then deleted from the vault after run completion

​
Nested Access for Structured Data

Instead of creating many flat variables, pass structured JSON and access nested properties: 
{patient.demographics.first_name}
{patient.insurance.policy_number}
{patient.emergency_contacts[0].phone}
See Structured Inputs for full documentation on nested access syntax and error handling.

​
Utilizing Prompt Images

Images can anchor your instructions with visual context. Add screenshots alongside your text in the dashboard editor to help the agent recognize ambiguous icons, menu locations, or tricky UI flows (especially in legacy apps).
  • Place images near the paragraph they illustrate for best results
  • Use images as guidance – the agent will still take fresh screenshots during execution
  • Keep images focused on the relevant UI area; include multiple images if needed
Runtime access to prompt images is available via the grab_reference_image tool by filename (tail only). After analyzing any reference or zoomed image, the agent should take a fresh full screenshot before issuing coordinate actions. When specifying zoom regions, consider leaving a small margin so labels/edges aren’t cropped.
Use your OS’s native Markup/annotation tools to add red circles and arrows around the exact UI elements you want the AI to focus on. Simple visual callouts dramatically improve recognition.

​
Understanding Trajectories and Cached Tools

Cyberdesk’s intelligent caching system uses trajectories to record successful workflow executions and replay them deterministically. When a workflow has an approved trajectory, it can complete in seconds instead of minutes. To maximize this feature while still doing dynamic actions, you must properly prompt the agent to use special dynamic tools.
More on Trajectories: Learn more in Trajectories 101.

​
Dynamic Tools in Cached Workflows

Even during cached trajectory replay, certain tools execute dynamically to capture fresh data: By explicitly prompting the agent to use these tools, you ensure dynamic data capture even when the rest of the workflow is cached.

​
The Essential Workflow Tools

​
1. Focused Action

Capture dynamic content or make context-aware decisions during cached runs.

When to use

  • Selecting different items from a list each run
  • Extracting dynamic data that changes between runs
  • Making decisions based on screen content
In prompts, refer to it as "focused_action"

​
2. Mark File for Export

Export files from the remote machine after workflow completion.

When to use

  • Downloading generated reports
  • Extracting processed files
  • Saving workflow outputs
In prompts, refer to it as "mark_file_for_export"

​
3. Save Screenshot as Run Attachment

Capture important moments as screenshots accessible after the run.

When to use

  • Documenting visual confirmation
  • Capturing patient X-rays or medical images
  • Recording transaction confirmations
In prompts, refer to it as "save_screenshot_as_run_attachment"

​
4. Grab Reference Image

Retrieve a user-provided prompt image during execution by filename (tail only).

When to use

  • Provide visual context to disambiguate UI elements
  • Help the agent recognize specific screens, dialogs, or popups
In prompts, refer to it as "grab_reference_image". After analyzing, take a fresh full screenshot before coordinate actions. See the dedicated page for details.

​
5. Copy to Clipboard

Execute Ctrl+C and automatically save clipboard contents as a runtime variable.

When to use

  • Extracting text from legacy systems
  • Capturing copyable values from read-only or context-menu-copy fields
  • Getting data from read-only fields
  • Quick text extraction with manual selection
In prompts, refer to it as "copy_to_clipboard"

​
6. Screenshot with Extract Prompt

Vision-based data extraction with flexible async processing modes (synchronous, batch-scoped, or run-scoped) and ability to set a runtime value

When to use

  • Non-copyable content (images, PDFs, charts, tables)
  • Parallel extraction across multiple views
  • When navigation doesn’t depend on extraction results
  • Quickly setting one off runtime values
In prompts, refer to it as "extract_prompt" parameter on screenshot action. For parallel processing, use process_async="batch" or process_async="run". See the dedicated page for full details on async modes.

​
Quick Start: Simple Extraction

"Take a screenshot with extract_prompt='Extract customer name, email, and 
phone number as JSON: {name: string, email: string, phone: string}'"

​
Parallel Extraction Across Multiple Screens

Make extraction 3-5x faster by using process_async="batch" when scrolling or navigating: 
"In one batched tool call, extract all product data by scrolling through the catalog:
- Take screenshot with extract_prompt='Extract all visible products as JSON 
  array' and process_async='batch'
- Scroll down
- Take screenshot with extract_prompt='Extract all visible products as JSON 
  array' and process_async='batch'
- Repeat until bottom

All extractions process in parallel before the next agent step."

​
Run-Scoped Extraction (Fully Non-Blocking)

For large extractions that don’t affect navigation, use process_async="run": 
"Take screenshot with extract_prompt='Extract complete dashboard analytics 
as detailed JSON, saving as a runtime value.' and process_async='run'.

Continue with other workflow tasks. The extraction runs in background and 
will be included in final output automatically."
Learn More: See the Extract Prompt page for detailed documentation on async modes, runtime variable integration, and advanced patterns.

​
7. Execute Terminal Command

Run PowerShell commands on the Windows machine.

When to use

  • System administration tasks
  • File operations
  • API calls or data processing
In prompts, refer to it as "execute_terminal_command"
If a workflow only needs a known set of terminal actions, configure the workflow’s terminal command allowlist in the editor. When present, Cyberdesk enforces that allowlist server-side before the command is sent to the machine.
Allowlist entries can use input, sensitive, runtime, and loop-item placeholders. Cyberdesk accepts either the template form or the resolved command for the current run.

​
8. Declare Task Failed

Terminate workflow execution when failure conditions are met.

When to use

  • Error handling
  • Validation failures
  • Preventing unnecessary continuation
In prompts, refer to it as "declare_task_failed"

​
Additional Advanced Tools

These are also important parts of the workflow-prompting surface, even if you do not use them in every workflow.

​
Declare Task Succeeded

Allow a focused action to end the workflow early when a specific dynamic success condition is met.
In prompts, refer to it as "declare_task_succeeded". Use it only inside focused_action when early success is explicitly intended.

​
Get Main Instructions

Let a focused action reread the main workflow instructions when it needs broader context to recover from an unexpected state.
In prompts, refer to it as "get_main_instructions". It is focused-action-only and should be granted deliberately.

​
Looping Tools

Use start_loop, end_loop_iteration, and skip_loop_iteration for repeated patterns across arrays or counts.
Use loops when the workflow genuinely repeats the same block of work across many items. Access the current item via {{loop_item}}.

​
Upsert Runtime Values

Use upsert_runtime_values when the agent should write or merge runtime values explicitly instead of only returning observations.
This is especially useful in focused actions and extraction flows when later steps depend on discovered values like {{report_path}}, {{missing_file}}, or accumulated arrays/objects.

​
Model Parameter

Use model="..." when a specific focused_action or extract_prompt step benefits from a different model than the workflow default.
Use model overrides sparingly. They are best for unusually hard reasoning or extraction tasks.

​
Complex Drag Actions

Use left_mouse_down and left_mouse_up when left_click_drag is not enough, such as drag flows that require hover pauses or multi-step hold behavior.
If a simple drag is enough, prefer left_click_drag. Reach for the lower-level drag actions only when the UI genuinely needs them.

​
Quick Examples

​
Using Focused Action

"When you reach the patient list, use focused_action to select the patient 
whose name matches {patient_name} and extract their ID number"

​
Exporting Files

"After generating the report, use mark_file_for_export to export the file 
located at C:\Reports\output.pdf"

​
Capturing Screenshots

"Once the payment confirmation appears, use save_screenshot_as_run_attachment 
to capture the confirmation screen as 'payment_confirmation.png'"

​
Copying to Clipboard

"Triple-click on the account number field to select it, then use 
copy_to_clipboard with key name 'account_number' to save it as 
{{account_number}} for use in the next form"

​
Running Terminal Commands

"Use execute_terminal_command to run 'Get-ChildItem C:\Exports\*.pdf | 
ConvertTo-Json' to list all PDF files in the exports folder"
For long-running commands: 
"Use execute_terminal_command with duration=120 (2 minutes) to run the report 
generation script and wait for completion before proceeding"

​
Handling Failures

"If the login screen shows 'Account Locked' or 'Invalid License', 
use declare_task_failed to terminate with message 'Unable to access system'"

​
Next Steps

For detailed information about each tool, including advanced usage patterns and real-world examples, explore some of the individual tool documentation pages:

Focused Action

Dynamic observations and decisions

Mark File for Export

File extraction from workflows

Save Screenshot

Visual documentation

Extract Prompt

Vision-based extraction with optional async extraction

Copy to Clipboard

Copy text and save as runtime variable

Grab Reference Image

Use prompt images at runtime

Execute Terminal

PowerShell automation

Declare Task Failed

Error handling

Declare Task Succeeded

End early on dynamic success

Get Main Instructions

Let focused actions recover with broader context

Looping Tools

Repeat stable blocks across items

Execute Batch Tools

Bundle multiple actions into one fast trajectory step

Upsert Runtime Values

Write and merge runtime values explicitly

Model Parameter

Override models for specific actions

Complex Drag Actions

Advanced drag and hold behavior