# Usage-Based Billing Source: https://docs.cyberdesk.io/additional-details/usage-based-billing Understand how Cyberdesk meters and bills your usage Cyberdesk uses usage-based billing to charge for automation runs. This page explains how billing works, what counts as billable usage, and how edge cases are handled. ## Step-Based Billing Step-based billing is the primary billing model. You are charged based on the number of steps executed during your runs. The specific rates per agentic step and cached step are determined by your contract with Cyberdesk. ### What is a Step? There are two types of billable steps: Each time the AI agent makes a decision and takes action. This includes: * Every agent iteration (reasoning + tool calls) * Each focused action execution * Each extraction prompt (after any LLM retries) **Note:** A batch of tool calls in a single agent turn counts as **one** agentic step. Each time a previously-learned action is replayed from cache (trajectory replay). This includes: * Every step executed from a matched trajectory * Replayed batched tool calls count as one cached step **per tool call inside the batch** * Steps with `skip_cache_detection` enabled or disabled **Note:** Cache detection settings don't affect billing—you're billed the same whether a step uses cache detection or not. Cached steps are faster and cheaper than agentic steps. ### Billing by Run Outcome What gets billed depends on how your run completes: | Outcome | Steps Billed? | Description | | -------------------------- | ------------- | ------------------------------------------------------------ | | **Success** | ✅ Yes | Run completed successfully | | **User Task Failed** | ✅ Yes | Agent explicitly failed the task per your instructions | | **Cancelled** | ✅ Yes | You cancelled the run mid-execution | | **Infrastructure Failure** | ❌ No | Cyberdesk system error (e.g., worker crash, connection lost) | **Infrastructure failures are never billed.** If our systems fail (worker crashes, connection timeouts, etc.), you won't be charged for any usage from that run attempt. ### Understanding "User Task Failed" vs "Infrastructure Failure" When a run ends in error, the billing outcome depends on *why* it failed: The agent explicitly called `declare_task_failed` because: * The task was impossible to complete (e.g., "file not found") * The application was in an unexpected state * The user's instructions led to a dead end This is billable because the agent did the work of determining the task couldn't be completed. The run failed due to Cyberdesk system issues: * Cyberdriver became unresponsive after multiple retries * Worker crashed or lost connection * Internal service errors You are never charged when our infrastructure fails. ### Run Retries If you retry a failed run, billing accumulates across attempts: ``` Attempt 1: 10 agentic steps, 5 cached steps (Success → billed) Attempt 2: 3 agentic steps, 0 cached steps (Infra Failure → not billed) Attempt 3: 7 agentic steps, 2 cached steps (Success → billed) ──────────────────────────────────────────────────────────────── Total billed: 17 agentic steps, 7 cached steps ``` Each attempt's usage is tracked separately, and the totals are cumulative. You can see per-attempt breakdowns in the run's `usage_metadata`. ### Viewing Usage Data You can view your usage data in multiple ways through the Cyberdesk Dashboard. #### Individual Run Usage On any run's detail page, scroll down to the **Run Message History** section. You'll see a **Usage Data Summary** card that displays: * Total agentic steps billed * Total cached steps billed * Billing outcome status (success, cancelled, etc.) Click **View Details** to see a full breakdown including per-attempt usage for retried runs. #### Aggregated Usage Across Runs On the **Runs** table page, select multiple runs using the checkboxes. The toolbar will automatically display the accumulated totals: * Total agentic steps billed across selected runs * Total cached steps billed across selected runs Use the **Created Date** filter to narrow down runs to a specific time period, and expand **Rows per page** to see more runs at once. #### API Access Every run includes a `usage_metadata` field with detailed billing information: ```json theme={null} { "billing_outcome": "success", "billing_reason": "Run successful - fully billed", "total_agentic_steps_billed": 15, "total_cached_steps_billed": 8, "total_agentic_steps": 15, "total_cached_steps": 8 } ``` For retried runs, you'll also see `individual_tries_metadata` with per-attempt details. #### Programmatic Usage Data You can programmatically retrieve aggregated usage data via the SDK. This is the same data that powers the **Billing and Usage** dashboard in Cyberdesk. ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!); // Get usage for January 2025 const { data: usage } = await client.usage.aggregate({ from_date: new Date('2025-01-01'), to_date: new Date('2025-01-31'), mode: 'simulated' // or 'billed' for customers on Stripe billing }); if (usage) { console.log(`Period: ${usage.period_start} to ${usage.period_end}`); console.log(`Runs counted: ${usage.runs_counted}`); console.log(`Agentic steps: ${usage.total_agentic_steps}`); console.log(`Cached steps: ${usage.total_cached_steps}`); } ``` ```python theme={null} from datetime import datetime from cyberdesk import CyberdeskClient, UsageMode import os async with CyberdeskClient(os.environ['CYBERDESK_API_KEY']) as client: # Get usage for January 2025 response = await client.usage.aggregate( from_date=datetime(2025, 1, 1), to_date=datetime(2025, 1, 31), mode=UsageMode.SIMULATED # or UsageMode.BILLED ) if response.data: usage = response.data print(f"Period: {usage.period_start} to {usage.period_end}") print(f"Runs counted: {usage.runs_counted}") print(f"Agentic steps: {usage.total_agentic_steps}") print(f"Cached steps: {usage.total_cached_steps}") ``` ```python theme={null} from datetime import datetime from cyberdesk import CyberdeskClient, UsageMode import os client = CyberdeskClient(os.environ['CYBERDESK_API_KEY']) # Get usage for January 2025 response = client.usage.aggregate_sync( from_date=datetime(2025, 1, 1), to_date=datetime(2025, 1, 31), mode=UsageMode.SIMULATED ) if response.data: usage = response.data print(f"Runs counted: {usage.runs_counted}") print(f"Agentic steps: {usage.total_agentic_steps}") print(f"Cached steps: {usage.total_cached_steps}") ``` **Response fields:** | Field | Description | | --------------------- | ---------------------------------------------------- | | `total_agentic_steps` | Total agentic steps in the period | | `total_cached_steps` | Total cached steps in the period | | `runs_counted` | Number of runs included in the aggregation | | `period_start` | Start of the queried period | | `period_end` | End of the queried period | | `mode` | The mode used for counting (`simulated` or `billed`) | **Usage modes:** * **`simulated`** (default): Uses `total_agentic_steps` and `total_cached_steps` from each run's `usage_metadata`, but excludes runs where `billing_outcome` is `infra_failure`. Use this for customers not yet on Stripe billing to see what they *would* be charged. * **`billed`**: Uses `total_agentic_steps_billed` and `total_cached_steps_billed` — the actual amounts sent to Stripe. Use this for customers on active Stripe billing. Use this endpoint to build custom usage dashboards, automate billing reconciliation, or integrate usage data into your own systems. **For early 2025 customers:** Billed usage data will only appear if you have worked with the Cyberdesk team to migrate your current Stripe invoice to usage-based billing pricing. Please [contact us](mailto:team@cyberdesk.io) if you need help with the migration. *** ## FAQ You'll be billed for any steps that were executed before cancellation. No. If an LLM call fails and retries internally, it still counts as one agentic step. You're only charged once the step completes (successfully or not). Yes. Cached steps replay previously-learned actions without invoking the AI, making them faster and cheaper. Check the `billing_outcome` field in `usage_metadata`: * `"infra_failure"` = Infrastructure issue (not billed) * `"user_task_failed"` = Agent determined task couldn't be completed (billed) Visit the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard/runs) to view your usage. You can see individual run usage in the run detail page, or select multiple runs in the runs table to see aggregated totals. *** ## Run-Based Billing (Legacy) Run-based billing is a legacy model. New customers use step-based billing. In the legacy model, you are charged a flat fee per completed run, regardless of the number of steps. ### How It Works A `run_completed` billing event is recorded when: * A run completes with status `success` * A run completes with status `error` due to explicit task failure (user-initiated via `declare_task_failed`) A `run_completed` event is **not** recorded when: * The run is cancelled * The run fails due to infrastructure issues ### Legacy Usage Metadata Legacy runs include additional fields in `usage_metadata`: ```json theme={null} { "billing_outcome": "success", "billing_reason": "Run successful - fully billed", "total_run_completed_billed": 1, "total_agentic_steps_billed": 15, "total_cached_steps_billed": 8, "total_agentic_steps": 15, "total_cached_steps": 8 } ``` The `total_run_completed_billed` field indicates whether the run completion was billed (1) or not (0). # Click mouse button Source: https://docs.cyberdesk.io/api-reference/computer/click-mouse-button /openapi.json post /v1/computer/{machine_id}/input/mouse/click Click the mouse button at specified coordinates. If coordinates are not provided, clicks at current position. # Copy to clipboard via Ctrl+C Source: https://docs.cyberdesk.io/api-reference/computer/copy-to-clipboard-via-ctrl+c /openapi.json post /v1/computer/{machine_id}/copy_to_clipboard Execute Ctrl+C and return clipboard contents with the specified key name. # Disable remote keepalive Source: https://docs.cyberdesk.io/api-reference/computer/disable-remote-keepalive /openapi.json post /v1/computer/{machine_id}/internal/keepalive/remote/disable # Drag mouse (native) Source: https://docs.cyberdesk.io/api-reference/computer/drag-mouse-native /openapi.json post /v1/computer/{machine_id}/input/mouse/drag Perform a drag operation using absolute coordinates with required start and optional duration. # Enable remote keepalive Source: https://docs.cyberdesk.io/api-reference/computer/enable-remote-keepalive /openapi.json post /v1/computer/{machine_id}/internal/keepalive/remote/enable # Execute PowerShell command Source: https://docs.cyberdesk.io/api-reference/computer/execute-powershell-command /openapi.json post /v1/computer/{machine_id}/shell/powershell/exec Execute PowerShell command on the machine. # Get Cyberdriver diagnostics Source: https://docs.cyberdesk.io/api-reference/computer/get-cyberdriver-diagnostics /openapi.json get /v1/computer/{machine_id}/internal/diagnostics Fetch diagnostic details from the Cyberdriver tunnel agent. # Get display dimensions Source: https://docs.cyberdesk.io/api-reference/computer/get-display-dimensions /openapi.json get /v1/computer/{machine_id}/display/dimensions Get the display dimensions of the machine. # Get mouse position Source: https://docs.cyberdesk.io/api-reference/computer/get-mouse-position /openapi.json get /v1/computer/{machine_id}/input/mouse/position Get the current mouse cursor position. # List directory contents Source: https://docs.cyberdesk.io/api-reference/computer/list-directory-contents /openapi.json get /v1/computer/{machine_id}/fs/list List directory contents on the machine. # Manage PowerShell session Source: https://docs.cyberdesk.io/api-reference/computer/manage-powershell-session /openapi.json post /v1/computer/{machine_id}/shell/powershell/session Create or destroy PowerShell sessions on the machine. # Move mouse cursor Source: https://docs.cyberdesk.io/api-reference/computer/move-mouse-cursor /openapi.json post /v1/computer/{machine_id}/input/mouse/move Move the mouse cursor to specified coordinates. # Read file contents Source: https://docs.cyberdesk.io/api-reference/computer/read-file-contents /openapi.json get /v1/computer/{machine_id}/fs/read Read file contents from the machine (base64 encoded). # Record remote keepalive activity Source: https://docs.cyberdesk.io/api-reference/computer/record-remote-keepalive-activity /openapi.json post /v1/computer/{machine_id}/internal/keepalive/remote/activity # Run PowerShell simple smoke command Source: https://docs.cyberdesk.io/api-reference/computer/run-powershell-simple-smoke-command /openapi.json post /v1/computer/{machine_id}/shell/powershell/simple Run Cyberdriver's built-in simple PowerShell smoke command. # Run PowerShell test smoke command Source: https://docs.cyberdesk.io/api-reference/computer/run-powershell-test-smoke-command /openapi.json post /v1/computer/{machine_id}/shell/powershell/test Run Cyberdriver's built-in PowerShell test command. # Scroll mouse wheel Source: https://docs.cyberdesk.io/api-reference/computer/scroll-mouse-wheel /openapi.json post /v1/computer/{machine_id}/input/mouse/scroll Scroll the mouse wheel in the specified direction by a number of steps. Optionally moves to (x, y) before scrolling. # Self-update Cyberdriver Source: https://docs.cyberdesk.io/api-reference/computer/self-update-cyberdriver /openapi.json post /v1/computer/{machine_id}/internal/update Trigger a Cyberdriver update through the tunnel. # Send key combination Source: https://docs.cyberdesk.io/api-reference/computer/send-key-combination /openapi.json post /v1/computer/{machine_id}/input/keyboard/key Send a key combination in XDO-style syntax (e.g., 'ctrl+a', 'alt+tab'). Join keys with `+` to press them together as one chord (e.g. `ctrl+a`); a chord may combine multiple modifiers (`ctrl`/`alt`/`shift`/`win`) but only one non-modifier key. Separate keys with spaces to press them one after another (e.g. `down down down`). Using `+` between repeated or multiple non-modifier keys (e.g. `down+down+down`) is invalid — use spaces instead (`down down down`). When `down` is omitted, the key is pressed and released normally. Pass `down=true` to press and hold the key (without releasing), and `down=false` to release a key that was previously held. # Shutdown Cyberdriver via tunnel Source: https://docs.cyberdesk.io/api-reference/computer/shutdown-cyberdriver-via-tunnel /openapi.json post /v1/machines/{machine_id}/cyberdriver/shutdown Request CyberDriver process shutdown through the existing tunnel proxy path. # Take a screenshot Source: https://docs.cyberdesk.io/api-reference/computer/take-a-screenshot /openapi.json get /v1/computer/{machine_id}/display/screenshot Take a screenshot of the machine's display. # Type text Source: https://docs.cyberdesk.io/api-reference/computer/type-text /openapi.json post /v1/computer/{machine_id}/input/keyboard/type Type text on the machine's keyboard. # Write file contents Source: https://docs.cyberdesk.io/api-reference/computer/write-file-contents /openapi.json post /v1/computer/{machine_id}/fs/write Write file contents to the machine. # Create Connection Source: https://docs.cyberdesk.io/api-reference/connections/create-connection /openapi.json post /v1/connections Create a new connection (typically done by the WebSocket handler). The machine must exist and belong to the authenticated organization. # Delete Connection Source: https://docs.cyberdesk.io/api-reference/connections/delete-connection /openapi.json delete /v1/connections/{connection_id} Delete a connection (not typically used - connections are managed automatically). The connection must belong to a machine owned by the authenticated organization. # Get Connection Source: https://docs.cyberdesk.io/api-reference/connections/get-connection /openapi.json get /v1/connections/{connection_id} Get a specific connection by ID. The connection must belong to a machine owned by the authenticated organization. # List Connections Source: https://docs.cyberdesk.io/api-reference/connections/list-connections /openapi.json get /v1/connections List all connections for the authenticated organization's machines. Supports pagination and filtering by machine and status. Returns connections with their associated machine data. # Update Connection Source: https://docs.cyberdesk.io/api-reference/connections/update-connection /openapi.json patch /v1/connections/{connection_id} Update a connection's status or timestamps. Only the fields provided in the request body will be updated. The connection must belong to a machine owned by the authenticated organization. # Address Book Peers Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/address-book-peers /openapi.json post /api/ab/peers # Address Book Personal Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/address-book-personal /openapi.json post /api/ab/personal # Address Book Settings Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/address-book-settings /openapi.json post /api/ab/settings # Address Book Shared Profiles Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/address-book-shared-profiles /openapi.json post /api/ab/shared/profiles # Address Book Tags Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/address-book-tags /openapi.json post /api/ab/tags/{_profile_guid} # Authorize Cyberdriver Connection Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/authorize-cyberdriver-connection /openapi.json post /api/cyberdriver/authorize-connection # Current User Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/current-user /openapi.json post /api/currentUser # Device Group Accessible Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/device-group-accessible /openapi.json get /api/device-group/accessible # Heartbeat Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/heartbeat /openapi.json get /api/heartbeat # Legacy Address Book Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/legacy-address-book /openapi.json get /api/ab # Login Options Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/login-options /openapi.json get /api/login-options # Mint Cyberdriver Connection Token Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/mint-cyberdriver-connection-token /openapi.json post /api/cyberdriver/connection-token # Oidc Auth Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/oidc-auth /openapi.json post /api/oidc/auth # Oidc Auth Query Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/oidc-auth-query /openapi.json get /api/oidc/auth-query # Oidc Callback Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/oidc-callback /openapi.json get /api/oidc/callback # Oidc Confirm Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/oidc-confirm /openapi.json post /api/oidc/confirm # Peers Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/peers /openapi.json get /api/peers # Save Legacy Address Book Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/save-legacy-address-book /openapi.json post /api/ab # Users Source: https://docs.cyberdesk.io/api-reference/cyberdriver-auth/users /openapi.json get /api/users # Downed machines webhook Source: https://docs.cyberdesk.io/api-reference/downed-machines-webhook /openapi.json webhook downed_machines Webhook schema for machines that appear down. Documentation-only. # Health Source: https://docs.cyberdesk.io/api-reference/health /openapi.json get /health Backward compatible health check - redirects to readiness check. # Health Db Source: https://docs.cyberdesk.io/api-reference/health-db /openapi.json get /health/db Legacy endpoint - checks database only. # Health Live Source: https://docs.cyberdesk.io/api-reference/health-live /openapi.json get /health/live Liveness probe - Is the process alive? Always returns 200 if the server can respond. Use this for: Kubernetes liveness probes, basic "is it up" checks. # Health Ready Source: https://docs.cyberdesk.io/api-reference/health-ready /openapi.json get /health/ready Readiness probe - Can the service handle requests? Checks all critical dependencies with timeouts. Use this for: Load balancer health checks, Better Stack monitoring. Returns 503 if any critical dependency is down. # Database Health Check Source: https://docs.cyberdesk.io/api-reference/health/database-health-check /openapi.json get /v1/health/db Database health check endpoint. Verifies database connectivity without authentication. # Health Check Source: https://docs.cyberdesk.io/api-reference/health/health-check /openapi.json get /v1/health Basic health check endpoint. Returns the service status without authentication. # Health Machines Source: https://docs.cyberdesk.io/api-reference/health/health-machines /openapi.json get /v1/health/machines Probe the dimensions endpoint for all connected machines in scope. Scope rules: - Standard keys/JWT: only machines in the authenticated organization - Cyberdesk root API key: machines across all organizations # API Reference Source: https://docs.cyberdesk.io/api-reference/introduction Complete reference for all Cyberdesk API endpoints ## Overview The Cyberdesk API provides programmatic access to all platform features, enabling you to automate desktop tasks at scale. Whether you're building custom integrations, managing fleets of machines, or orchestrating complex workflows, our API gives you full control. View the complete OpenAPI specification ## Base URL All API requests should be made to: ``` https://api.cyberdesk.io ``` ## Authentication Most API endpoints require authentication using Bearer tokens. Include your API key in the Authorization header: ```bash theme={null} Authorization: Bearer YOUR_API_KEY ``` You can find your API key in the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard) under Settings. The public health endpoints, `/v1/health` and `/v1/health/db`, do not require authentication. Keep your API key secure and never expose it in client-side code or public repositories. ## Core Resources The Cyberdesk API is organized around these main resources: Virtual or physical desktops connected via Cyberdriver Automation blueprints that define tasks to be executed Instances of workflow executions on specific machines Active connections between machines and Cyberdesk Recorded sequences of actions for workflow replay ## Quick Start with SDKs While you can use the API directly, we recommend using our official SDKs for a better developer experience: Type-safe client with full IntelliSense support Async/sync client with comprehensive type hints ## Rate Limits Authenticated API requests are subject to rate limits. Dashboard-authenticated JWT traffic uses a 60-second fixed window with default limits of `480` requests per user and `4000` requests per organization. These dashboard JWT limits are enforced internally and do not expose public `X-RateLimit-*` headers. Public API key traffic uses separate rate-limit buckets. When those limits are enabled for your environment, responses include standard headers such as `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset`, and blocked requests return HTTP `429 Too Many Requests` with a `Retry-After` header. If you have concerns about rate limits or need higher limits for your specific use case, please [contact our team](mailto:founders@cyberdesk.io) and we'll work with you to find a solution. Public endpoints such as health checks are exempt from authentication and HTTP rate limiting. ## Error Handling The API uses standard HTTP status codes to indicate success or failure: | Status Code | Description | | ----------- | ---------------------------------------------- | | 200 | Success | | 201 | Created | | 400 | Bad Request - Invalid parameters | | 401 | Unauthorized - Invalid or rejected credentials | | 403 | Forbidden - Missing credentials | | 404 | Not Found - Resource doesn't exist | | 429 | Too Many Requests - Rate limit exceeded | | 500 | Internal Server Error | Most application errors are returned in a structured JSON body: ```json theme={null} { "error": "HTTPException", "message": "Not authenticated", "error_code": null, "details": null, "timestamp": "2026-03-27T16:00:00Z" } ``` Request validation errors from FastAPI use a different shape with a `detail` array. Check the OpenAPI schema for the exact response model used by each endpoint. ## Pagination List endpoints support pagination using `skip` and `limit` parameters: ```bash theme={null} GET /v1/runs?skip=20&limit=10 ``` Paginated responses follow this structure: ```json theme={null} { "items": [...], "total": 100, "skip": 20, "limit": 10 } ``` ## Including Related Resources Many endpoints support the `include` query parameter to fetch related resources in a single request, following the JSON:API pattern. This reduces the number of API calls needed to get complete data. ### How It Works When you use the `include` parameter, related resources are returned in a separate `included` array in the response. The original ID fields remain unchanged for backward compatibility. ```bash theme={null} GET /v1/runs/abc-123?include=workflow,machine ``` Response: ```json theme={null} { "id": "abc-123", "workflow_id": "wf-456", "machine_id": "mach-789", "status": "success", "included": [ { "type": "workflow", "id": "wf-456", "name": "My Workflow", "main_prompt": "..." }, { "type": "machine", "id": "mach-789", "name": "Desktop-1", "status": "connected" } ] } ``` ### Available Includes | Resource | Includable Fields | Example | | ---------- | -------------------------------------- | --------------------------- | | Run | `workflow`, `machine`, `machine.pools` | `?include=workflow,machine` | | Trajectory | `workflow` | `?include=workflow` | | Machine | `pools` | `?include=pools` | | Pool | `machines` | `?include=machines` | ### SDK Usage ```python theme={null} # Get a run with related workflow and machine run = await client.runs.get( "run-id", include=["workflow", "machine"] ) # Access included resources for resource in run.data.included or []: if resource.type == "workflow": print(f"Workflow: {resource.name}") elif resource.type == "machine": print(f"Machine: {resource.name}") ``` ```typescript theme={null} // Get a run with related workflow and machine const { data: run } = await client.runs.get("run-id", { include: ["workflow", "machine"] }); // Access included resources for (const resource of run.included || []) { if (resource.type === "workflow") { console.log(`Workflow: ${resource.name}`); } else if (resource.type === "machine") { console.log(`Machine: ${resource.name}`); } } ``` ### List Endpoints When using `include` on list endpoints, related resources from all items are deduplicated: ```bash theme={null} GET /v1/runs?include=workflow ``` If multiple runs reference the same workflow, it only appears once in the `included` array. Including many relationships on large lists can impact performance. Use sparingly for list operations. ## Next Steps Browse all available API endpoints in the sidebar Sign in to get your API key from the dashboard Get help from our community and team # Create Cyberdriver Web Session Source: https://docs.cyberdesk.io/api-reference/machines/create-cyberdriver-web-session /openapi.json post /v1/machines/{machine_id}/cyberdriver/web-session Mint a short-lived browser session for Cyberdriver 1.x web streaming. # Create Machine Source: https://docs.cyberdesk.io/api-reference/machines/create-machine /openapi.json post /v1/machines Create a new machine. The machine will be associated with the authenticated organization. # Delete Machine Source: https://docs.cyberdesk.io/api-reference/machines/delete-machine /openapi.json delete /v1/machines/{machine_id} Delete a machine. Associated runs are PRESERVED with machine_id set to NULL. This ensures run history is never lost when a machine is deleted. Connections and request logs are cascade deleted. The machine must belong to the authenticated organization. # Get Machine Source: https://docs.cyberdesk.io/api-reference/machines/get-machine /openapi.json get /v1/machines/{machine_id} Get a specific machine by ID. The machine must belong to the authenticated organization. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. # Get Machine Pools Source: https://docs.cyberdesk.io/api-reference/machines/get-machine-pools /openapi.json get /v1/machines/{machine_id}/pools Get all pools that a machine belongs to. # List Machines Source: https://docs.cyberdesk.io/api-reference/machines/list-machines /openapi.json get /v1/machines List all machines for the authenticated organization. Supports pagination and filtering by status, name, or substring search. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. # Update Machine Source: https://docs.cyberdesk.io/api-reference/machines/update-machine /openapi.json patch /v1/machines/{machine_id} Update a machine's information. Only the fields provided in the request body will be updated. The machine must belong to the authenticated organization. For machine_sensitive_parameters: provide actual secret values and they will be stored in Basis Theory. Only aliases will be stored in the database. # Update Machine Pools Source: https://docs.cyberdesk.io/api-reference/machines/update-machine-pools /openapi.json put /v1/machines/{machine_id}/pools Update a machine's pool assignments. This replaces all existing pool assignments with the new ones. # Create Model Configuration Source: https://docs.cyberdesk.io/api-reference/model-configurations/create-model-configuration /openapi.json post /v1/model-configurations # Delete Model Configuration Source: https://docs.cyberdesk.io/api-reference/model-configurations/delete-model-configuration /openapi.json delete /v1/model-configurations/{model_configuration_id} # Get Model Configuration Source: https://docs.cyberdesk.io/api-reference/model-configurations/get-model-configuration /openapi.json get /v1/model-configurations/{model_configuration_id} # List Model Configurations Source: https://docs.cyberdesk.io/api-reference/model-configurations/list-model-configurations /openapi.json get /v1/model-configurations # Update Model Configuration Source: https://docs.cyberdesk.io/api-reference/model-configurations/update-model-configuration /openapi.json patch /v1/model-configurations/{model_configuration_id} # Add Machines To Pool Source: https://docs.cyberdesk.io/api-reference/pools/add-machines-to-pool /openapi.json post /v1/pools/{pool_id}/machines Add machines to a pool. # Create Pool Source: https://docs.cyberdesk.io/api-reference/pools/create-pool /openapi.json post /v1/pools Create a new pool for organizing machines. # Delete Pool Source: https://docs.cyberdesk.io/api-reference/pools/delete-pool /openapi.json delete /v1/pools/{pool_id} Delete a pool. This will not delete the machines in the pool. # Get Pool Source: https://docs.cyberdesk.io/api-reference/pools/get-pool /openapi.json get /v1/pools/{pool_id} Get a specific pool by ID. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. Note: The `include_machines` parameter is deprecated. Use `include=machines` instead. # List Pools Source: https://docs.cyberdesk.io/api-reference/pools/list-pools /openapi.json get /v1/pools List all pools for the organization. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. # Remove Machines From Pool Source: https://docs.cyberdesk.io/api-reference/pools/remove-machines-from-pool /openapi.json delete /v1/pools/{pool_id}/machines Remove machines from a pool. # Update Pool Source: https://docs.cyberdesk.io/api-reference/pools/update-pool /openapi.json patch /v1/pools/{pool_id} Update a pool's details. # Create Request Log Source: https://docs.cyberdesk.io/api-reference/request-logs/create-request-log /openapi.json post /v1/request-logs Create a new request log. The machine must exist and belong to the authenticated organization. This is typically called when a request is initiated. # Delete Request Log Source: https://docs.cyberdesk.io/api-reference/request-logs/delete-request-log /openapi.json delete /v1/request-logs/{log_id} Delete a request log (not typically used - logs are kept for analytics). The log must belong to a machine owned by the authenticated organization. # Get Request Log Source: https://docs.cyberdesk.io/api-reference/request-logs/get-request-log /openapi.json get /v1/request-logs/{log_id} Get a specific request log by ID. The log must belong to a machine owned by the authenticated organization. # List Request Logs Source: https://docs.cyberdesk.io/api-reference/request-logs/list-request-logs /openapi.json get /v1/request-logs List all request logs for the authenticated organization's machines. Supports pagination and filtering by machine, HTTP method, and status code. # Update Request Log Source: https://docs.cyberdesk.io/api-reference/request-logs/update-request-log /openapi.json patch /v1/request-logs/{log_id} Update a request log with response data. This is typically called when a request completes to add the response details. Only the fields provided in the request body will be updated. The log must belong to a machine owned by the authenticated organization. # Root Source: https://docs.cyberdesk.io/api-reference/root /openapi.json get / Root endpoint with service information # Create Run Attachment Source: https://docs.cyberdesk.io/api-reference/run-attachments/create-run-attachment /openapi.json post /v1/run-attachments Create a new run attachment. This endpoint is primarily for internal use. File uploads typically happen through the run creation endpoint. # Delete Run Attachment Source: https://docs.cyberdesk.io/api-reference/run-attachments/delete-run-attachment /openapi.json delete /v1/run-attachments/{attachment_id} Delete a run attachment. This will remove both the database record and the file from Supabase storage. # Download Run Attachment Source: https://docs.cyberdesk.io/api-reference/run-attachments/download-run-attachment /openapi.json get /v1/run-attachments/{attachment_id}/download Download a run attachment file. Returns the raw file content as a streaming response. # Get Run Attachment Source: https://docs.cyberdesk.io/api-reference/run-attachments/get-run-attachment /openapi.json get /v1/run-attachments/{attachment_id} Get a specific run attachment by ID. Returns attachment metadata only. Use the download endpoint to get file content. # Get Run Attachment Download Url Source: https://docs.cyberdesk.io/api-reference/run-attachments/get-run-attachment-download-url /openapi.json get /v1/run-attachments/{attachment_id}/download-url Get a signed download URL for a run attachment file. Returns a signed URL that triggers automatic download when accessed. Args: attachment_id: The ID of the attachment to download expires_in: URL expiration time in seconds (10-3600). Default: 300 (5 minutes) # List Run Attachments Source: https://docs.cyberdesk.io/api-reference/run-attachments/list-run-attachments /openapi.json get /v1/run-attachments List all run attachments for the authenticated organization. Supports pagination and filtering by run ID and attachment type. # Update Run Attachment Source: https://docs.cyberdesk.io/api-reference/run-attachments/update-run-attachment /openapi.json put /v1/run-attachments/{attachment_id} Update a run attachment. Currently only supports updating the expiration date. # Run complete webhook Source: https://docs.cyberdesk.io/api-reference/run-complete-webhook /openapi.json webhook run_complete Webhook schema for run completion events. Documentation-only. # Bulk Create Runs Source: https://docs.cyberdesk.io/api-reference/runs/bulk-create-runs /openapi.json post /v1/runs/bulk Create multiple runs with the same configuration. This endpoint creates multiple runs efficiently: - All runs are created in a single database transaction - Temporal workflows are started asynchronously - Returns immediately with created run details Maximum 1000 runs can be created in a single request. # Create Run Source: https://docs.cyberdesk.io/api-reference/runs/create-run /openapi.json post /v1/runs Create a new run. The workflow must exist and belong to the authenticated organization. If machine_id is not provided, an available machine will be automatically selected. The run will be created with SCHEDULING status and a Temporal workflow will be started asynchronously. # Create Run Chain Source: https://docs.cyberdesk.io/api-reference/runs/create-run-chain /openapi.json post /v1/runs/chain Create a multi-step chain that runs on a single reserved session/machine. - Starts a new session unless session_id is provided (then runs on existing session). - Accepts shared_inputs/sensitive/file_inputs and per-step file_inputs. - machine_id > pool_id when starting a new session; both ignored if session_id provided. - Client contract: once runs are persisted, failures in downstream Temporal dispatch do not change the HTTP success envelope; response still contains run_ids and those runs are transitioned to ERROR with failure metadata for deterministic polling. # Delete Run Source: https://docs.cyberdesk.io/api-reference/runs/delete-run /openapi.json delete /v1/runs/{run_id} Delete a run. This will also delete all associated attachments and their files. The run must belong to the authenticated organization. # Generate Run Trajectory Source: https://docs.cyberdesk.io/api-reference/runs/generate-run-trajectory /openapi.json post /v1/runs/{run_id}/generate-trajectory Mark the latest trajectory for a run as generated. This promotes whichever trajectory is currently latest by `updated_at`. If a duplicated trajectory tied to this run is the latest row, this endpoint may return/promote that duplicate. # Get Run Source: https://docs.cyberdesk.io/api-reference/runs/get-run /openapi.json get /v1/runs/{run_id} Get a specific run by ID. The run must belong to the authenticated organization. Returns the run with its associated workflow and machine data. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. # Get Run Image Signed Url Source: https://docs.cyberdesk.io/api-reference/runs/get-run-image-signed-url /openapi.json get /v1/runs/{run_id}/image/signed-url Get a temporary signed URL for a screenshot stored in a run's message history. The requested image URL must be an exact `supabase://run-images/...` reference present in the authenticated organization's run. # Get Run Trajectory Source: https://docs.cyberdesk.io/api-reference/runs/get-run-trajectory /openapi.json get /v1/runs/{run_id}/trajectory Get the latest trajectory associated with a run. Returns generated and non-generated trajectories so run details can expose generation state. Selection is based on newest `updated_at` for the run. If a duplicated trajectory linked to the run is now the latest row, this endpoint will return that duplicate. # List Runs Source: https://docs.cyberdesk.io/api-reference/runs/list-runs /openapi.json get /v1/runs List all runs for the authenticated organization. Supports pagination and filtering by workflow, machine, and status. Returns runs with their associated workflow and machine data. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. Resources are deduplicated across all items in the list. # Retry Run Source: https://docs.cyberdesk.io/api-reference/runs/retry-run /openapi.json post /v1/runs/{run_id}/retry Retry an existing run in-place (same run_id). - Rejects if run is active (scheduling or running). - Always clears previous outputs/history/output attachments. - Replaces input attachments if `file_inputs` are provided. - Optionally overrides inputs, sensitive inputs, session/machine/pools. - Triggers immediate assignment attempt unless the session is busy. # Update Run Source: https://docs.cyberdesk.io/api-reference/runs/update-run /openapi.json patch /v1/runs/{run_id} Update a run's data. Only the fields provided in the request body will be updated. The run must belong to the authenticated organization. # Create Trajectory Source: https://docs.cyberdesk.io/api-reference/trajectories/create-trajectory /openapi.json post /v1/trajectories Create a new trajectory for a workflow. The workflow must exist and belong to the authenticated organization. The trajectory_data must contain at least one item. # Delete Trajectory Source: https://docs.cyberdesk.io/api-reference/trajectories/delete-trajectory /openapi.json delete /v1/trajectories/{trajectory_id} Delete a trajectory. The trajectory must belong to the authenticated organization. # Duplicate Trajectory Source: https://docs.cyberdesk.io/api-reference/trajectories/duplicate-trajectory /openapi.json post /v1/trajectories/{trajectory_id}/duplicate Duplicate a trajectory with fresh copies of all images. Creates a new trajectory with the same data as the source, but with all images copied to new paths in storage (copy-on-write semantics). The new trajectory starts unapproved and gets a name like "Original Name (Copy)". # Get Latest Trajectory For Workflow Source: https://docs.cyberdesk.io/api-reference/trajectories/get-latest-trajectory-for-workflow /openapi.json get /v1/workflows/{workflow_id}/latest-trajectory Get the latest trajectory for a specific workflow. Returns the most recently updated trajectory for the workflow. The workflow must belong to the authenticated organization. # Get Trajectory Source: https://docs.cyberdesk.io/api-reference/trajectories/get-trajectory /openapi.json get /v1/trajectories/{trajectory_id} Get a specific trajectory by ID. The trajectory must belong to the authenticated organization. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. Example: `?include=workflow` to include the associated workflow data. # List Trajectories Source: https://docs.cyberdesk.io/api-reference/trajectories/list-trajectories /openapi.json get /v1/trajectories List all trajectories for the authenticated organization. Supports pagination and filtering by workflow, run, approval, and generated status. Only approved trajectories are used during workflow execution. Returns trajectories with their associated workflow data. Use the `include` parameter to fetch related resources in the response. Related resources are returned in the `included` array following the JSON:API pattern. # Update Trajectory Source: https://docs.cyberdesk.io/api-reference/trajectories/update-trajectory /openapi.json patch /v1/trajectories/{trajectory_id} Update a trajectory's data. Only the fields provided in the request body will be updated. The trajectory must belong to the authenticated organization. # Get Usage Aggregate Source: https://docs.cyberdesk.io/api-reference/usage/get-usage-aggregate /openapi.json get /v1/usage/aggregate Aggregate usage (agentic and cached steps) for a given date range. Two modes are supported: - **simulated** (default): Uses total_agentic_steps and total_cached_steps from usage_metadata, but excludes runs where billing_outcome is 'infra_failure' (these would have been free). Use this for customers not yet on Stripe billing. - **billed**: Uses total_agentic_steps_billed and total_cached_steps_billed. Use this for customers on active Stripe billing. # Create Tag Group Source: https://docs.cyberdesk.io/api-reference/workflow-tag-groups/create-tag-group /openapi.json post /v1/workflow-tag-groups Create a new workflow tag group. Tag groups enable mutual exclusivity - when a workflow has a tag from a group, adding another tag from the same group will automatically remove the first one. # Delete Tag Group Source: https://docs.cyberdesk.io/api-reference/workflow-tag-groups/delete-tag-group /openapi.json delete /v1/workflow-tag-groups/{group_id} Delete a workflow tag group. Tags that belong to this group will become ungrouped (group_id set to null). The tags themselves are not deleted. # Get Tag Group Source: https://docs.cyberdesk.io/api-reference/workflow-tag-groups/get-tag-group /openapi.json get /v1/workflow-tag-groups/{group_id} Get a specific workflow tag group by ID. # List Tag Groups Source: https://docs.cyberdesk.io/api-reference/workflow-tag-groups/list-tag-groups /openapi.json get /v1/workflow-tag-groups List all workflow tag groups for the organization. Groups are returned ordered by their `order` field for consistent display. Tags within a group are mutually exclusive - only one can be assigned to a workflow. # Reorder Tag Groups Source: https://docs.cyberdesk.io/api-reference/workflow-tag-groups/reorder-tag-groups /openapi.json put /v1/workflow-tag-groups/reorder Reorder workflow tag groups. Provide a list of group IDs in the desired order. The order field of each group will be updated to match its position in the list. # Update Tag Group Source: https://docs.cyberdesk.io/api-reference/workflow-tag-groups/update-tag-group /openapi.json patch /v1/workflow-tag-groups/{group_id} Update a workflow tag group. # Archive Tag Source: https://docs.cyberdesk.io/api-reference/workflow-tags/archive-tag /openapi.json post /v1/workflow-tags/{tag_id}/archive Archive a workflow tag. Archived tags cannot be assigned to new workflows but remain on existing workflows. Archived tags are hidden from the tag list by default. # Create Tag Source: https://docs.cyberdesk.io/api-reference/workflow-tags/create-tag /openapi.json post /v1/workflow-tags Create a new workflow tag. Tags can optionally belong to a group for mutual exclusivity. # Delete Tag Source: https://docs.cyberdesk.io/api-reference/workflow-tags/delete-tag /openapi.json delete /v1/workflow-tags/{tag_id} Delete a workflow tag. This is a hard delete - the tag is permanently removed and will be unassigned from all workflows. Consider using archive instead if you want to preserve existing assignments. # Get Tag Source: https://docs.cyberdesk.io/api-reference/workflow-tags/get-tag /openapi.json get /v1/workflow-tags/{tag_id} Get a specific workflow tag by ID. # List Tags Source: https://docs.cyberdesk.io/api-reference/workflow-tags/list-tags /openapi.json get /v1/workflow-tags List all workflow tags for the organization. Tags are returned ordered by their group (ungrouped first), then by order within group. Each tag includes its workflow_count indicating how many workflows use it. # Reorder Tags Source: https://docs.cyberdesk.io/api-reference/workflow-tags/reorder-tags /openapi.json put /v1/workflow-tags/reorder Reorder workflow tags. Provide a list of tag IDs in the desired order. The order field of each tag will be updated to match its position in the list. Note: Tags should be reordered within their respective groups. # Unarchive Tag Source: https://docs.cyberdesk.io/api-reference/workflow-tags/unarchive-tag /openapi.json post /v1/workflow-tags/{tag_id}/unarchive Unarchive a workflow tag. Restores an archived tag so it can be assigned to workflows again. # Update Tag Source: https://docs.cyberdesk.io/api-reference/workflow-tags/update-tag /openapi.json patch /v1/workflow-tags/{tag_id} Update a workflow tag. # Add tags to a workflow Source: https://docs.cyberdesk.io/api-reference/workflows/add-tags-to-a-workflow /openapi.json post /v1/workflows/{workflow_id}/tags Add one or more tags to a workflow. For tags that belong to a group (mutual exclusivity), adding a new tag from that group will automatically remove any existing tag from the same group. # Bulk add tags to multiple workflows Source: https://docs.cyberdesk.io/api-reference/workflows/bulk-add-tags-to-multiple-workflows /openapi.json post /v1/workflows/bulk/tags Add tags to multiple workflows at once. For each workflow, mutual exclusivity is enforced: if a tag belongs to a group and the workflow already has a tag from that group, the existing tag is replaced. # Create Workflow Source: https://docs.cyberdesk.io/api-reference/workflows/create-workflow /openapi.json post /v1/workflows Create a new workflow. The workflow will be associated with the authenticated organization. # Delete a workflow prompt image Source: https://docs.cyberdesk.io/api-reference/workflows/delete-a-workflow-prompt-image /openapi.json delete /v1/workflows/prompt-image Delete a workflow prompt image from storage. # Delete Workflow Source: https://docs.cyberdesk.io/api-reference/workflows/delete-workflow /openapi.json delete /v1/workflows/{workflow_id} Delete a workflow. This will also delete all associated runs and trajectories. The workflow must belong to the authenticated organization. # Duplicate Workflow Source: https://docs.cyberdesk.io/api-reference/workflows/duplicate-workflow /openapi.json post /v1/workflows/{workflow_id}/duplicate Duplicate a workflow, including prompt images and optional generated trajectories. Copies everything except version history, and prefixes the name with "(Copy)". # Get signed URL for a prompt image Source: https://docs.cyberdesk.io/api-reference/workflows/get-signed-url-for-a-prompt-image /openapi.json get /v1/workflows/prompt-image/signed-url Get a fresh signed URL for an existing workflow prompt image. # Get tags for a workflow Source: https://docs.cyberdesk.io/api-reference/workflows/get-tags-for-a-workflow /openapi.json get /v1/workflows/{workflow_id}/tags Get all tags assigned to a workflow. # Get Workflow Source: https://docs.cyberdesk.io/api-reference/workflows/get-workflow /openapi.json get /v1/workflows/{workflow_id} Get a specific workflow by ID. The workflow must belong to the authenticated organization. # Get Workflow Versions Source: https://docs.cyberdesk.io/api-reference/workflows/get-workflow-versions /openapi.json get /v1/workflows/{workflow_id}/versions Get the version history of a workflow. Returns a list of previous versions with their prompts and timestamps. The workflow must belong to the authenticated organization. # List workflow prompt images Source: https://docs.cyberdesk.io/api-reference/workflows/list-workflow-prompt-images /openapi.json get /v1/workflows/prompt-images List all prompt images uploaded for the organization. # List Workflows Source: https://docs.cyberdesk.io/api-reference/workflows/list-workflows /openapi.json get /v1/workflows List all workflows for the authenticated organization. Supports pagination and returns workflows ordered by updated_at descending. Use `tag_ids` to filter workflows that have ALL specified tags (AND logic). Use `include=tags,post_run_checks` to include related workflow resources. Legacy `include_tags=true` remains supported, but `include` wins when both are provided. # Merge Workflow Source: https://docs.cyberdesk.io/api-reference/workflows/merge-workflow /openapi.json post /v1/workflows/{workflow_id}/merge Merge the current workflow into another workflow. Updates the target workflow with all data from the source workflow (except version history), optionally copies trajectories, and deletes the source workflow. # Remove a tag from a workflow Source: https://docs.cyberdesk.io/api-reference/workflows/remove-a-tag-from-a-workflow /openapi.json delete /v1/workflows/{workflow_id}/tags/{tag_id} Remove a tag from a workflow. # Update Workflow Source: https://docs.cyberdesk.io/api-reference/workflows/update-workflow /openapi.json patch /v1/workflows/{workflow_id} Update a workflow's prompts. The current version will be saved to the version history. Only the fields provided in the request body will be updated. The workflow must belong to the authenticated organization. # Upload a workflow prompt image Source: https://docs.cyberdesk.io/api-reference/workflows/upload-a-workflow-prompt-image /openapi.json post /v1/workflows/prompt-image Upload an image to use in workflow prompts. The returned `supabase_url` can be embedded directly in workflow prompt HTML: ```html Description ``` When the workflow runs, Cyberdesk automatically resolves these URLs to display the image to the AI agent. Supported formats: PNG, JPEG, GIF, WebP. Maximum size: 10MB. # Async Extraction Patterns Source: https://docs.cyberdesk.io/concepts/async-extraction-patterns Understanding synchronous, batch-scoped, and run-scoped async extraction modes ## Overview Cyberdesk provides flexible async extraction modes that let you optimize workflow performance based on when and how you need extracted data. Understanding these patterns is key to building fast, efficient workflows. Async extraction works seamlessly with [Cyberdesk's trajectory caching system](/concepts/trajectories). During trajectory replay, extract prompts re-execute to capture fresh data, so you get both the speed benefits of caching and the flexibility of dynamic data extraction. ## The Three Processing Modes ### Synchronous (Default) **When**: `process_async` is omitted **Behavior**: Extraction blocks until complete **Processing Time**: 2-5 seconds per extraction **Use When**: * You need the result immediately for the next decision * Extracting a single value * The extraction determines workflow branching * Simple workflows with \< 5 total extractions **Example**: ```text theme={null} "Navigate to order details. Take a screenshot with extract_prompt='Extract the order status as one word: Pending, Processing, or Shipped' to determine if the order needs manual intervention." ``` **Timing Diagram**: ``` Agent Step 1 → Extract (3s) → Agent Step 2 → Extract (3s) → Agent Step 3 Total: 6 seconds of extraction time ``` ### Batch-Scoped Async **When**: `process_async="batch"` **Behavior**: When multiple screenshot extractions run in the same batched tool phase, they run in parallel and complete before the next agent step. If Cyberdesk executes the screenshot as a standalone tool call instead, it falls back to synchronous extraction. **Processing Time**: \~3 seconds for entire batch (no matter how many extractions) **Use When**: * Scrolling through lists or paginated content * Extracting from multiple sequential views * Extractions don't depend on each other * Results should be ready for next agent decision * Want to store runtime variables from extractions before next agent turn **Runtime Values**: Like any `extract_prompt` call, the extraction agent can call `upsert_runtime_values` when your prompt explicitly tells it to save or store values. In batch mode, those values become available before the next agent step once the batch finishes. **Example**: ```text theme={null} "Scroll through the product catalog and extract all data: - Take screenshot with extract_prompt='Extract visible products as JSON' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract visible products as JSON' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract visible products as JSON' and process_async='batch' All extractions complete in parallel before next agent turn." ``` **Timing Diagram**: ``` Agent Step: [Screenshot + Extract] → [Scroll] → [Screenshot + Extract] → [Scroll] → [Screenshot + Extract] └─────────── All 3 extractions run in parallel (3s total) ──────────┘ ↓ Agent Step 2 Total: ~3 seconds for all extractions combined ``` **Performance Benefit**: 3-5x faster than synchronous when extracting from multiple views in one batched phase ### Run-Scoped Async **When**: `process_async="run"` **Behavior**: Extraction runs completely in background for entire workflow, only awaited at final output generation **Requirement**: The workflow must have an `output_schema`; otherwise Cyberdesk returns an error and asks you to use synchronous or batch mode instead **Processing Time**: Non-blocking, completes while workflow continues **Use When**: * Large data extractions not needed for navigation * Extraction is only for final output * You want maximum parallelism * Need to set runtime variables from extraction that won't be used until later **Runtime Values**: Like any `extract_prompt` call, the extraction agent can call `upsert_runtime_values` when your prompt explicitly tells it to save or store values. In run scope, those values become available once the background extraction finishes. **Example**: ```text theme={null} "Navigate to analytics dashboard. Take screenshot with extract_prompt='Extract complete analytics data: all metrics, charts, KPIs, trends as detailed JSON' and process_async='run' Continue with report generation workflow. The analytics extraction will complete in the background and be included automatically in final output." ``` **Timing Diagram**: ``` Agent Step 1 → [Start Extract (non-blocking)] → Agent Step 2 → Agent Step 3 → Agent Step 4 ↓ (running in background) ↓ ↓ [Extraction completes while workflow continues] ↓ └─────────────────────────────────────→ Final Output Generation (waits for completion) Total: 0 seconds of blocking time, extraction happens in parallel with workflow ``` **Performance Benefit**: Maximum parallelism, zero blocking time during workflow execution ## Comparing the Modes | Aspect | Synchronous | Batch-Scoped | Run-Scoped | | --------------------- | ------------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------- | | **Blocking** | ✅ Blocks each time | ✅ Blocks at end of batch | ❌ Non-blocking | | **Parallelism** | ❌ Sequential | ✅ Within batch | ✅ Across entire run | | **Best For** | Single extraction, decisions | Lists, pagination | Large output data | | **Result Available** | Immediately | Before next step | At final output | | **Runtime Variables** | ✅ Yes, if the prompt explicitly asks to save/store them | ✅ Yes, if the prompt explicitly asks to save/store them | ✅ Yes, if the prompt explicitly asks to save/store them | | **Performance** | Slowest (N × 3s) | Fast (3s per batch) | Fastest (0s blocking) | ## Performance Examples ### Scenario: Extract from 10 Pages of Data **Synchronous**: ``` Page 1: Extract (3s) → Navigate Page 2: Extract (3s) → Navigate ... Page 10: Extract (3s) Total: 30 seconds of extraction time ``` **Batch-Scoped**: ``` Batch 1: [Page 1 Extract, Navigate, Page 2 Extract, Navigate, Page 3 Extract] All 3 extractions in parallel: 3s Batch 2: [Page 4 Extract, Navigate, Page 5 Extract, Navigate, Page 6 Extract] All 3 extractions in parallel: 3s Batch 3: [Page 7 Extract, Navigate, Page 8 Extract, Navigate, Page 9 Extract] All 3 extractions in parallel: 3s Batch 4: [Page 10 Extract] 1 extraction: 3s Total: 12 seconds of extraction time (2.5x faster) ``` **Run-Scoped** (if extraction not needed for navigation): ``` Start: Launch extraction for all pages → Continue workflow All 10 extractions run in background while workflow completes Total: 0 seconds of blocking time (10x faster) ``` ## Advanced Pattern: Hybrid Extraction Combine multiple modes for optimal performance: ```text theme={null} "Navigate to customer portal. Step 1 - Quick Decision (Synchronous): Take screenshot with extract_prompt='Extract customer account status: Active, Suspended, or Closed' to determine workflow path. Step 2 - List Processing (Batch-Scoped): If status is Active, extract recent transactions: - Take screenshot with extract_prompt='Extract visible transactions as JSON' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract visible transactions as JSON' and process_async='batch' - Repeat for all pages Step 3 - Detailed Analytics (Run-Scoped): Take screenshot with extract_prompt='Extract complete account analytics: spending patterns, category breakdown, payment history' and process_async='run' Continue with report generation. The analytics extraction completes in background." ``` **Result**: * Fast decision making (synchronous where needed) * Efficient list processing (batch-scoped parallelism) * Zero blocking for large data (run-scoped for final output) ## Extraction Modes with Runtime Variables All `extract_prompt` modes use the extraction agent, and that agent can call `upsert_runtime_values` when your prompt explicitly tells it to save or store runtime values. The main difference is timing: **Synchronous**: Variables are available immediately when the extraction returns **Batch-Scoped**: Variables are available before the next agent step once the batch finishes **Run-Scoped**: Variables are available when the background extraction completes ### The Extraction Agent Every `extract_prompt` call uses the extraction agent. Async modes (`batch` and `run`) add concurrency on top of the same capabilities: 1. **Call upsert\_runtime\_values** to store specific fields 2. **Provide final observations** as text 3. **Do both**: Store values AND provide observations ### System Prompt (All Extraction Modes) When an extraction runs, the extraction agent receives guidance like: ``` You are an extraction assistant. You have two capabilities: 1. Call upsert_runtime_values to store specific extracted values that should be available throughout the workflow as {{key_name}} placeholders. 2. Provide final observations as text describing what you see on screen. You can do BOTH: store specific values AND provide observations, or just do one or the other. Your final text message will be recorded as the extraction result. ``` ### Example: Synchronous with Runtime Variables (Available Immediately) ```text theme={null} "Open the customer profile. Take screenshot with extract_prompt='Extract customer_id and membership_tier and store them as runtime variables using upsert_runtime_values. Then provide a short summary of the visible account status.' Use {{customer_id}} immediately in the next step." ``` **What Happens**: 1. Screenshot taken 2. Extraction agent analyzes screenshot 3. Calls `upsert_runtime_values({customer_id: "C-1024", membership_tier: "Gold"})` 4. Provides observation about the visible account status 5. **Variables are available immediately** when the extraction returns 6. Agent proceeds with `{{customer_id}}` and `{{membership_tier}}` available ### Example: Batch-Scoped with Runtime Variables (Available Before Next Step) ```text theme={null} "Scroll through order list: - Take screenshot with extract_prompt='Extract order_id as runtime variable using upsert_runtime_values, then describe order status and customer name' and process_async='batch' - Use {{order_id}} to determine if this order needs special handling - If yes, click on order - Repeat for next order" ``` **What Happens**: 1. Screenshot taken 2. Extraction agent analyzes screenshot 3. Calls `upsert_runtime_values({order_id: "ORD-123"})` 4. Provides observation about status and customer 5. All batch extractions complete in parallel 6. **Variables available before next agent step** - can be used immediately 7. Agent proceeds with `{{order_id}}` available ### Example: Run-Scoped with Runtime Variables (Available When Extraction Completes) ```text theme={null} "Open invoice {invoice_number}. Take screenshot with extract_prompt='Extract the invoice_date and total_amount and store them as runtime variables using upsert_runtime_values. Then provide a detailed description of all line items, tax breakdown, and payment terms.' and process_async='run' Continue with payment workflow. The {{invoice_date}} and {{total_amount}} variables will be available once extraction completes in background, and the full invoice details will be in the final output." ``` **What Happens**: 1. Extraction starts in background 2. Workflow continues with other tasks 3. Extraction agent analyzes screenshot (in background) 4. Calls `upsert_runtime_values({invoice_date: "2024-01-15", total_amount: 1250.00})` 5. **Variables become available** once extraction completes 6. Agent then provides detailed observation about line items, taxes, etc. 7. Both the runtime variables and observation text are included in final output ### Example: Pure Observation (No Runtime Variables) ```text theme={null} "Take screenshot with extract_prompt='Describe all customer information visible on screen including contact details, order history, and preferences. Format as detailed JSON.' and process_async='run' This large extraction runs in background and will be in final output." ``` ### Example: Multiple Runtime Variables ```text theme={null} "Take screenshot with extract_prompt='Extract customer_id, order_total, and expected_delivery_date as runtime variables using upsert_runtime_values. Then provide a summary of order contents, shipping address, and special instructions.' and process_async='run' Use {{customer_id}} and {{order_total}} in subsequent workflow steps once available." ``` ### Array and Object Operators When accumulating data across multiple extractions (e.g., scrolling through a list), use MongoDB-style operators to append to arrays instead of replacing values: | Operator | Description | Example | | ---------- | -------------------------------- | ---------------------------------------- | | `$append` | Append item to array | `{"items": {"$append": "new_item"}}` | | `$prepend` | Prepend item to array | `{"items": {"$prepend": "first"}}` | | `$concat` | Concatenate arrays | `{"items": {"$concat": ["a", "b"]}}` | | `$merge` | Shallow merge objects | `{"config": {"$merge": {"key": "val"}}}` | | `$remove` | Remove first occurrence by value | `{"tags": {"$remove": "old"}}` | | `$pop` | Remove last element | `{"stack": {"$pop": true}}` | #### Example: Accumulating Extracted Items Across Pages ```text theme={null} "Scroll through the products list and extract all items: Page 1: - Take screenshot with extract_prompt='Extract all visible products as JSON array. For each product, call upsert_runtime_values with {\"products\": {\"$append\": {...product data...}}} to add it to the running list.' and process_async='batch' Page 2: - Scroll down - Take screenshot with extract_prompt='Extract all visible products as JSON array. For each product, call upsert_runtime_values with {\"products\": {\"$append\": {...product data...}}} to add it to the running list.' and process_async='batch' After all pages, {{products}} contains the complete list of all extracted products." ``` The `$append` operator creates the array if it doesn't exist, so you don't need to initialize `{{products}}` before the first extraction. ## Choosing the Right Mode Use this decision tree: ``` Do you need the extraction result to make the next decision? ├─ YES → Use Synchronous │ Example: "Extract status to determine next action" │ └─ NO → Is this a list/multiple views? ├─ YES → Use Batch-Scoped Async │ Example: "Scroll and extract from each page" │ └─ NO → Is this only for final output? ├─ YES → Use Run-Scoped Async │ Example: "Extract analytics for report" │ └─ DEPENDS → Can the value arrive later while the workflow keeps going? ├─ YES → Use Run-Scoped Async (requires `output_schema`) │ Example: "Extract invoice_number for later use" │ └─ NO → Use Synchronous if < 3 extractions Use Batch-Scoped if >= 3 independent extractions in one batched phase ``` ## Real-World Patterns ### Healthcare: Patient Record Processing ```text theme={null} "Navigate to patient {patient_mrn}. Quick Check (Synchronous): Take screenshot with extract_prompt='Extract patient age as number' to determine if pediatric workflow is needed. Medical History (Batch-Scoped): Navigate through each section of medical history: - Take screenshot with extract_prompt='Extract diagnosis history' and process_async='batch' - Go to medications tab - Take screenshot with extract_prompt='Extract current medications' and process_async='batch' - Go to allergies tab - Take screenshot with extract_prompt='Extract allergies' and process_async='batch' Comprehensive Record (Run-Scoped): Take screenshot of complete patient chart with extract_prompt='Extract full patient record including all demographics, vitals, lab results, imaging reports, and visit notes. Store patient_mrn as runtime variable.' and process_async='run' Continue with documentation workflow using {{patient_mrn}} for file naming." ``` ### E-Commerce: Inventory Extraction ```text theme={null} "Log into inventory system. Category Check (Synchronous): Take screenshot with extract_prompt='Count visible categories' to determine navigation depth. Per-Category Extraction (Batch-Scoped): For each category: - Navigate to category - Take screenshot with extract_prompt='Extract all products as JSON' and process_async='batch' - Repeat Full Catalog Analytics (Run-Scoped): Take screenshot with extract_prompt='Extract complete inventory analytics: stock levels, trends, low-stock alerts, reorder recommendations. Store total_products and low_stock_count as runtime variables.' and process_async='run' Generate inventory report using {{total_products}} and {{low_stock_count}} in report header." ``` ### Finance: Transaction Processing ```text theme={null} "Open account {account_number}. Balance Check (Synchronous): Take screenshot with extract_prompt='Extract current balance as number' to verify sufficient funds. Recent Transactions (Batch-Scoped): Scroll through transaction history: - Take screenshot with extract_prompt='Extract visible transactions' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract visible transactions' and process_async='batch' - Repeat Account Analysis (Run-Scoped): Take screenshot with extract_prompt='Extract complete account analysis: spending categories, monthly trends, unusual patterns. Store account_type and risk_level as runtime variables.' and process_async='run' Continue with report generation using {{account_type}} and {{risk_level}} for classification." ``` ## Best Practices ### 1. Start with Synchronous, Optimize Later Begin with simple synchronous extraction, then optimize bottlenecks: ```text theme={null} # Initial (works, but slow) "Extract field A (sync) → Extract field B (sync) → Extract field C (sync)" # Optimized (3x faster) "Extract all three fields with process_async='batch' in one batch" ``` ### 2. Use Batch-Scoped for Lists Any time you're iterating (scrolling, clicking next, navigating pages), use batch-scoped: ```text theme={null} "For each item in list: - Extract data with process_async='batch' - Navigate to next All extractions complete in parallel" ``` ### 3. Use Run-Scoped for Final Output Only If the extracted data doesn't influence navigation or decisions, make it run-scoped: ```text theme={null} "Take screenshot with extract_prompt='Extract complete report data' and process_async='run' Continue workflow - extraction completes in background" ``` ### 4. Combine with Other Extraction Methods Use [copy\_to\_clipboard](/workflow-prompting/copy-to-clipboard) for fast copyable text, and [extract\_prompt](/workflow-prompting/extract-prompt) for vision-based extraction: ```text theme={null} "Extract order data: - Triple-click Order ID, use copy_to_clipboard with key 'order_id' (instant) - Take screenshot with extract_prompt='Extract order items' and process_async='batch' (parallel) - Take screenshot with extract_prompt='Extract shipping analytics' and process_async='run' (background) Optimal performance using all available methods" ``` ### 5. Set Runtime Variables from Run-Scoped Extractions When you need specific values mid-workflow but also want large extractions: ```text theme={null} "Take screenshot with extract_prompt='Extract customer_id and order_total as runtime variables. Then extract complete order details, history, and preferences as detailed JSON.' and process_async='run' The {{customer_id}} and {{order_total}} become available once extraction completes, while full details are in final output." ``` ## Performance Metrics Based on typical workflow patterns: | Scenario | Synchronous | Batch-Scoped | Run-Scoped | Speedup | | --------------------------- | ----------- | ------------ | ---------- | ------- | | 1 extraction | 3s | 3s | 0s\* | 1x | | 5 extractions (sequential) | 15s | 3-6s | 0s\* | 2.5-5x | | 10 extractions (sequential) | 30s | 9-12s | 0s\* | 2.5-10x | | 20 extractions (sequential) | 60s | 18-24s | 0s\* | 2.5-10x | \*Run-scoped shows 0s blocking time but still processes in background; workflow continues unblocked ## Common Patterns Summary | Pattern | Mode | Example | | ----------------------------- | ------------ | ----------------------------------------------------- | | **Decision Making** | Synchronous | "Extract status to determine next step" | | **List Processing** | Batch-Scoped | "Scroll and extract from each page" | | **Final Output Data** | Run-Scoped | "Extract analytics for report" | | **Background Runtime Values** | Run-Scoped | "Extract and store ID for later use" | | **Mixed Requirements** | Hybrid | Sync for decisions + Batch for lists + Run for output | ## Migration Guide ### From Synchronous to Batch-Scoped **Before**: ```text theme={null} "Extract from page 1 Navigate Extract from page 2 Navigate Extract from page 3" ``` **After**: ```text theme={null} "Extract from page 1 with process_async='batch' Navigate Extract from page 2 with process_async='batch' Navigate Extract from page 3 with process_async='batch'" ``` **Benefit**: 3x faster with no other changes ### From Batch-Scoped to Run-Scoped **Before**: ```text theme={null} "Extract complete data with process_async='batch' Continue workflow (must wait for extraction)" ``` **After**: ```text theme={null} "Extract complete data with process_async='run' Continue workflow (extraction in background)" ``` **Benefit**: Zero blocking time, maximum parallelism **Caution**: Only use if extraction result not needed for navigation ## Summary * **Synchronous**: Simple, reliable, blocks until complete. Use for decisions or values you need immediately. * **Batch-Scoped**: Parallel within batch, 3-5x faster for lists. Use for iteration. * **Run-Scoped**: Maximum parallelism, zero blocking. Use for output-only data. Choose based on when you need the data, not just "async is faster" - the right pattern depends on your workflow structure. For more information, see: * [Extract Prompt](/workflow-prompting/extract-prompt) - Detailed extraction syntax and examples * [Trajectories 101](/concepts/trajectories) - How caching amplifies these performance benefits # Clear Values Source: https://docs.cyberdesk.io/concepts/clear-values Use the __CLEAR__ sentinel value to intentionally clear an existing field before continuing Use `__CLEAR__` when you want Cyberdesk to actively blank out a field instead of leaving its current value untouched. `__CLEAR__` is explicit. Cyberdesk preserves it as the input value during validation, then turns it into a clear action only when a `type` step runs on the currently focused field. ## Simple Use Case **Problem**: Some fields already contain text, and leaving an input blank is not enough when you want the final value to be empty. **Solution**: Use `__CLEAR__` to clear the currently focused field with `ctrl+a backspace`. ## Example ```python theme={null} # User wants to remove the pre-filled middle name input_values = {"middle_name": "__CLEAR__"} ``` If a later workflow step types `{middle_name}` into the focused field, Cyberdesk sends the clear-field shortcut instead of typing the literal text `__CLEAR__`. ## How It Differs From `__EMPTY__` * `__CLEAR__`: actively clears the current field * `__EMPTY__`: skips typing and leaves the current field untouched * omitted / blank / `null` optional values: do not clear the field unless you explicitly pass `__CLEAR__` If you want skip behavior instead, see [Empty Values](/concepts/empty-values). ## When to Use * **Pre-filled forms** where the field already contains a value * **Resetting optional fields** that should end blank * **Overriding machine defaults or existing UI state** with an intentionally empty result ## Best Practice Use `__CLEAR__` only when the workflow step is actually typing into the correct focused field and you want the final visible value to be empty. Cyberdesk does not infer it from blank or omitted values; you must pass `__CLEAR__` explicitly. For plain skip behavior, use [Empty Values](/concepts/empty-values) instead. # Complex Drag Actions Source: https://docs.cyberdesk.io/concepts/complex-drag-actions Perform drag operations with hold durations using left_mouse_down and left_mouse_up Some applications require more than a simple drag-and-drop. They need you to: * **Hold at the destination** before releasing (e.g., drag-to-reorder with hover delay) * **Pause mid-drag** to trigger hover states or tooltips * **Perform multi-step drag sequences** with precise timing The `left_mouse_down` and `left_mouse_up` actions give you granular control over the mouse button state, enabling these complex drag patterns. ## Available Actions Press and hold the left mouse button without releasing. Optionally move to coordinates first. Release a previously pressed left mouse button. Optionally move to coordinates first. ## Basic Usage Pattern To perform a drag with a hold duration at the destination: Use `left_mouse_down` with the starting coordinates to press and hold the mouse button. Use `mouse_move` to drag to the target position while the button is held. Use `wait` to hold at the destination for the required duration. If you omit `duration`, production currently defaults to `1` second. Waits must be non-negative and can be at most `1800` seconds. Use `left_mouse_up` to release the mouse button. ## Example: Drag with 2-Second Hold Here's an example workflow prompt that drags an element and holds for 2 seconds before releasing: ``` 1. Click on the item I want to drag 2. Use left_mouse_down at the item's position to start the drag 3. Use mouse_move to move to the drop target 4. Use wait for 2 seconds to hold at the target 5. Use left_mouse_up to release and complete the drop ``` ## When to Use | Scenario | Solution | | ----------------------------------- | --------------------------------------------------------------- | | Simple drag-and-drop | Use `left_click_drag` with optional `duration` | | Drag with hold at destination | Use `left_mouse_down` → `mouse_move` → `wait` → `left_mouse_up` | | Drag with multiple stops | Chain multiple `mouse_move` and `wait` actions between down/up | | Trigger hover states while dragging | Add `wait` actions at intermediate positions | ## Comparison with left\_click\_drag The `left_click_drag` action is simpler and handles most drag operations: ``` left_click_drag(start_coordinate=(100, 100), coordinate=(300, 300), duration=0.5) ``` In actual tool inputs, `start_coordinate` and `coordinate` are passed as `[x, y]` arrays. This performs the entire drag in one action. The `duration` parameter controls **how long the drag takes from mouse down to mouse up**, not how long to hold at the destination. For scenarios requiring a **hold at the destination**, use the manual approach with `left_mouse_down` and `left_mouse_up`. The drag-related actions on this page, including `left_mouse_down`, `left_mouse_up`, `mouse_move`, `wait`, and `left_click_drag`, are available in both the main agent and focused agent contexts. ## Tips * Always pair `left_mouse_down` with `left_mouse_up` to avoid leaving the mouse in a pressed state * Use `wait` with specific durations (e.g., 1-3 seconds) for hover triggers * Take screenshots between actions if you need to verify the drag state * If a drag fails, the agent can use `left_mouse_up` to reset the mouse state before retrying # Custom Cache Detection Prompt Source: https://docs.cyberdesk.io/concepts/custom-cache-detection Override cache detection logic with your own validation rules Custom Cache Detection Prompts let you override the default cache-detection context for a specific recorded step. When set, Cyberdesk still compares the cached state (Desktop #1) with the current state (Desktop #2), but your instructions become the rules the model should use for whether the recorded action would still replay successfully. For an introduction to trajectories and how they work, see [Trajectories 101](/concepts/trajectories). This page focuses specifically on custom cache detection prompts that give you full control over trajectory validation. ## The Problem During trajectory replay, the system compares current screen states against cached screenshots to determine if it can safely reuse recorded actions. However, the default cache detection logic might not always know: * Which UI elements are critical vs. cosmetic for a specific step * What minor differences are acceptable (e.g., timestamps, dynamic content) * Special conditions that should invalidate the cache * Context about why a particular screen state was captured Without this context, cache detection may: * Reject valid matches due to irrelevant differences (false negatives) * Accept invalid matches by missing critical changes (false positives) * Require human review to understand failures ## The Solution Custom Cache Detection Prompts give you full control over the validation logic. When you set a custom prompt: * **Default-context override**: Your prompt replaces the standard workflow and step context Cyberdesk normally uses for cache detection * **Step-specific**: Each eligible recorded trajectory step can have its own custom prompt * **Human-written**: Provide exact validation criteria that automated systems can't infer * **Direct Control**: You define exactly what constitutes a cache hit vs miss for that recorded action When a custom prompt is set, it replaces the default workflow and step context that Cyberdesk normally provides for cache detection. The model still receives the two screenshots and, when applicable, a note about the shared red-circle annotation. You must explicitly define what makes a cache hit vs cache miss, because those instructions are the only rules the model should use for the decision. ## How It Works ### During Cache Detection When comparing screen states with a custom prompt: 1. **Capture**: System takes a screenshot of the current state (Desktop #2) 2. **Load Cache**: Retrieves the cached screenshot from the trajectory (Desktop #1) 3. **Prompt Framing**: Cyberdesk wraps your instructions in a short action-replay prompt instead of the normal default context 4. **Decision**: The system evaluates and returns cache hit or cache miss The cache detection agent receives a prompt shaped like this: ``` Your task is to determine whether a cached action (originally successfully executed on desktop #1's state) would successfully replay on desktop #2's current state. Desktop #1 is verified to be a desired state to execute the cached action. The user has provided the following custom cache detection instructions, that you should strictly adhere to. They are the only rules that apply to this decision. [Your custom prompt here] Note: Both screenshots have a red circle annotation at the same coordinate to help you identify if the UI has shifted. ``` The red-circle note is only included when the recorded step has coordinate annotations. ### In the Trajectory Editor You can add or edit custom prompts in the trajectory viewer for recorded steps that have a pre-check snapshot: 1. Navigate to a trajectory in the Workflows section 2. Expand a recorded step to view details 3. Find the `Custom Cache Detection Prompt Override` field in the pre-check snapshot area 4. Add your prompt with explicit hit/miss criteria 5. Save changes to update the trajectory Steps added manually in the UI cannot use custom cache detection because they do not have a reference screenshot, and loop-control steps such as `end_loop_iteration` do not run cache detection. You can also open a cache comparison in run history, click `Improve Cache Detection`, and save the same override back to the underlying trajectory step. ## Example Use Cases Since your custom prompt replaces the default context for that step, you must be explicit about what defines a cache hit vs miss. ### Validating Critical Elements ``` Cache hit if the "Submit Order" button is visible and enabled. Cache miss if the button is missing, disabled, or in a different position. Ignore the order number displayed, as it will differ between runs. ``` ### Checking Layout Structure ``` Cache hit if the product list shows exactly 3 items in the same layout. Cache miss if the item count differs or the layout has changed. Item names will vary based on parameters - ignore text content, focus on structure. ``` ### Focus Area Validation ``` Only validate the left sidebar navigation. Cache hit if the sidebar shows the same menu items in the same positions. Cache miss if sidebar items are different or rearranged. Ignore the main content area entirely - it displays real-time data. ``` ### Conditional States ``` Cache hit if EITHER: - A modal is open showing "Processing..." status, OR - The modal is closed and a results table is visible Cache miss if neither condition is met. ``` ### Form Validation ``` Cache hit if the form has 4 input fields in vertical layout with labels: Name, Email, Phone, Address. Cache miss if any field is missing or the layout differs. Ignore the actual values in the fields - they will differ based on parameters. ``` ## Best Practices ### Always Define Both Outcomes Since you're replacing the default cache-detection context, explicitly state both when it's a cache hit and when it's a cache miss. Good: "Cache hit if the dialog is visible. Cache miss if it's not visible or obscured." Less Good: "The dialog should be visible" ### Be Specific About What to Check Good: "Cache hit if three buttons are arranged horizontally at the bottom of the form." Less Good: "Check the buttons are correct" ### Explicitly State What to Ignore Good: "Ignore timestamps, 'Last Updated' labels, and dynamic content. Focus only on the form structure." Less Good: Assuming the system knows what to ignore ### Consider Parameter Variations Good: "The username field will contain different values based on parameters - ignore the actual text, just verify the field exists." Less Good: "Username should be John" ### Handle Edge Cases Good: "Cache hit if either the success modal OR the results table is visible. Cache miss if neither is present." Less Good: "Something might be wrong" ### Keep It Clear Aim for clear, unambiguous criteria. Remember: your prompt replaces Cyberdesk's default cache-detection context for this step. ## When to Use Custom Prompts Consider adding a custom prompt when: * A step frequently has false positives or negatives * The validation criteria are subtle or nuanced * Dynamic content makes default comparison unreliable * You've observed specific failure patterns * The step is critical to workflow success * You need precise control over what defines a match You don't need custom prompts for every step - only where the default cache detection logic isn't working well for your use case. ## Technical Details ### Data Storage Custom prompts are stored in the `pre_check_snapshot` object of each trajectory step: ```json theme={null} { "pre_check_snapshot": { "screenshot": "...", "last_agent_thought": "...", "custom_cache_detection_instructions": "Your prompt here", "coordinates": { "x": 512, "y": 384 } } } ``` ### How Override Works When a custom prompt is set, the cache detection agent receives: ``` Your task is to determine whether a cached action (originally successfully executed on desktop #1's state) would successfully replay on desktop #2's current state. Desktop #1 is verified to be a desired state to execute the cached action. The user has provided the following custom cache detection instructions, that you should strictly adhere to. They are the only rules that apply to this decision. [Your custom prompt] Note: Both screenshots have a red circle annotation at the same coordinate to help you identify if the UI has shifted. ``` The note about red circles is only included if coordinate annotations are present. When no custom prompt is set, the system uses its default cache detection logic, which considers workflow context, the recorded step/action, the agent's last thought, parameter-level differences, and coordinate-shift hints when available. ### Backward Compatibility Trajectories without custom prompts continue to work normally using the default cache detection logic. The field is optional and defaults to empty/unset. # Desktop Parameters Source: https://docs.cyberdesk.io/concepts/desktop-parameters Machine-specific input values that automatically populate runs on specific desktops Desktop Parameters allow you to configure machine-specific values that automatically populate workflows running on that desktop. This is useful for credentials, file paths, or IDs that are unique to each machine. Despite the name, these are workflow input defaults tied to a machine, not display settings like resolution or scaling. ## The Problem You have workflows that need different input values depending on which machine runs them: * Different login credentials for each machine * Machine-specific file paths or configurations * Hardware-specific settings or IDs You can't hardcode these in your workflow prompts because you don't always know which machine will be assigned to a run. ## The Solution Set Desktop Parameters on each machine. When a run is assigned to that machine, the parameters automatically populate the effective input payload used by that run. ## How It Works 1. **Configure Parameters**: In the desktop details page, add parameters with names and values in the "Edit" dialog 2. **Use in Workflows**: Reference them in prompts using standard syntax: `{parameter_name}` or `{$sensitive_parameter}` 3. **Automatic Population**: When machine context is known, desktop parameters merge into the effective input payload used for validation and execution 4. **Storage Behavior**: Desktop parameters stay on the machine record; they are not copied into the run's stored input values unless you also pass them explicitly at run creation 5. **Priority**: Run-level input/sensitive values override desktop parameter defaults, and desktop parameters override pool parameter defaults 6. **Input Schema Compatibility**: You can mark desktop-parameter keys as required in workflow `input_schema`; when machine context is already known, those values participate in create-time validation automatically. For pool/auto assignment, required-key checks that may be satisfied by desktop parameters are deferred until a machine is assigned. Need literal template-style text in the prompt instead of resolving a variable? Escape the opening delimiter: * `\{customer_name}` renders as literal `{customer_name}` * `\{{current_status}}` renders as literal `{{current_status}}` * `\{$support_pin}` renders as literal `{$support_pin}` ## Example ### Setup Desktop Parameters In the desktop details page, click the "Edit" button in the top right, then configure parameters for Machine A: ```json theme={null} { "username": "user_machine_a", "api_url": "https://api-region-east.example.com", "database_path": "C:\\MachineA\\data.db" } ``` ### Use in Workflow ``` Log in to the system at {api_url} using username {username}. Then open the database at {database_path} and run the query. ``` ### Result When this workflow runs on Machine A, the system automatically replaces: * `{api_url}` → `https://api-region-east.example.com` * `{username}` → `user_machine_a` * `{database_path}` → `C:\\MachineA\\data.db` ## Sensitive Parameters For sensitive values like passwords or API keys, mark parameters as "Sensitive": 1. In the edit desktop dialog, check the "Sensitive" checkbox when adding a parameter 2. Enter the sensitive value (it will be obscured as you type) 3. The value is stored securely in Basis Theory 4. Use in prompts with sensitive syntax: `{$password}`. Configure the key as `password`; the `$` prefix is only used when referencing it in prompts or input schemas. ### Example with Sensitive Parameters Desktop parameters: ```json theme={null} { "username": "admin_user", "password": "●●●●●●" } ``` Workflow prompt: ``` Log in using {username} and password {$password}. ``` When machine context is known, the system securely resolves the stored password for validation and execution without exposing the plaintext value in the UI. ## Configuration via SDK You can also set desktop parameters programmatically: ```python Python theme={null} from cyberdesk import Cyberdesk client = Cyberdesk(api_key="your_api_key") # Update machine with parameters client.machines.update( machine_id="machine-id", machine_parameters={ "username": "machine_specific_user", "config_path": "/opt/app/config.json" }, machine_sensitive_parameters={ "api_key": "actual_secret_value_123", # Will be stored securely, only revealed at the last mile, never logged "password": "actual_password_456" } ) ``` ```typescript TypeScript theme={null} import { Cyberdesk } from 'cyberdesk'; const client = new Cyberdesk({ apiKey: 'your_api_key' }); // Update machine with parameters await client.machines.update('machine-id', { machine_parameters: { username: 'machine_specific_user', config_path: '/opt/app/config.json' }, machine_sensitive_parameters: { api_key: 'actual_secret_value_123', // Will be stored securely, only revealed at the last mile, never logged password: 'actual_password_456' } }); ``` ## Best Practices ### ✅ Good Use Cases * Machine-specific credentials or API keys * Hardware-specific paths or configurations * Region-specific URLs or endpoints * Machine-assigned identifiers ### ❌ Avoid * Workflow-specific logic (use input values instead) * Data that changes frequently (parameters are relatively static) * Large datasets (use file attachments instead) ## Priority and Overrides When run input values, desktop parameters, and pool parameters are available: 1. **Run-level values take priority** and override desktop and pool defaults 2. Desktop parameters override pool parameters for the same key 3. Pool parameters fill missing keys only when the run explicitly selected the pool 4. Desktop parameters persist across runs and are not deleted after execution An explicit run-level `__EMPTY__` value is treated as an intentional override, so it will suppress the desktop default for that key. Omitted/null/blank values still allow desktop defaults to populate. If you want the run to actively blank out the field instead of simply suppressing the desktop default, use [`__CLEAR__`](/concepts/clear-values). For shared defaults across a group of desktops, see [Pool Parameters](/concepts/pool-parameters). ## Security * Sensitive desktop parameters are stored in Basis Theory * Values are never displayed in the UI after being saved * Only parameter names are visible; values show as "●●●●●●" * Sensitive values are securely resolved for validation/execution and never logged * Desktop parameters are only accessible to runs on that specific machine # Empty Values Source: https://docs.cyberdesk.io/concepts/empty-values Handle optional form fields in workflows with the __EMPTY__ sentinel value When developers want to skip certain form fields in workflows, Cyberdesk provides the `__EMPTY__` sentinel value. If you want to actively clear an already-filled field instead, use [`__CLEAR__`](/concepts/clear-values). Cyberdesk also treats optional empty strings and `null` values as "not provided" when building run inputs, so they are normalized to `__EMPTY__` before execution. ## Simple Use Case **Problem**: Sometimes workflows have optional form fields that should be skipped when no data is provided. **Solution**: Use `__EMPTY__` to skip typing while maintaining workflow structure. ## Example ```python theme={null} # User provides username but no email input_values = {"username": "john_doe", "email": "__EMPTY__"} # Agent behavior: type_text("john_doe") # Types the username type_text("__EMPTY__") # Skips typing (no email provided) ``` ## Automatic `__EMPTY__` for Missing Nested Fields When using [structured inputs](/concepts/structured-inputs) with nested access, missing fields automatically become `__EMPTY__`: ```python theme={null} # Prompt uses: {patient.middle_name} # Input only has first and last name: input_values = { "patient": { "first_name": "John", "last_name": "Doe" # middle_name is not provided } } # Result: {patient.middle_name} → __EMPTY__ ``` This allows workflows to gracefully handle optional nested fields without requiring explicit `__EMPTY__` values. ## Nested Access on `__EMPTY__` Values If a root variable is `__EMPTY__` and you try to access a nested property on it, the result is also `__EMPTY__`: ```python theme={null} # Prompt uses: {extra_info.notes} # User provides nothing for extra_info (auto-converted to __EMPTY__) input_values = {"extra_info": "__EMPTY__"} # Result: {extra_info.notes} → __EMPTY__ ``` This allows workflows to gracefully skip entire optional sections without failing. **Type errors are different**: If you try to access a nested property on a value that is a concrete type like a string or number (e.g., `{patient.name.first}` when `patient.name` is `"John Doe"`), the run fails immediately with a clear error. Only missing fields and `__EMPTY__` values become `__EMPTY__`. ## Input Schema Validation When a workflow has an input schema, `__EMPTY__` is treated as "not provided" for validation. * Object fields set to `__EMPTY__` are ignored during schema validation. * Array entries set to `__EMPTY__` are removed before validation. * This applies to auto-generated `__EMPTY__` values too, including optional inputs left blank as empty strings or `null`. This lets optional fields stay blank without failing validation, while required fields still need a real value unless your schema allows them to be omitted. ## When to Use * **Optional form fields** that might not always be filled * **Progressive workflows** where some data is provided later * **A/B testing** different workflow variants * **Graceful degradation** when inputs are missing * **Nested optional fields** in structured input objects ## Best Practice ```python theme={null} # ✅ Good: Simple optional step type_text("__EMPTY__", conditional="input_values['optional_field'] == '__EMPTY__'") # ❌ Avoid: Complex logic - split workflows instead ``` This is a simple convenience feature for handling occasional missing data - not for complex workflow logic. If you need to blank out an existing value instead of skipping, see [Clear Values](/concepts/clear-values). # Generating Output Data Source: https://docs.cyberdesk.io/concepts/generating-output-data How observations, runtime values, and extractions transform into structured workflow output ## Overview When a Cyberdesk workflow reaches a terminal state, you often want structured data as output - not just "task completed." This page explains how Cyberdesk transforms various types of captured data into structured JSON output that matches your schema. This transformation process works whether your workflow runs from scratch or uses [cached trajectories](/concepts/trajectories). Dynamic tools like `focused_action` and `extract_prompt` always capture fresh data, even during trajectory replay. ## The Output Data Pipeline During workflow execution, Cyberdesk captures data from multiple sources: **Observations** - Dynamic data extracted by: * `focused_action` - Context-aware decisions and extractions * `extract_prompt` - Vision-based data extraction (sync, batch, or run-scoped) **Runtime Values** - Variables set by: * `copy_to_clipboard` - Clipboard-based extraction * `upsert_runtime_values` - Direct variable setting (from async extract\_prompt) * `focused_action` - Variable assignment via `{{variable}}` syntax At terminal completion, the **Transformation Agent** combines all observations and runtime values to generate structured `output_data` JSON matching your schema. ## Components of Output Data ### 1. Output Schemas Define the structure of your expected output data when creating a workflow. **Example Schema**: ```json theme={null} { "type": "object", "properties": { "patient_mrn": { "type": "string" }, "vital_signs": { "type": "object", "properties": { "blood_pressure": { "type": "string" }, "heart_rate": { "type": "number" }, "temperature": { "type": "number" } } }, "medications": { "type": "array" }, "lab_results": { "type": "array" } } } ``` **Format**: `output_schema` must be valid JSON Schema. The one special passthrough marker is `{"only_runtime_values": true}`, which skips transformation and returns the runtime values map as-is. **Purpose**: * Defines expected structure and data types * Guides the transformation agent * Enables validation and type checking * Makes output predictable and consistent **Optional but Recommended**: You don't need an output schema for every workflow, but having one ensures consistent, structured data that's easy to parse and use in downstream systems. ### 2. Observations Captured during workflow execution by multiple tools: * [focused\_action](/workflow-prompting/focused-action) - Dynamic observations and decisions * [extract\_prompt](/workflow-prompting/extract-prompt) - Vision-based data extraction (sync, batch, or run-scoped) **How They're Created**: **Via focused\_action**: ```text theme={null} "Use focused_action to extract the patient vital signs from the screen and note: blood pressure, heart rate, temperature, and oxygen saturation." ``` **Via extract\_prompt** (any mode): ```text theme={null} "Take screenshot with extract_prompt='Extract patient vital signs as JSON: {blood_pressure, heart_rate, temperature, oxygen_saturation}'" ``` **What Gets Stored**: * The timestamp when the observation was made * The extracted observation text, or an error when extraction fails * The screenshot captured at that moment * Source metadata such as cached/extraction mode details * In focused actions and some extraction paths, the instruction that produced the observation **Example Observation Entries**: **From focused\_action**: ```json theme={null} { "timestamp": "2024-01-15T14:30:22Z", "instruction": "Extract patient vital signs", "observation": "Blood Pressure: 120/80 mmHg, Heart Rate: 72 bpm, Temperature: 98.6°F, O2 Saturation: 98%", "screenshot": "base64_image_data", "cached": false } ``` **From extract\_prompt** (with source field): ```json theme={null} { "timestamp": "2024-01-15T14:30:25Z", "instruction": "screenshot.extract: Extract medications as JSON array", "observation": "[{\"name\": \"Aspirin\", \"dosage\": \"81mg\"}, ...]", "screenshot": "base64_image_data", "cached": false, "source": "screenshot_extract", "zoom_bounding_box": [100, 200, 500, 600] } ``` **Key Characteristics**: * Dynamic: Re-evaluated on every run, even in cached workflows * Multi-source: From focused\_action, extract\_prompt (sync/batch/run) * Contextual: Includes the instruction for clarity * Visual: Screenshot attached for verification * Structured: Consistently formatted for transformation ### 3. Runtime Values Set during workflow execution via: * [focused\_action](/workflow-prompting/focused-action) with `{{variable}}` syntax * [copy\_to\_clipboard](/workflow-prompting/copy-to-clipboard) with key names * [extract\_prompt](/workflow-prompting/extract-prompt) with `process_async` (batch or run) and `upsert_runtime_values` **How They're Created**: **Via Focused Action**: ```text theme={null} "Use focused_action to find the invoice number on screen and save it as `{{invoice_number}}`" ``` **Via Copy to Clipboard**: ```text theme={null} "Triple-click the account number and use copy_to_clipboard with key name 'account_number'" ``` **Via Async Extraction** (batch or run scoped): ```text theme={null} "Take screenshot with extract_prompt='Extract customer_id and order_total as runtime variables using upsert_runtime_values' and process_async='batch'" ``` or ```text theme={null} "Take screenshot with extract_prompt='Extract customer_id and order_total as runtime variables using upsert_runtime_values' and process_async='run'" ``` **What Gets Stored**: ```json theme={null} { "invoice_number": "INV-2024-001", "account_number": "1234567890", "customer_id": "CUST-5678", "order_total": 1250.00 } ``` **Key Characteristics**: * Immediate: Available as `{{variable_name}}` in subsequent workflow steps * Flexible: Can be strings, numbers, or simple objects * Persistent: Included in final output transformation * Reusable: Can be used multiple times in workflow Runtime values are perfect for: * IDs and reference numbers needed later in workflow * Values used in file naming or path construction * Data that determines workflow branching * Key metrics that appear in multiple places in output ### 4. The Transformation Agent At the end of a terminal run, if an output schema is defined, the **transformation agent** converts all captured observations and runtime values into structured JSON. This includes `success`, `task_failed`, `cancelled`, and `error` runs. If a run ends early, Cyberdesk still attempts to generate schema-shaped `output_data` from the captured data and any schema defaults. **Input to Transformation Agent**: 1. Your defined output schema 2. All observations (from focused\_action and extract\_prompt) 3. All runtime values (from copy\_to\_clipboard, upsert\_runtime\_values, focused\_action) **Process**: ``` Transformation Agent receives: - Output Schema: {patient_mrn: string, vitals: object, ...} - Observations: [ {instruction: "Extract vitals", observation: "BP: 120/80, HR: 72..."}, {instruction: "screenshot.extract: medications", observation: "[{name: 'Aspirin'...}]"} ] - Runtime Values: {patient_mrn: "MRN12345", patient_age: 45} Transformation Agent analyzes and maps: - "patient_mrn" → use runtime value "MRN12345" - "vitals" → extract from observation "Blood pressure 120/80..." - "medications" → extract from observation "Medications: aspirin..." - "lab_results" → use extraction result Transformation Agent outputs: { "patient_mrn": "MRN12345", "vital_signs": { "blood_pressure": "120/80", "heart_rate": 72, "temperature": 98.6 }, "medications": ["aspirin 81mg daily", "lisinopril 10mg daily"], "lab_results": [...] } ``` **System Prompt** (simplified): ``` You are a data transformation assistant. Extract structured data from observations and runtime values, formatting according to the provided JSON schema. Be precise and only include information actually captured. If a required field cannot be determined, use null or appropriate default value. Runtime values are extracted using copy_to_clipboard or focused actions - they often represent important identifiers that should be included in output. ``` **Key Characteristics**: * Intelligent: Maps observations to schema fields semantically * Comprehensive: Includes runtime values automatically * Type-aware: Converts strings to numbers, arrays, etc. as needed * Validated: Ensures output matches schema structure ## Output Data Optimization Features The transformation agent supports two powerful optimization features to reduce LLM-induced lossiness and improve efficiency. ### 1. Direct Runtime Values Output If you've already collected exactly what you need via runtime variables and don't need any LLM transformation, you can skip the transformation step entirely. **How to Use**: Set your output schema to: ```json theme={null} {"only_runtime_values": true} ``` This immediately returns the runtime values map as-is, without any LLM processing. **Example**: ```text theme={null} "Navigate to order details page. Triple-click Order ID and use copy_to_clipboard with key name 'order_id' Triple-click Customer ID and use copy_to_clipboard with key name 'customer_id' Take screenshot with extract_prompt='Extract all order line items, shipping details, and payment info. Store as runtime variables using upsert_runtime_values' and process_async='run' Navigate to next page..." ``` **Runtime Values Collected**: ```json theme={null} { "order_id": "ORD-2024-5678", "customer_id": "CUST-1234", "line_items": [...], "shipping_details": {...}, "payment_info": {...} } ``` **Output Schema**: `{"only_runtime_values": true}` **Result**: The runtime values are returned exactly as collected, with zero lossiness. **When to Use**: * ✅ All data is already in runtime values * ✅ You want zero LLM-induced modifications * ✅ Data structure is already exactly what you need * ✅ You're using focused actions or extract\_prompt extensively to set runtime values **Benefits**: * ⚡ Instant - no LLM transformation call * 🎯 Zero lossiness - exact values preserved * 💰 Cheaper - no transformation tokens * 🔒 Predictable - no chance of LLM hallucination ### 2. Automatic Runtime Value Referencing The transformation agent is smart enough to reference existing runtime values directly instead of regenerating them, reducing lossiness and token usage. **How It Works**: When transforming observations into output data, the transformation agent can automatically detect when a value should come from runtime values instead of being regenerated. It uses internal template syntax to reference these values, which are then deterministically substituted with the exact values. **Example**: **Runtime Values Collected**: ```json theme={null} { "long_transcript": "This is a very long transcript of a customer call that spans multiple paragraphs and contains detailed conversation history...", "customer_id": "CUST-5678", "order_items": [ {"sku": "WIDGET-A", "name": "Premium Widget", "price": 29.99}, {"sku": "GADGET-B", "name": "Super Gadget", "price": 49.99} ] } ``` **Observations**: ```json theme={null} [ { "instruction": "Extract customer sentiment", "observation": "Customer was satisfied with service, rated 9/10" } ] ``` **Output Schema**: ```json theme={null} { "type": "object", "properties": { "customer_id": {"type": "string"}, "sentiment": {"type": "string"}, "rating": {"type": "number"}, "full_transcript": {"type": "string"}, "order_summary": {"type": "object"} } } ``` **Final Output** (automatic optimization): ```json theme={null} { "customer_id": "CUST-5678", "sentiment": "satisfied", "rating": 9, "full_transcript": "This is a very long transcript of a customer call that spans multiple paragraphs and contains detailed conversation history...", "order_summary": { "items": [ {"sku": "WIDGET-A", "name": "Premium Widget", "price": 29.99}, {"sku": "GADGET-B", "name": "Super Gadget", "price": 49.99} ], "item_count": 2 } } ``` The transformation agent automatically references `customer_id`, `long_transcript`, and `order_items` from runtime values instead of regenerating them, ensuring exact values are preserved. **Benefits**: * 🎯 Zero lossiness - exact runtime values preserved * ⚡ Faster - less content to generate * 💰 Cheaper - fewer output tokens * 🔒 Reliable - no chance of LLM typos in long values This optimization happens automatically - you don't need to do anything special. The transformation agent is instructed to use internal referencing when appropriate. ## Complete Flow Example ### Healthcare Workflow **1. Define Output Schema**: ```json theme={null} { "type": "object", "properties": { "patient_mrn": { "type": "string" }, "date_of_birth": { "type": "string" }, "vital_signs": { "type": "object", "properties": { "blood_pressure": { "type": "string" }, "heart_rate": { "type": "number" }, "temperature": { "type": "number" } } }, "current_medications": { "type": "array" }, "latest_lab_results": { "type": "object", "properties": { "test_date": { "type": "string" }, "results": { "type": "array" } } } } } ``` **2. Workflow Execution**: ```text theme={null} "Log into EHR system with {username} and {$password}. Navigate to patient search and search for {patient_name}. In the patient demographics section: - Triple-click on the Medical Record Number and use copy_to_clipboard with key name 'patient_mrn' - Triple-click on Date of Birth and use copy_to_clipboard with key name 'date_of_birth' Navigate to Vitals tab. Use focused_action to extract current vital signs and note: blood pressure, heart rate, and temperature. Navigate to Medications tab. Take screenshot with extract_prompt='Extract all current medications as JSON array with fields: name, dosage, frequency' and process_async='batch' Navigate to Lab Results tab. Take screenshot with extract_prompt='Extract most recent lab results as JSON with test_date and results array' and process_async='run' Continue with documentation workflow." ``` **3. Data Captured During Execution**: **Runtime Values** (from copy\_to\_clipboard): ```json theme={null} { "patient_mrn": "MRN12345", "date_of_birth": "1985-03-15" } ``` **Focused Observations** (from focused\_action): ```json theme={null} [ { "timestamp": "2024-01-15T14:30:22Z", "instruction": "Extract current vital signs", "observation": "Blood Pressure: 120/80 mmHg, Heart Rate: 72 bpm, Temperature: 98.6°F", "screenshot": "...", "cached": false } ] ``` **Extraction Results** (from extract\_prompt): ```json theme={null} [ { "scope": "batch", "prompt": "Extract all current medications as JSON array", "result": [ {"name": "Aspirin", "dosage": "81mg", "frequency": "once daily"}, {"name": "Lisinopril", "dosage": "10mg", "frequency": "once daily"} ] }, { "scope": "run", "prompt": "Extract most recent lab results", "result": { "test_date": "2024-01-10", "results": [ {"test": "CBC", "value": "Normal", "reference": "Normal"}, {"test": "Glucose", "value": "95", "reference": "70-100"} ] } } ] ``` **4. Transformation** (at run completion): The transformation agent receives all captured data and the schema, then produces: ```json theme={null} { "patient_mrn": "MRN12345", "date_of_birth": "1985-03-15", "vital_signs": { "blood_pressure": "120/80", "heart_rate": 72, "temperature": 98.6 }, "current_medications": [ {"name": "Aspirin", "dosage": "81mg", "frequency": "once daily"}, {"name": "Lisinopril", "dosage": "10mg", "frequency": "once daily"} ], "latest_lab_results": { "test_date": "2024-01-10", "results": [ {"test": "CBC", "value": "Normal", "reference": "Normal"}, {"test": "Glucose", "value": "95", "reference": "70-100"} ] } } ``` This structured output is now available via the API and can be used by downstream systems! ## Run-Scoped Extraction with Runtime Variables Run-scoped extractions have a unique capability: they can both store runtime variables AND provide comprehensive observations. ### The Extraction Agent Loop When using `process_async="run"`, the extraction becomes a proper agent with access to `upsert_runtime_values`: **System Prompt**: ``` You are an extraction assistant. You have two capabilities: 1. Call upsert_runtime_values to store specific extracted values that should be available throughout the workflow as `{{key_name}}` placeholders. 2. Provide final observations as text describing what you see on screen. You can do BOTH: store specific values AND provide observations, or just do one or the other. Your final text message will be recorded as the extraction result. ``` ### Example: Store Key Fields + Comprehensive Extraction ```text theme={null} "Navigate to order details page. Take screenshot with extract_prompt='Extract order_id and customer_id and store them as runtime variables using upsert_runtime_values. Then extract complete order details including all line items, shipping info, payment details, and order history as detailed JSON.' and process_async='run' Continue generating shipping label using `{{order_id}}` and `{{customer_id}}` in the label. The full order details will be available in final output." ``` **What Happens**: 1. **Extraction Agent Analyzes Screenshot** 2. **Calls upsert\_runtime\_values**: ```json theme={null} { "order_id": "ORD-2024-5678", "customer_id": "CUST-1234" } ``` 3. **Provides Detailed Observation**: ```json theme={null} { "order_id": "ORD-2024-5678", "customer_id": "CUST-1234", "line_items": [ {"sku": "WIDGET-A", "quantity": 2, "price": 29.99}, {"sku": "GADGET-B", "quantity": 1, "price": 49.99} ], "shipping": { "address": "123 Main St, City, State 12345", "method": "Standard", "tracking": "TRACK123456" }, "payment": { "method": "Credit Card", "last_4": "4242", "amount": 109.97 }, "order_history": [...] } ``` 4. **Runtime Values Immediately Available**: * `{{order_id}}` can be used in shipping label * `{{customer_id}}` can be used in customer lookup 5. **Full Details in Final Output**: * Complete observation included in transformation * Both runtime values and detailed data in output\_data JSON ### Benefits This pattern gives you: * **Immediate access** to key identifiers via runtime variables * **Non-blocking extraction** of comprehensive data * **Single extraction** instead of multiple separate calls * **Flexible output** tailored to your needs ## Best Practices ### 1. Design Your Schema First Before writing workflow prompts, define your output schema: ```json theme={null} { "type": "object", "properties": { "primary_id": { "type": "string" }, "core_data": { "type": "object" }, "details": { "type": "array" }, "metadata": { "type": "object" } } } ``` ### 2. Use the Right Tool for Each Data Type | Data Type | Best Tool | Example | | ----------------------- | ---------------------------------------------------------------------------------- | ----------------------------- | | IDs, Numbers (copyable) | [copy\_to\_clipboard](/workflow-prompting/copy-to-clipboard) | Account numbers, order IDs | | Dynamic decisions | [focused\_action](/workflow-prompting/focused-action) | Status checks, validations | | Large extractions | [extract\_prompt](/workflow-prompting/extract-prompt) with `process_async="run"` | Analytics, comprehensive data | | List processing | [extract\_prompt](/workflow-prompting/extract-prompt) with `process_async="batch"` | Scrolling through tables | ### 3. Set Runtime Variables for Key Identifiers If a value is used multiple times or in file naming, make it a runtime variable: ```text theme={null} "Extract customer_id and save as `{{customer_id}}` for use in: - File naming: Report`{{customer_id}}`.pdf - API calls: GET /api/customers/`{{customer_id}}` - Output data: customer_id field" ``` ### 4. Use Focused Actions for Critical Observations Use focused\_action when: * The observation requires decision-making * You need to verify something visually * The data determines workflow branching * You want to ensure dynamic re-evaluation in cached runs ### 5. Use Run-Scoped Extraction for Output-Only Data If data is only needed in final output (not for navigation), use run-scoped: ```text theme={null} "Take screenshot with extract_prompt='Extract complete report data' and process_async='run' This doesn't block the workflow - extraction happens in background." ``` ### 6. Request JSON Format Always request JSON for structured data: ```text theme={null} extract_prompt='Extract order data as JSON: {order_id: string, items: array, total: number, status: string}' ``` ### 7. Use Async Extraction for Runtime Variables When using `process_async` (batch or run), extraction agents can call `upsert_runtime_values`: ```text theme={null} "Take screenshot with extract_prompt='Extract order_id as runtime variable using upsert_runtime_values, then describe the order details' and process_async='batch'" ``` This stores `{{order_id}}` for later use while also providing comprehensive observations. ### 8. Include Type Information Help the transformation agent by specifying types: ```text theme={null} extract_prompt='Extract metrics as JSON: {revenue: number, customers: number, growth_rate: number (as percentage), categories: array of strings}' ``` ## Common Patterns ### Pattern 1: ID + Details Extract ID first (fast), then comprehensive details (async): ```text theme={null} "Triple-click the Order ID and use copy_to_clipboard with key name 'order_id' Take screenshot with extract_prompt='Extract complete order details as JSON' and process_async='run' Continue workflow using `{{order_id}}` for file naming. Details in final output." ``` ### Pattern 2: Decision + Data Make decision synchronously, extract data asynchronously: ```text theme={null} "Take screenshot with extract_prompt='Extract account status: Active, Suspended, or Closed' to determine next steps. If Active, take screenshot with extract_prompt='Extract complete account history and analytics as detailed JSON' and process_async='run' Continue with appropriate workflow path." ``` ### Pattern 3: Iterative Extraction + Summary Extract from multiple views, then summarize: ```text theme={null} "For each page in report: - Take screenshot with extract_prompt='Extract page data as JSON' and process_async='batch' - Go to next page After all pages, take screenshot with extract_prompt='Extract summary statistics and totals' and process_async='run' All data available in final output." ``` ### Pattern 4: Mixed Sources Combine all extraction methods: ```text theme={null} "Extract customer data using optimal method for each field: Copyable fields (fast): - copy_to_clipboard for customer_id - copy_to_clipboard for account_number Dynamic observation (for decisions): - focused_action to check account status and extract current balance Comprehensive data (for output): - extract_prompt with process_async='run' for complete transaction history All values combined in final structured output." ``` ## Output Data Access ### Via API After workflow completes, access output\_data: ```python theme={null} from cyberdesk import CyberdeskClient, RunCreate import time client = CyberdeskClient(api_key="your_api_key") # Create and wait for run response = client.runs.create_sync( RunCreate( workflow_id="workflow_123", input_values={"patient_name": "John Doe"}, ) ) run = response.data while run.status in ["scheduling", "running"]: time.sleep(2) run = client.runs.get_sync(run.id).data # Access structured output if it was generated if run.output_data is not None: output_data = run.output_data print(f"Patient MRN: {output_data['patient_mrn']}") print(f"Vitals: {output_data['vital_signs']}") print(f"Medications: {output_data['current_medications']}") ``` ### Via Webhooks Receive output\_data when run completes: ```python theme={null} from fastapi import FastAPI, Request from cyberdesk.webhooks import verify_webhook, RunCompletedEvent app = FastAPI() @app.post("/webhooks/cyberdesk") async def handle_webhook(request: Request): # Verify webhook signature body = await request.body() signature = request.headers.get("x-cyberdesk-signature") verify_webhook(body, signature, webhook_secret="your_secret") # Parse event data = await request.json() event = RunCompletedEvent.from_dict(data) output_data = event.run.output_data if output_data is not None: # Process output_data print(f"Received output: {output_data}") return {"ok": True} ``` ## Troubleshooting ### Output Data is None **Causes**: * No output schema defined * Transformation failed **Solutions**: * Define an output schema in workflow settings * Use schema defaults when you want terminal runs with limited captured data to still produce structured output * Check run logs for transformation errors ### Missing Fields in Output **Causes**: * Field not captured during workflow * Field name mismatch between schema and observations * Transformation couldn't map observation to field **Solutions**: * Verify observations contain the expected data * Use clear, descriptive field names in schema * Request JSON with explicit field names in extractions ### Incorrect Data Types **Causes**: * Schema specifies number but observation has string * Vision model returned unexpected format **Solutions**: * Specify types in extraction prompts: "Extract age as number" * Use runtime values for precise extractions * Request "as JSON with types: \{field: number}" ### Runtime Variables Not in Output **Cause**: * Transformation agent didn't include them **Solution**: * Runtime values are automatically included - check schema field names match variable names * If mismatch, transformation agent will try to map semantically ## Summary Output data generation in Cyberdesk is a powerful pipeline that combines: 1. **Observations** - Dynamic data from focused\_action and extract\_prompt 2. **Runtime Values** - Immediate identifiers and metrics 3. **Transformation Agent** - Intelligent mapping to your schema All observations (whether from focused\_action or extract\_prompt) are stored together and transformed into structured output. By understanding this pipeline and using the right tools for each data type, you can build workflows that produce consistent, structured output data ready for integration with any downstream system. **Note**: The observations list contains data from both `focused_action` and `extract_prompt` tools. They're all stored together and transformed into the final output. For detailed information, see: * [Trajectories 101](/concepts/trajectories) - How caching and replay work * [Focused Action](/workflow-prompting/focused-action) - Dynamic observations * [Copy to Clipboard](/workflow-prompting/copy-to-clipboard) - Fast clipboard extraction * [Extract Prompt](/workflow-prompting/extract-prompt) - Vision-based extraction * [Async Extraction Patterns](/concepts/async-extraction-patterns) - Performance optimization # Holding a Key Source: https://docs.cyberdesk.io/concepts/holding-a-key Press and hold a key while performing other actions by using the key action's down parameter. Some workflows need a keyboard modifier to be **held down while another action runs** — for example, holding `shift` while clicking to extend a selection, holding `ctrl` while clicking multiple files, or holding `alt` while scrolling to trigger an alternate gesture. The `key` action supports an optional `down` parameter that controls whether the key is pressed, held, or released. Pair a `down=true` call with a later `down=false` call on the same key so the key never ends up stuck in the pressed state. **Key syntax:** Use `+` to press keys **together** as one chord, e.g. `ctrl+a` or `alt+tab`. A chord can combine multiple modifiers (`ctrl`, `alt`, `shift`, `win`) but only one non-modifier key. Use **spaces** to press keys **one after another**, e.g. `down down down` (tap Down three times) or `ctrl+c ctrl+v`. Joining repeated or multiple non-modifier keys with `+` (e.g. `down+down+down`) is invalid — use spaces instead. ## The `down` parameter `down` is tri-state: | `down` value | Behavior | | ----------------- | ------------------------------------------------------------------------------------------------------------- | | omitted (default) | Full press: press the key down and release it immediately. Backwards-compatible with the normal `key` action. | | `true` | Press the key down and **hold it** without releasing. | | `false` | Release a key that was previously held with `down=true`. | `down` only applies to the `key` action. It is ignored for `type` and every other action. ## Basic usage pattern To hold a key while performing another action: Call `key` with the key chord and `down=true`. The key stays pressed until you release it. Call `left_click`, `mouse_move`, `scroll`, or any other action while the key is held. Call `key` again with the **same key** and `down=false` to release it. ## Example: shift-click to extend a selection ``` 1. Use `left_click` at the first item's coordinates to select it. 2. Use `key` with text="shift" and down=true to press and hold Shift. 3. Use `left_click` at the last item's coordinates to extend the selection. 4. Use `key` with text="shift" and down=false to release Shift. ``` ## Example: ctrl-click to toggle multi-select ``` 1. Use `left_click` to select the first item. 2. Use `key` with text="ctrl" and down=true to press and hold Ctrl. 3. Use `left_click` to toggle each additional item. 4. Use `key` with text="ctrl" and down=false to release Ctrl. ``` ## Example: ctrl-scroll to zoom ``` 1. Use `key` with text="ctrl" and down=true to press and hold Ctrl. 2. Use `scroll` with scroll_direction="up" and scroll_amount=3 to zoom in. 3. Use `key` with text="ctrl" and down=false to release Ctrl. ``` ## Guidelines * **Always pair `down=true` with a matching `down=false`.** If you hold a modifier and forget to release it, every subsequent action in the workflow will be affected, and downstream typing or shortcuts will behave unexpectedly. * **Release the same key you pressed.** If you pressed `shift` down, release `shift`, not `lshift` or `shiftleft`. * **Avoid key chords with `down=true`.** `down` is designed for simple modifier keys like `shift`, `ctrl`, `alt`, and `win`. Do not use it with chords such as `ctrl+shift`. * **Prefer the default full-press form when you don't need to hold.** If you just need `ctrl+a` as a one-shot, use `action="key"` with `text="ctrl+a"` and no `down` parameter. That's the normal behavior. * **Take a screenshot after releasing if you need to verify the result.** The screenshot after a `down=true` call shows the screen *while the key is held*, which is usually not what you want to verify against. ## Comparison with modifier-click shortcuts For simple modifier combinations like `ctrl+a` (select all), `ctrl+c` (copy), or `alt+tab` (switch window), keep using the default `key` action with the chord directly: ``` action="key", text="ctrl+a" ``` This presses and releases the chord in one atomic step. You only need the `down` parameter when the held key has to span **another action**, such as a click, a mouse move, or a scroll. # Input Validation Source: https://docs.cyberdesk.io/concepts/input-validation Validate workflow run inputs with JSON Schema before execution `input_schema` lets a workflow define the expected shape of run inputs using JSON Schema. When enabled, Cyberdesk validates the **effective** input payload: * `input_values` * `sensitive_input_values` * selected pool parameters (when pools are explicitly selected) * machine/session-provided values (when available) When multiple sources provide the same key, Cyberdesk uses run-level values first, then machine-provided defaults, then selected pool defaults. This includes explicit `__EMPTY__` values; omitted/null/blank keys still allow defaults to fill. `__CLEAR__` is preserved as a normal string during validation and later clears the field at typing time instead of being treated as omitted. Sensitive inputs are exposed to schema validation with a `$`-prefixed root key. For example, `sensitive_input_values.api_key` is available in schema as `$api_key`. This also means input and sensitive values can share a base name without collision: * input key: `customer_id` * sensitive key in schema: `$customer_id` This helps catch bad payloads early, before a run spends time on machine setup and execution. ## Why this exists Input variables in prompts are flexible, but flexibility can hide mistakes: * misspelled keys * missing required fields * wrong types (`"123"` vs `123`) * malformed nested objects `input_schema` makes these issues explicit and gives path-level errors. ## Where validation happens Validation is intentionally hybrid: 1. **Dashboard preflight (client-side):**\ As you prepare a run, the dashboard validates current inputs against the selected workflow's `input_schema` (when present).\ When pools are selected, pool parameters are merged into preflight validation. When machine context is known (for example, selected machine or an existing session with a reserved machine), known machine parameters are merged too.\ When machine context is not known yet (for example, auto/pool assignment), missing required root keys that may come from machine parameters are deferred, and strict checks continue later. 2. **API create-time (server-side):**\ The API validates run payloads and returns structured `422` errors for schema failures.\ When selected pools are known, pool-provided values are merged at create-time. When machine/session context is known, machine-provided values are merged and strict checks run at create-time.\ When machine context is not known yet, Cyberdesk still validates concrete user-provided and selected-pool values immediately, but defers root-level missing `required` keys that may be satisfied by machine defaults after assignment. 3. **Execution-time strict validation (worker):**\ Validation runs again after refs are resolved and execution context is fully known. If a payload contains unresolved refs (`$ref`), full strict validation may be deferred until execution-time when those values are available. ## Defining an input schema Add `input_schema` on your workflow as a JSON Schema object: ```json theme={null} { "type": "object", "required": ["accountId", "$apiKey"], "properties": { "accountId": { "type": ["string", "number", "boolean", "object", "array"] }, "$apiKey": { "type": ["string", "number", "boolean", "object", "array"] }, "amount": { "type": "number", "minimum": 0 }, "customer": { "type": "object", "required": ["name"], "properties": { "name": { "type": "string" }, "email": { "type": "string", "format": "email" } } } }, "additionalProperties": true } ``` By default in the dashboard: * detected prompt input and sensitive variables are auto-added to `properties` * all detected variables are added to `required` * each auto-generated property defaults to `type: ["string", "number", "boolean", "object", "array"]` * `additionalProperties` defaults to `true` * structured prompt paths (for example `{$nested.variable.hi}`) auto-generate nested object/array schema shape with required path keys `pattern` validation is fully supported for string schemas (for example `^\\d{2}-\\d{2}-\\d{4}$` for `MM-DD-YYYY`), and validation failures include path-level messages like `$.date: must match pattern ...`. ## Prompt-linked schema sync while editing In the workflow editor, `input_schema` stays synced to prompt variables as you type. * Sync is debounced while editing. * On **Save**, Cyberdesk performs a final sync pass. * If you customize constraints/descriptions, Cyberdesk preserves those edits and only updates variable **names/paths** when the prompt variable is renamed. * Sensitive variables stay mapped to `$`-prefixed schema keys. You can monitor this in the prompt toolbar: * **Input Schema - Syncing…** while reconciliation is running * **Input Schema - Synced** when complete Hover the indicator to quickly review what the sync does and jump to the schema editor. ## Chains and refs In chains, each step is validated against **that step's workflow** `input_schema`. Validation uses the merged payload for each step: * shared inputs + shared sensitive inputs * step inputs + step sensitive inputs Refs are supported: ```json theme={null} { "customer": { "$ref": "step1.outputs.result.customer" } } ``` ### Ref validation behavior Cyberdesk validates refs in layers: 1. **Create-time compatibility checks** * For chain refs to earlier steps in the same request, Cyberdesk validates the ref path/type against the producing step's `output_schema`. * For refs to existing runs in an existing session, Cyberdesk validates alias/path/type compatibility using available source metadata (`output_schema`, and concrete `output_data` types when present). * Create-time checks are intentionally permissive about **existence** when the source output may not be materialized yet (for example queued/scheduling runs in the same session). 2. **Execution-time strict checks** * After refs are resolved to concrete values, strict input schema validation runs again in the worker. * This is where required referenced values must actually exist. This means a ref can pass create-time compatibility but still fail at execution if the referenced value is missing or resolves to the wrong concrete type. Example: * Producer `output_schema` allows `data` as optional / nullable. * Consumer `input_schema` requires `data` as `string`. * If producer's actual `output_data` omits `data` (or sets it to `null`), consumer run fails when refs are resolved/executed. When refs are unresolved at create-time (for example future in-chain outputs), final strict validation still runs at execution-time after values are resolved. ## Error format (422) Schema validation errors return typed details so clients can render precise messages: ```json theme={null} { "detail": { "message": "Input schema validation failed", "error_code": "INPUT_SCHEMA_VALIDATION_FAILED", "details": [ { "path": "$.customer.email", "message": "must match format \"email\"" } ] } } ``` ## Dashboard UX notes * Workflow editor supports AI-assisted `input_schema` generation. * Run creation validates both form and JSON input modes. * Chain mode validates each step independently and reports step-scoped errors. ## Best practices * Start with `type: object` + `required` for critical fields. * Add constraints (`format`, `minimum`, enums) for high-signal validation. * Keep schema close to prompt expectations. * Use optional fields for truly optional inputs; avoid over-constraining. For nested prompt access patterns like `{customer.email}` or `{$credentials.token}`, pair this with [Structured Inputs](/concepts/structured-inputs). # Model Configuration Source: https://docs.cyberdesk.io/concepts/model-configuration Configure which AI models power your workflow agents and actions Model configuration determines which AI models run your workflows. You can set workflow-level defaults for the **main agent** and **cache detection**, and optionally **override the model per-action** directly in your prompts for `focused_action` and `extract_prompt` operations. The runtime applies defaults differently depending on the action: * `focused_action` inherits the workflow's main agent model unless you override it * `extract_prompt` does not inherit the workflow's main agent model; without an override, it uses Cyberdesk's extraction default (currently `Gemini 3.1 Pro Preview (Medium)`) ## Workflow-Level Model Configuration When you create or edit a workflow in the dashboard, you can select which model the main agent should use and which model should handle cache detection. ### Selecting a Model 1. Open the **Workflows** page in your dashboard 2. Click to create a new workflow or edit an existing one 3. In the workflow editor, use the **Main Model** and **Cache Detection Model** selectors 4. Choose from the available model configurations 5. Save your workflow If you leave either selector on **System Default**, Cyberdesk resolves the current system default and saves that model on the workflow when you save it. ### Current Default Assignments Cyberdesk currently applies these defaults automatically: | Role | Current model | | ----------------------------------------------- | ------------------------------------------------- | | Main agent default | `Sonnet 4.6 (Medium)` | | `extract_prompt` default | `Gemini 3.1 Pro Preview (Medium)` | | Post-run attachment image check default | `GPT 5.6 Sol (Low)` | | Webhook/output transformation default | `GPT 5.5 (Low)` | | Cache detection default | `Gemini 3 Flash Preview (Minimal, Short Timeout)` | | Cache detection fallback | `GPT 5.5` | | Fallback 1 when the primary agent model fails | `Vertex Sonnet 4.5 (Thinking)` | | Fallback 2 when the primary and Fallback 1 fail | `Bedrock Sonnet 4.5 (Thinking)` | Older `Gemini 3 Pro Preview` system configurations are retired. Existing workflow references and prompt overrides that used them are migrated to the matching `Gemini 3.1 Pro Preview` configurations. The model picker also includes other Cyberdesk system defaults and any custom model configurations owned by your organization. Current OpenAI system defaults include the `GPT 5.6` family (`Sol`, `Terra`, and `Luna` tiers) alongside `GPT 5.5`, `GPT 5.5 (Low)`, `GPT 5.5 (Medium)`, `GPT 5.5 (High)`, and `GPT 5.5 (XHigh)`. Older `GPT 5.4` configurations may appear in archived-model sections for existing workflows, but new defaults use `GPT 5.5`. ### Newly Available Model Options Cyberdesk also includes selectable configurations for newer provider models that are available without changing the platform defaults: * `GPT 5.6 Sol`, `GPT 5.6 Terra`, and `GPT 5.6 Luna` reasoning-effort variants (base, `Low`, `Medium`, `High`, and `XHigh`) for main workflow agents and computer-use tasks. Sol is the flagship tier for the hardest reasoning and agentic coding; Terra is the balanced everyday tier; Luna is the cost-optimized, high-volume tier for fast extraction, post-run checks, cache-style checks, and classification. * `Sonnet 5`, `Sonnet 5 (Low)`, `Sonnet 5 (Medium)`, and `Sonnet 5 (High)` for main workflow agents and computer-use tasks. * `Gemini 3.5 Flash (Minimal)`, `Gemini 3.5 Flash (Low)`, `Gemini 3.5 Flash (Medium)`, and `Gemini 3.5 Flash (High)` for main workflow agents, computer-use tasks, fast multimodal extraction, and cache-style checks. * `Gemini 3.5 Flash (Minimal, Short Timeout)` for computer-use selection and cache detection paths where quickly falling back is more useful than waiting on a long primary response. ## Per-Action Model Overrides The most powerful feature of model configuration is the ability to **specify a different model for individual actions** directly in your workflow prompts. This works for: * **`focused_action`** — dynamic decisions and observations * **`extract_prompt`** — vision-based data extraction from screenshots ### How to Specify a Model in Your Prompt Use the `model="Model Name"` parameter in your prompt text: ```text theme={null} Take a screenshot with extract_prompt="Extract all invoice data as JSON" and model="Sonnet 4.5" ``` ```text theme={null} Use focused_action with model="Sonnet 4.5 (Thinking)" to find and click on the patient whose name is {patient_name} ``` ### Using the Model Picker in the Prompt Editor The prompt editor provides easy access to the model picker: 1. **Slash menu**: Type `/` and select "Model Override" to insert `model=""` 2. **Tab autocomplete**: Start typing `model` and press Tab to autocomplete 3. **Direct typing**: Type `model=""` and place your cursor inside the quotes Once your cursor is inside the `model=""` quotes, a dropdown appears showing all available models. Use arrow keys to navigate, and press Enter or Tab to select. Choosing **System Default** clears the per-action override. Hover over a model in the dropdown to see its details, including whether it supports computer use, the provider, and configuration parameters. ### Computer Use Models vs. Extraction Models **Important**: For `focused_action`, prefer models that are marked as computer use models. The model picker indicates which models support computer use, and the editor warns if you pick a model that is not marked for computer use. For `extract_prompt` (vision-based extraction), any configured vision-capable model can be used. When you select a non-computer-use model, you'll see a toast warning: > "This model isn't a known computer use model. Only use this for screenshots with extract\_prompt." ### Example: Hybrid Model Strategy Use different models for different parts of your workflow: ```text theme={null} Navigate to the invoice details page. Use focused_action with model="Sonnet 4.5 (Thinking)" to verify the invoice status shows "Approved" before proceeding. Take a screenshot with extract_prompt="Extract all line items as JSON: {item_name, quantity, unit_price, total}" and model="Sonnet 4.5" and process_async="batch" Scroll down and take another screenshot with extract_prompt="Extract payment details and due date" and model="Sonnet 4.5" and process_async="batch" ``` This strategy allows you to: * Use a thinking model for complex decisions in `focused_action` * Use a faster model for bulk extraction with `extract_prompt` * Optimize for both accuracy and cost ## Automatic Fallbacks Cyberdesk automatically handles model failures with a fallback chain: 1. **Primary model** fails (rate limit, timeout, etc.) 2. **Fallback 1** is attempted (currently `Vertex Sonnet 4.5 (Thinking)`) 3. **Fallback 2** is attempted if Fallback 1 also fails (currently `Bedrock Sonnet 4.5 (Thinking)`) This ensures your workflows remain resilient even during provider outages. ## Custom Model Configurations Want to use a specific model, provider, or configuration? Cyberdesk can set up organization-owned model configurations for teams that need a custom provider, model version, API key, or endpoint. Custom model configurations are scoped to your organization. Once configured, they appear in the same workflow selectors and prompt-editor model pickers as Cyberdesk's system defaults. The dashboard currently lets you select them, but not create or edit them in a dedicated UI. ### What You Can Customize * **Provider**: Choose from any supported provider * **Model**: Select specific model versions * **Temperature**: Control response randomness * **Max tokens**: Set output length limits * **Timeout**: Configure request timeouts * **API keys**: Use your own provider API keys for billing and rate limits * **Base URL**: Route to an approved custom provider endpoint when needed For providers Cyberdesk does not supply keys for, include your own API key when creating the configuration. Cyberdesk stores the secret securely and persists only an alias. ### Custom OpenAI-Compatible Providers If your model exposes an OpenAI-compatible API, Cyberdesk can configure it as an organization-scoped OpenAI-compatible model. This is useful for custom fine-tuned models, private gateways, or provider endpoints that use OpenAI-style chat completions. For custom OpenAI-compatible endpoints, Cyberdesk needs: * The provider base URL * The model identifier * Your provider API key * Whether the model should be available for workflow agents, focused actions, extraction, or cache detection Custom OpenAI-compatible base URLs require your own API key. Cyberdesk does not send Cyberdesk-provided OpenAI keys to custom endpoints. ### Need Help Setting One Up? Contact the Cyberdesk team: * **Email**: [founders@cyberdesk.io](mailto:founders@cyberdesk.io) * **Discord**: [Join our community](https://discord.gg/ws5ddx5yZ8) Include details about: * Which provider and model you want to use * Any custom base URL or OpenAI-compatible endpoint * Any specific parameters (temperature, max tokens, etc.) * Whether you'll provide your own API key ## Supported Providers Cyberdesk uses [LangChain's `init_chat_model`](https://docs.langchain.com/oss/python/integrations/providers/overview) under the hood, and the production app currently allows a curated set of providers that match our installed integrations and runtime validation. This includes: Claude models including Sonnet, Opus, and Haiku variants GPT-4, GPT-5, and other OpenAI models Access models through AWS infrastructure Gemini models via Vertex AI or Google AI Azure OpenAI and Azure AI services Groq, Mistral, Cohere, Together, and others For the broader ecosystem of providers and capabilities supported by LangChain itself, see the [LangChain integrations documentation](https://docs.langchain.com/oss/python/integrations/providers/overview). Cyberdesk's exact allowlist is narrower and is enforced by the API when you create custom model configurations. ## Best Practices System defaults are optimized for most use cases. Only customize if you have specific requirements. Use computer-use models for `focused_action`, and consider faster/cheaper models for bulk `extract_prompt` operations. When switching models, test your workflows thoroughly. Different models may behave differently on the same tasks. Track run success rates after model changes. Some models may perform better on specific workflow types. ## Quick Reference | Use Case | Recommended Approach | | ----------------------------------- | ------------------------------------------------------ | | Main workflow agent | Set at workflow level in dashboard | | Dynamic decisions during navigation | `focused_action` with computer-use model | | Vision-based data extraction | `extract_prompt` (any vision model works) | | Bulk extraction for output | `extract_prompt` with `process_async` and faster model | | Cost optimization | Override with cheaper model for extraction tasks | ## FAQ Yes. Custom model configurations can use your own provider API keys. This gives you control over billing and rate limits. Contact the Cyberdesk team to set this up. Note that this will most likely result in a change to your Cyberdesk plan. Cyberdesk monitors provider announcements and updates system defaults accordingly. For custom configurations, we'll notify you in advance and help migrate to newer model versions. Yes! Use the `model="Model Name"` parameter in your prompts to override the model for specific `focused_action` or `extract_prompt` operations. This is the recommended way to optimize for accuracy and cost. `focused_action` works best with models that understand computer use—clicking, typing, and navigating. The model picker shows which models support computer use, and the editor warns if you choose a model that is not marked for it. Non-computer-use models are still fine for `extract_prompt`, which only needs vision/extraction capability. Start with the system defaults. If you need more reasoning power for complex decisions, try a higher-reasoning model. For bulk extraction where speed matters, consider a faster vision-capable model. The model details panel in the picker shows each model's characteristics. # Per-run Model Overrides Source: https://docs.cyberdesk.io/concepts/per-run-model-overrides Choose the main agent model for a single run without mutating the workflow Per-run model overrides let you choose the main agent, or computer-use, model when you create a run. Use this when your application decides which model to use at trigger time. For example, your runner UI might route simple jobs to one model and harder jobs to another, while still using one shared Cyberdesk workflow. ## Why this exists Normally, a run uses the workflow's saved `model_metadata`. That is the right default when the workflow owns model selection. If your system launches many concurrent runs from one workflow, patching the workflow's `model_metadata` before each run can race: * run A should use model A * run B should use model B * both runs use the same workflow * patching the workflow would mutate shared state With a per-run model override, each run carries its own model selection. The workflow's saved model configuration stays unchanged. ## 1. List Available Models First, fetch the model configurations available to your organization: ```bash theme={null} curl "https://api.cyberdesk.io/v1/model-configurations" \ -H "Authorization: Bearer $CYBERDESK_API_KEY" ``` ```typescript theme={null} const { data: modelConfigurations } = await client.model_configurations.list(); ``` ```python theme={null} response = client.model_configurations.list_sync() model_configurations = response.data ``` Each model configuration includes an `id`, `name`, provider details, and flags such as `is_computer_use_model` and `is_archived`. For the main agent, choose a model where: * `is_computer_use_model` is `true` * `is_archived` is `false` * the model matches your cost, latency, and reliability needs Show users the model `name` in your UI, but store and send the model configuration `id` when creating a run. ## 2. Create a Run With a Model Override Use `main_agent_model_id` when you only need to override the main agent model: ```bash theme={null} curl -X POST "https://api.cyberdesk.io/v1/runs" \ -H "Authorization: Bearer $CYBERDESK_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_id": "workflow-uuid", "main_agent_model_id": "model-configuration-uuid", "input_values": { "account_id": "acct_123" } }' ``` ```typescript theme={null} const { data: run } = await client.runs.create({ workflow_id: "workflow-uuid", main_agent_model_id: "model-configuration-uuid", input_values: { account_id: "acct_123", }, }); ``` ```python theme={null} from cyberdesk import RunCreate run = client.runs.create_sync( RunCreate( workflow_id=workflow_id, main_agent_model_id=model_configuration_id, input_values={"account_id": "acct_123"}, ) ) ``` This applies only to the run you create. The workflow's saved `model_metadata` is not changed. ## Full Metadata Shape You can also send `model_metadata` directly. This is useful if you want to override more model roles later while keeping the same payload shape as workflow-level model metadata: ```json theme={null} { "workflow_id": "workflow-uuid", "model_metadata": { "main_agent_model_id": "model-configuration-uuid" } } ``` If both `main_agent_model_id` and `model_metadata.main_agent_model_id` are present, they must match. ## Fallback Behavior Per-run model metadata is merged on top of the workflow's model metadata: * provided run-level fields win * omitted run-level fields fall back to the workflow * if no run-level model override is provided, behavior is unchanged For example, if the workflow has cache detection and fallback models configured, and the run only provides `main_agent_model_id`, the run uses: * run-level `main_agent_model_id` * workflow-level cache detection model * workflow-level fallback models ## Validation Cyberdesk validates model override IDs when the run is created. The request fails with `400` if the model configuration: * does not exist * is not accessible to your organization * conflicts between `main_agent_model_id` and `model_metadata.main_agent_model_id` ## When to Use This Use per-run model overrides when: * your runner UI lets users pick a model at trigger time * different workflow steps or jobs should use different main agent models * you batch many concurrent runs of one workflow * the workflow should remain a stable template instead of mutable runtime state Do not use this when the workflow itself should remain the single source of truth for model selection. In that case, update the workflow's model configuration and create runs without `main_agent_model_id`. ## Related * [Model Configuration](/concepts/model-configuration) * [Per-run Prompt Overrides](/concepts/per-run-prompt-overrides) # Per-run Prompt and Model Overrides Source: https://docs.cyberdesk.io/concepts/per-run-prompt-overrides Pin a run to a specific prompt or main agent model without mutating the workflow Per-run overrides let you pass `main_prompt` or a main agent model override directly when creating a run. Use this when your system owns prompt or model selection and wants a run to execute a specific version without updating the workflow's stored defaults. ## Why this exists Normally, a run executes the workflow's current `main_prompt` and `model_metadata`. That works well when the workflow itself is the source of truth, but it can create races if an external workflow builder stores prompt versions separately: * run A should execute prompt v3 or model A * run B should execute prompt v4 or model B * both runs use the same workflow * patching the workflow before each run would mutate shared state With per-run overrides, each run carries its own prompt text or model metadata. Concurrent runs can use different prompt or model versions without racing on the workflow record. ## How it works `POST /v1/runs` accepts optional `main_prompt`, `main_agent_model_id`, and `model_metadata` fields: * If `main_prompt` is provided, that run executes the supplied prompt text. * If `main_prompt` is omitted or `null`, the run falls back to the workflow's stored `main_prompt`. * If `main_agent_model_id` is provided, that run uses the supplied main agent `ModelConfiguration.id`. * If `model_metadata` is provided, its non-null fields override the workflow's `model_metadata` for that run. * The workflow's stored prompt and model metadata are not changed. * The run response includes `main_prompt` and `model_metadata`; `null` means the run uses workflow-level fallback. Cyberdesk does not store prompt version history for you. Store version history and "latest vs pinned" logic in your system, then send the resolved prompt text when creating a run. ## Example: pinned prompt version ```typescript theme={null} const { data: run } = await client.runs.create({ workflow_id: 'workflow-uuid', main_prompt: 'Prompt v3: open the CRM, find the account, and update the renewal date.', input_values: { account_id: 'acct_123', renewal_date: '2026-07-01' } }); ``` ```python theme={null} run = client.runs.create_sync( RunCreate( workflow_id=workflow_id, main_prompt="Prompt v3: open the CRM, find the account, and update the renewal date.", input_values={ "account_id": "acct_123", "renewal_date": "2026-07-01", }, ) ) ``` ```bash theme={null} curl -X POST "https://api.cyberdesk.io/v1/runs" \ -H "Authorization: Bearer $CYBERDESK_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_id": "workflow-uuid", "main_prompt": "Prompt v3: open the CRM, find the account, and update the renewal date.", "input_values": { "account_id": "acct_123", "renewal_date": "2026-07-01" } }' ``` ## Example: always use latest workflow prompt Omit `main_prompt` to keep the existing behavior: ```json theme={null} { "workflow_id": "workflow-uuid", "input_values": { "account_id": "acct_123" } } ``` This run uses whatever `main_prompt` is currently stored on the workflow when the run executes. ## Example: per-run main agent model Use `main_agent_model_id` when you only need to override the CUA/main agent model: ```json theme={null} { "workflow_id": "workflow-uuid", "main_agent_model_id": "model-configuration-uuid", "input_values": { "account_id": "acct_123" } } ``` You can also provide the full `model_metadata` shape when you need to override more than the main agent model: ```json theme={null} { "workflow_id": "workflow-uuid", "model_metadata": { "main_agent_model_id": "model-configuration-uuid" } } ``` If both `main_agent_model_id` and `model_metadata.main_agent_model_id` are present, they must match. For a full walkthrough of finding available model configuration IDs and choosing computer-use models, see [Per-run Model Overrides](/concepts/per-run-model-overrides). ## Input variables and validation When `main_prompt` is provided on the run, Cyberdesk uses that prompt for prompt-variable handling on that run. For example, if the override references `{account_id}`, that variable is treated as part of the run's prompt even if the workflow's stored prompt does not currently reference it. If the workflow has an `input_schema`, the run's inputs are still validated against the workflow schema. Overrides only change runtime prompt/model selection for that run; they do not create or change the workflow schema. ## Interaction with Post-run Checks Per-run prompt and model overrides do not change the workflow-level Post-run Check configuration. One important guard stays tied to the stored workflow prompt: a run can only skip Post-run Checks through `declare_task_succeeded` with `skip_post_run_checks=True` when the workflow's stored prompt explicitly authorizes that behavior. Adding those words only in a per-run override does not authorize skipping checks. ## When to use this Use per-run overrides when: * you keep prompt versions in your own workflow builder * you want a run pinned to a specific prompt version * you choose the main agent model at trigger time * you batch concurrent runs of the same workflow with different prompt or model versions * you want the workflow's stored prompt to remain a mirror or default, not the runtime source of truth for every run Do not use this when the workflow itself should remain the only editable source of prompt and model truth. In that case, update the workflow and create runs without per-run overrides. # Pool Parameters Source: https://docs.cyberdesk.io/concepts/pool-parameters Shared workflow input defaults that apply when a run explicitly selects one or more desktop pools Pool parameters are shared workflow input defaults stored on a desktop pool. When a run explicitly selects that pool, Cyberdesk adds the pool's parameters to the effective inputs used for validation and execution. Use them for values that are true for every desktop in a pool: * A tenant, workspace, or customer account ID * A regional URL or environment name * A shared folder path * Shared credentials or API keys used by that pool Pool parameters are not desktop display settings. They are workflow inputs that can satisfy prompt variables such as `{region}` or sensitive prompt variables such as `{$shared_api_key}`. ## Mental Model Think of pool parameters as the bottom layer of the run input stack: 1. Pool parameters fill in shared defaults when a selected pool provides them. 2. Desktop parameters override those defaults after Cyberdesk knows which desktop will run the workflow. 3. Run inputs override both when you pass an ad hoc value for a single run. That means pool parameters are a good place for stable, shared context. They are not a good place for per-customer run data, one-off values, or anything that should change from run to run. ## When They Apply Pool parameters apply only when the run explicitly selects a pool through `pool_ids` or the dashboard's pool-based machine selection. They do not apply just because the assigned desktop belongs to a pool. If you choose a specific desktop for the run, Cyberdesk uses that desktop's parameters, but it does not automatically pull parameters from every pool that desktop belongs to. This rule keeps runs predictable. A desktop can belong to multiple pools for routing, organization, or access control without accidentally changing workflow inputs. ## Basic Example Suppose a pool named `West Coast Claims` has these pool parameters: ```json theme={null} { "region": "us-west", "claims_portal_url": "https://claims-west.example.com", "tenant_id": "tenant_west_123" } ``` Your workflow prompt can reference them like normal inputs: ```text theme={null} Open {claims_portal_url}, log in to tenant {tenant_id}, and process the claim in the {region} region. ``` When you start a run and select the `West Coast Claims` pool, Cyberdesk includes those values in the effective input payload. You do not need to pass them again as run inputs unless you want to override them for that run. ## Priority And Overrides When the same key exists in more than one place, Cyberdesk uses this priority order: 1. Run input or sensitive run input 2. Desktop parameter or sensitive desktop parameter on the assigned desktop 3. Pool parameter or sensitive pool parameter from the selected pool For example: ```json theme={null} { "selected_pool.pool_parameters.region": "us-west", "assigned_desktop.machine_parameters.region": "us-west-2", "run.input_values.region": "sandbox" } ``` The workflow sees `sandbox`, because run input wins. If the run omits `region`, the workflow sees `us-west-2`. If the assigned desktop also omits `region`, the workflow sees `us-west`. An explicit run-level `__EMPTY__` value is still an override. Omitted, null, or blank values allow lower-priority defaults to fill in. To actively clear a populated field during desktop entry, use [`__CLEAR__`](/concepts/clear-values). ## Plain And Sensitive Values Pool parameters come in two forms: * Plain parameters are stored as normal structured values and referenced as `{key}`. * Sensitive parameters are stored securely and referenced as `{$key}`. For sensitive parameters, configure the key without the `$` prefix: ```json theme={null} { "shared_api_key": "actual-secret-value" } ``` Reference it in the workflow with the `$` prefix: ```text theme={null} Authenticate with {$shared_api_key}. ``` The same convention applies to input schemas. Sensitive values are exposed to schema validation with `$`-prefixed root keys, such as `$shared_api_key`. Plain pool parameters can be strings, numbers, booleans, objects, or arrays. For nested prompt access like `{customer.portal.url}`, see [Structured Inputs](/concepts/structured-inputs). ## Multiple Selected Pools A run can select more than one pool. Cyberdesk merges parameters from all selected pools before the run starts. If two selected pools define the same parameter key, Cyberdesk rejects the run instead of choosing one silently: ```text theme={null} Selected pools define duplicate parameter keys: region: Pool A (...), Pool B (...) ``` Rename or remove one of the duplicate keys before starting the run. This applies to both plain and sensitive pool parameter keys. ## Validation Behavior When a run selects pools, Cyberdesk loads those pool parameters during create-time validation. This means required workflow inputs can be satisfied by selected pool parameters before a desktop has been assigned. If the final desktop is not known yet, Cyberdesk still validates the concrete values it already has: * Run inputs * Sensitive run inputs * Selected pool parameters * Selected sensitive pool parameters Required inputs that may come from desktop parameters can be deferred until execution time, after Cyberdesk assigns a desktop and can merge that desktop's values. Validation runs again at execution time with the full effective payload. That final payload includes selected pool defaults, assigned desktop defaults, run inputs, and resolved sensitive values. ## Dashboard Setup 1. Go to the dashboard and open the Desktops area. 2. Click **View Pools**. 3. Create or edit a pool. 4. Add plain or sensitive pool parameters in the pool parameter editor. 5. Start a run and choose pool-based machine selection. 6. Select the pool whose defaults should apply. If you start a run by choosing a specific desktop instead, pool parameters do not apply. Move shared values to [Desktop Parameters](/concepts/desktop-parameters) for that desktop, or pass them as run inputs. ## SDK Setup ```typescript TypeScript theme={null} import { createCyberdeskClient } from 'cyberdesk'; const client = createCyberdeskClient('your-api-key'); await client.pools.update('pool-id', { pool_parameters: { region: 'us-west', claims_portal_url: 'https://claims-west.example.com', tenant_id: 'tenant_west_123' }, pool_sensitive_parameters: { shared_api_key: 'actual-secret-value' } }); ``` ```python Python theme={null} from cyberdesk import CyberdeskClient, PoolUpdate client = CyberdeskClient(api_key="your-api-key") client.pools.update_sync( pool_id="pool-id", data=PoolUpdate( pool_parameters={ "region": "us-west", "claims_portal_url": "https://claims-west.example.com", "tenant_id": "tenant_west_123", }, pool_sensitive_parameters={ "shared_api_key": "actual-secret-value", }, ), ) ``` Then create a run with `pool_ids` instead of a specific `machine_id`: ```typescript TypeScript theme={null} await client.runs.create({ workflow_id: 'workflow-id', pool_ids: ['pool-id'], input_values: { claim_id: 'CLM-123' } }); ``` ```python Python theme={null} from cyberdesk import RunCreate client.runs.create_sync( RunCreate( workflow_id="workflow-id", pool_ids=["pool-id"], input_values={ "claim_id": "CLM-123", }, ), ) ``` ## Choosing The Right Parameter Type Use pool parameters when a value is shared across every desktop in a selected pool. Use desktop parameters when a value depends on the assigned desktop, such as machine-specific credentials, local file paths, or desktop-specific account IDs. Use run inputs when a value belongs to a specific run, such as a claim number, invoice ID, customer name, or any value that should be supplied by the caller. ## Common Gotchas * Pool membership alone does not apply pool parameters; the run must explicitly select the pool. * Selecting a specific desktop bypasses pool parameters. * Duplicate keys across selected pools fail fast. * Run inputs override pool parameters, even when the pool value exists. * Sensitive parameter keys are configured without `$` and referenced with `$` in prompts and schemas. # Post-run Checks Source: https://docs.cyberdesk.io/concepts/post-run-checks Verify exported files, screenshots, and structured output after workflow execution finishes ## Overview Post-run Checks let you add workflow-level verification that runs **after** the main automation steps finish. They are useful when "the workflow clicked through the UI" is not enough and you also want Cyberdesk to confirm that the run actually produced: * the file you expected * the screenshot you expected * the structured output you expected * the semantic result you expected Think of Post-run Checks as a confidence-boosting double-check, not as a replacement for good workflow instructions. The workflow still does the work. Post-run Checks verify the outcome. ## Why They Exist In many workflows, completion is not the same thing as correctness. Examples: * A report download workflow may finish, but the PDF might never have been exported. * A form-submission workflow may end on a confirmation page, but you may want a saved screenshot as proof. * A data-extraction workflow may produce `output_data`, but you may still want to verify that the contents are complete and sensible. Post-run Checks close that gap. ## Where They Live Post-run Checks are defined on the **workflow**. That means: * you configure them once on the workflow * every new run of that workflow can inherit them * the run stores a snapshot of the effective checks it started with Changing a workflow later does **not** rewrite historical run results. Existing runs keep the post-run check snapshot they started with. ## When They Run The lifecycle is: 1. The main workflow execution runs first. 2. If the run has eligible Post-run Checks, the run enters `running_checks`, which appears in the UI as **Running Checks**. 3. Cyberdesk executes the checks after the main run finishes. 4. The final terminal status is decided only after those checks complete. For image checks that depend on run attachments, Cyberdesk may start the model verification in the background as soon as a matching attachment is saved during the run. This is only a speed optimization: the run still appears as **Running** during the main automation, switches to **Running Checks** only after the main execution finishes, and waits for any in-flight check work before deciding the final status. `run_complete` and other terminal-completion semantics now mean: **the main execution finished, and any Post-run Checks finished too**. ### Early Success Skips Sometimes a workflow intentionally exits early from a `focused_action` because the task is already complete. In that specific path, the usual Post-run Check evidence may not exist yet: screenshots may not have been saved, files may not have been exported, and `output_data` may be incomplete. If your focused action is explicitly allowed to call [`declare_task_succeeded`](/workflow-prompting/declare-task-succeeded), you can tell it to skip Post-run Checks for that early-success path: ```text theme={null} Use focused_action to check whether the invoice is already marked Paid. If it is already Paid, call declare_task_succeeded with skip_post_run_checks=True and explain that no further action was needed. ``` When `skip_post_run_checks=True` is used, Cyberdesk does not run the configured Post-run Checks. Instead, each check snapshot is marked `success` with a message explaining that it was skipped because the focused action declared early success with Post-run Checks disabled. Cyberdesk also skips trajectory generation for that early-success branch because it is an idempotent shortcut, not the workflow's normal reusable path. ### Standalone Runs vs Sessions and Chains Cyberdesk handles machine/session ownership differently depending on the run shape: * **Standalone runs** usually release the machine before Post-run Checks begin. * **Session and chain runs** can keep the machine/session claimed until Post-run Checks finish. This matters if you are watching machine lifecycle, session closure, or downstream automation that expects a machine to be released only after the whole run is truly done. ## What Post-run Checks Are Not Post-run Checks are **not** a complete definition of workflow success. If you need to define a requirement where, if a certain condition isn’t met, an agent should actively try to fulfill it, define the successful action in your prompt and use a focused action to verify it, possibly setting a runtime value boolean (and output schema) to indicate success. Learn more at [`focused_action`](/workflow-prompting/focused-action) and [Generating Output Data](/concepts/generating-output-data). Our team is working on a complete form of success criteria, which will include during and post-run checks. ## The Four Initial Check Types Cyberdesk currently supports four initial Post-run Check types. ### 1. Attachment Exists This check verifies that one or more expected run attachments exist. Use it for: * exported PDFs * CSVs * generated files * saved screenshots * one-file-per-item loop workflows This is the simplest and most deterministic Post-run Check. ### 2. Image Check This check asks an AI model to inspect one or more image attachments against a natural-language rule. Use it for: * confirmation screenshots * chart screenshots * receipts * dashboards * visual QA Examples: * "Verify the screenshot shows a successful payment confirmation and a visible confirmation number." * "Verify the saved chart shows the last 30 days and includes all four vital signs." If you plan to verify screenshots visually, first create those screenshots as run attachments with [`save_screenshot_as_run_attachment`](/workflow-prompting/save-screenshot). ### 3. Output Data Passes Schema Validation This is the **auto-managed** Post-run Check. If your workflow has an output schema, Cyberdesk automatically keeps this check in sync. You do not manually author it as a separate custom check. It verifies that the final structured `output_data` conforms to the workflow's output schema. This is the best way to ensure: * required fields exist * data shape is valid * types line up with your schema Learn more about output transformation and schemas in [Generating Output Data](/concepts/generating-output-data). ### 4. Output Data Check This check asks an AI model to evaluate the final structured `output_data` against a natural-language rule. Use it when schema validation alone is not enough. Examples: * "Verify that every extracted invoice line item has a positive amount." * "Verify the structured output clearly identifies a patient MRN and appointment date." * "Verify the extracted order looks complete and not partially parsed." Output-data agentic checks are most useful when you care about **semantic correctness**, not just shape correctness. ## Attachment Targeting Modes The two attachment-based checks, **Attachment Exists** and **Image Check**, support three targeting modes. ### Exact Filenames Use this when you know the exact attachment names ahead of time. Examples: * `invoice.pdf` * `confirmation.png` * `daily_report.csv` Best for: * stable filenames * deterministic exports * named screenshots #### Exact Mode Behavior * You provide one or more filenames. * Filenames should include file extensions. * At runtime, Cyberdesk looks for those exact attachment filenames. * If a required attachment is missing, the check fails. In the workflow editor, Cyberdesk may suggest exact filenames by scanning your prompt for known attachment-producing patterns such as [`save_screenshot_as_run_attachment`](/workflow-prompting/save-screenshot) and [`mark_file_for_export`](/workflow-prompting/mark-file-for-export). These are suggestions only. You can always add or edit filenames manually. ### Regex Use this when the filename changes between runs, but follows a consistent pattern. Examples: * `^invoice_.*\.pdf$` * `^receipt_[0-9]{8}\.png$` * `^export_.*\.csv$` Best for: * timestamps * generated IDs * date-based filenames * dynamic naming conventions #### Regex Mode Behavior * Matching is done against the attachment **filename**. * Matching is **full-string** and **case-sensitive**. * Zero matches fail the check. * You can optionally specify an expected match count. Expected match count can come from: * a literal integer * an input or sensitive reference that resolves to an integer * an input or sensitive reference that resolves to an array, in which case Cyberdesk uses the array length ### Loop Items Use this when the workflow should produce **one attachment per loop item**. Examples: * `receipt_{{loop_item.name}}.png` * `claim_{{loop_item.claim_id}}.pdf` * `summary_{{loop_item}}.csv` Best for: * one-file-per-item workflows * iterating over arrays * exporting a file for each selected record #### Loop Items Mode Behavior You provide: * `loop_input` * `loop_item_filename_template` * optionally, **Optional loop attachments** Cyberdesk resolves the expected filenames by applying the template once per item in the loop input. The `loop_input` semantics match the looping system: * a JSON array means "iterate over this array" * an integer `n` means "iterate over items `0..n-1`" If the loop input resolves to an unsupported value, the check fails with a configuration/data error. When **Optional loop attachments** is enabled, missing attachments for individual loop items are skipped instead of failing the check. Use this when a loop only sometimes saves an attachment for an item. The check still evaluates any matching attachments it does find. If every loop item is skipped because no matching attachments exist, the check succeeds and records a message explaining that all items were skipped. Learn more about loop semantics and `{{loop_item}}` in [Looping Tools](/workflow-prompting/looping-tools). Need literal template-style text in a Post-run Check prompt or filename template? Escape the opening delimiter: * `\{customer_name}` renders as literal `{customer_name}` * `\{{current_status}}` renders as literal `{{current_status}}` * `\{$support_pin}` renders as literal `{$support_pin}` This works anywhere Post-run Checks accept templated text, including check prompts and filename templates. ## Producing Attachments That Checks Can Validate Post-run Checks work on **run attachments**, not arbitrary machine state. That means your workflow should intentionally produce the attachments you want to verify. ### For screenshots Use [`save_screenshot_as_run_attachment`](/workflow-prompting/save-screenshot). Good for: * confirmation pages * receipts * dashboards * visual evidence ### For files created or downloaded on the machine Use [`mark_file_for_export`](/workflow-prompting/mark-file-for-export). Good for: * PDFs * CSVs * generated documents * downloaded exports If you plan to add an attachment-based Post-run Check, it is usually worth naming your saved screenshot or exported file clearly and consistently. Predictable filenames make exact targeting much easier than regex. ## Writing Good Check Prompts The AI-based checks are: * **Image Check** * **Output Data Check** The best prompts are narrow and outcome-focused. ### Good prompt characteristics * State the pass condition clearly. * Mention the evidence that matters. * Say what should count as failure. * Avoid asking the model to judge unrelated parts of the workflow. ### Better Image Check Prompt ```text theme={null} Verify the screenshot shows a successful submission confirmation. The page should include a visible confirmation number and should not show any validation or error messages. ``` ### Better Output Data Check Prompt ```text theme={null} Verify the output data looks complete. Every invoice line item should have a description, quantity, unit price, and positive total amount. ``` ### Weaker Prompts to Avoid ```text theme={null} Check that this looks good. ``` ```text theme={null} Make sure nothing is wrong. ``` These are too vague and produce less reliable results. ## How Statuses Work When Post-run Checks exist, the run may enter **Running Checks** after the main workflow steps are done. That means: * the automation steps may be finished * the run is **not terminal yet** * Cyberdesk is still verifying the outcome ### Final status behavior If the main execution succeeded: * all checks pass → final run status is `success` * one or more checks fail → final run ends on a failure path * a check hits an infrastructure-level problem → final run ends on `error` If the main execution already ended in `error` or `task_failed`: * Cyberdesk can still record Post-run Check results for observability * the final run status stays on that failure path If the run is cancelled: * cancelled before checks start → checks are skipped * cancelled during Running Checks → unfinished checks are cancelled Some deployments map failed checks to `task_failed`, while others map them to `error` for compatibility with existing integrations. Either way, the run did **not** pass its verification criteria. ### Cleanup prompts after check failure Workflows can include one optional, workflow-level **Post-run Check failure cleanup prompt**. Cyberdesk only runs this prompt when one or more Post-run Checks fail during a session or chained run, after the post-run check results are known and before the remaining queued runs are cancelled. Use cleanup prompts for recovery steps that should happen when any verification failure would otherwise leave downstream session steps in a bad state, such as closing a modal, undoing a partially submitted record, logging out, or returning the app to a safe page. Leave it blank for standalone runs or workflows where no machine-side cleanup is needed. When cleanup runs, Cyberdesk appends a marker to `run_message_history` with `event_type: "post_run_check_failure_cleanup_start"`. The run details page uses this marker to jump from the Post-run Checks section and timeline navigation to the independent cleanup agent's message history. Screenshots captured after the marker are labeled as cleanup screenshots in the filmstrip viewer. ## Accessing Results in Code Post-run Check results live on `run.post_run_checks`. You can access them from: * `get_run` * `run_complete` webhook payloads * `list_runs` when you request `fields=post_run_checks` Important details: * `run.post_run_checks` is an **array**, not an object keyed by check name * each item includes `name`, `status`, `error_message`, `messages`, and `matched_filenames` * if you want to look checks up by name, keep those names stable and unique within the workflow * the best place to react automatically is the `run_complete` webhook, because it fires only after Post-run Checks finish ```ts theme={null} const checks = run.post_run_checks ?? [] const checksByName = new Map( checks.filter((check) => check.name).map((check) => [check.name!, check]), ) const invoiceCheck = checksByName.get("Invoice PDF exists") if (invoiceCheck && invoiceCheck.status !== "success") { console.log("status:", invoiceCheck.status) console.log("error:", invoiceCheck.error_message) console.log("messages:", invoiceCheck.messages) console.log("matched files:", invoiceCheck.matched_filenames) } ``` ```python theme={null} checks = run.post_run_checks or [] checks_by_name = {check.name: check for check in checks if check.name} invoice_check = checks_by_name.get("Invoice PDF exists") status = getattr(invoice_check.status, "value", invoice_check.status) if invoice_check else None if invoice_check and status != "success": print("status:", status) print("error:", invoice_check.error_message) print("messages:", invoice_check.messages) print("matched files:", invoice_check.matched_filenames) ``` If you are polling runs via `list_runs`, remember to request `post_run_checks` explicitly in the `fields` list. `get_run` and `run_complete` already include it. ## Example End-to-End Pattern Imagine a workflow that: 1. downloads an invoice PDF 2. saves a confirmation screenshot 3. extracts structured invoice data You might configure: * **Attachment Exists** * exact filename: `invoice.pdf` * **Image Check** * exact filename: `invoice_confirmation.png` * prompt: "Verify the screenshot shows a successful invoice export with no visible errors." * **Output Data Passes Schema Validation** * auto-managed because the workflow has an output schema * **Output Data Check** * prompt: "Verify the structured output contains invoice number, invoice date, vendor name, and a positive total." This gives you both: * deterministic checks on concrete artifacts * semantic checks on the final output ## Best Practices 1. Prefer **exact filenames** when names are stable. 2. Use **regex** only when the variability is real and predictable. 3. Use **loop\_items** when the workflow intentionally produces one attachment per item, and enable optional loop attachments only when some loop items legitimately produce no attachment. 4. Pair **schema validation** with **output data checks** when you care about both shape and meaning. 5. Keep AI check prompts narrow and specific. 6. Add a workflow-level cleanup prompt in sessions or chains when any failed verification needs machine-side cleanup before the session ends. 7. Treat Post-run Checks as verification, not as a substitute for good workflow instructions. 8. Review check results in run details when tuning a workflow. ## Related Docs * [Save Screenshot as Run Attachment](/workflow-prompting/save-screenshot)\ Learn how to create screenshot attachments that image checks can evaluate. * [Mark File for Export](/workflow-prompting/mark-file-for-export)\ Learn how to export files from the machine as run attachments. * [Looping Tools](/workflow-prompting/looping-tools)\ Learn how `start_loop`, `{{loop_item}}`, and loop inputs work. * [Generating Output Data](/concepts/generating-output-data)\ Learn how output schemas and `output_data` are produced and validated. * [Declare Task Succeeded](/workflow-prompting/declare-task-succeeded)\ Learn how early success detection during main execution differs from post-run verification. # Priority Runs Source: https://docs.cyberdesk.io/concepts/priority-runs Schedule urgent runs ahead of normal queued work without interrupting active runs or session order Priority runs let you mark urgent automation work so Cyberdesk considers it before normal queued runs when a compatible machine becomes available. Priority is a queue-selection preference, not preemption. It never interrupts a run that is already executing. ## How priority scheduling works When Cyberdesk assigns queued runs to machines, it evaluates currently eligible priority runs before normal runs: 1. Priority runs with a specific machine requirement 2. Priority runs that can use any eligible machine 3. Normal runs with a specific machine requirement 4. Normal runs that can use any eligible machine Within each group, runs retain their existing first-in, first-out order based on creation time. Priority only changes the order of eligible queued work. Machine availability, pool membership, connection state, and other matching requirements still apply. ## Sessions and chains keep their order Priority does not let a later run skip an earlier run in the same session or chain. If step two is marked as priority while step one is still queued, step two remains blocked until step one is assigned and completed according to the normal session sequence. For chain creation, one `is_priority` value applies to every step in the chain. The steps still execute in their declared order. ## Create a priority run Set `is_priority` to `true` when creating a run. The field defaults to `false`. In the new run dialog, select **Priority run**. The same checkbox is available for single runs, bulk runs, and chains. Priority runs are identified in the runs table and on the run detail page. ```bash theme={null} curl -X POST "https://api.cyberdesk.io/v1/runs" \ -H "Authorization: Bearer $CYBERDESK_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_id": "workflow-uuid", "is_priority": true }' ``` ```typescript theme={null} const { data: run, error } = await client.runs.create({ workflow_id: 'workflow-uuid', is_priority: true }); ``` ```python theme={null} from cyberdesk import RunCreate response = client.runs.create_sync( RunCreate( workflow_id="workflow-uuid", is_priority=True, ) ) ``` ## Bulk runs and chains Bulk creation applies the value to every run in the request: ```typescript theme={null} await client.runs.bulkCreate({ workflow_id: 'workflow-uuid', count: 25, is_priority: true }); ``` Chain creation applies the value to every step: ```typescript theme={null} await client.runs.chain({ is_priority: true, steps: [ { workflow_id: 'workflow-step-one' }, { workflow_id: 'workflow-step-two' } ] }); ``` ## Retries An in-place retry preserves the run's current priority by default. You can also promote or demote the run for its next scheduling attempt by including `is_priority` in the retry request. ```typescript theme={null} await client.runs.retry('run-uuid', { is_priority: true }); ``` The dashboard preselects the run's current value in the retry dialog. Change **Priority run** before submitting to override it without creating a new run ID. ## Capacity planning Priority runs are always considered before normal eligible runs. A sustained stream of priority work can therefore delay normal queued runs. Use priority for work with a genuine scheduling need, such as time-sensitive customer operations, incident response, or deadline-bound processing. If most work is marked priority, the queue behaves much like a normal queue and regular runs may wait longer than expected. # Run Screenshots Source: https://docs.cyberdesk.io/concepts/run-screenshots Fetch signed URLs for screenshots referenced in run message history ## Overview Completed runs can include screenshots in `run_message_history`. These screenshots are stored as stable `supabase://run-images/...` references, not public HTTPS URLs: ```json theme={null} { "type": "image", "image_url": "supabase://run-images/message-history/org_123/1762547216743_5002.png" } ``` Use the run screenshot signed URL endpoint when you need to programmatically view, download, archive, or audit those images outside the Cyberdesk dashboard. ## Why Signed URLs? Run screenshots are private organization data. The `supabase://` reference is stable and safe to store, but it is not directly fetchable. A signed URL is a temporary HTTPS URL that grants read access to one screenshot. Signed URLs expire after the requested `expires_in` value. The maximum is 3600 seconds. ## Authorization Model The endpoint is scoped to both the run and the image reference: * Your API key must belong to the run's organization. * The `image_url` must be an exact `supabase://run-images/...` URL present in that run's `run_message_history`. * Cyberdesk verifies the storage path belongs to the same organization before returning a signed URL. This means you can safely pass the screenshot references you read from a run response without exposing other organizations' images. ## Endpoint ```http theme={null} GET /v1/runs/{run_id}/image/signed-url?image_url={image_url}&expires_in=3600 Authorization: Bearer cd_... ``` Example response: ```json theme={null} { "supabase_url": "supabase://run-images/message-history/org_123/1762547216743_5002.png", "signed_url": "https://...", "expires_in": 3600 } ``` ## Curl Example ```bash theme={null} curl -G "https://api.cyberdesk.io/v1/runs/$RUN_ID/image/signed-url" \ -H "Authorization: Bearer $CYBERDESK_API_KEY" \ --data-urlencode "image_url=supabase://run-images/message-history/org_123/1762547216743_5002.png" \ --data-urlencode "expires_in=3600" ``` ## TypeScript Example ```typescript theme={null} const { data: run, error } = await client.runs.get("run-id"); if (error || !run?.run_message_history) { throw error ?? new Error("Run has no message history"); } const imageUrl = run.run_message_history .flatMap((message) => Array.isArray(message.content) ? message.content : []) .find((block) => block?.type === "image" && typeof block.image_url === "string") ?.image_url; if (!imageUrl) { throw new Error("No screenshot found in run message history"); } const { data: signed } = await client.runs.getImageSignedUrl("run-id", imageUrl, { expires_in: 3600, }); console.log(signed?.signed_url); ``` ## Python Example ```python theme={null} response = await client.runs.get("run-id") if response.error or not response.data or not response.data.run_message_history: raise response.error or RuntimeError("Run has no message history") image_url = None for message in response.data.run_message_history: content = message.get("content", []) if not isinstance(content, list): continue for block in content: if block.get("type") == "image" and isinstance(block.get("image_url"), str): image_url = block["image_url"] break if image_url: break if not image_url: raise RuntimeError("No screenshot found in run message history") signed = await client.runs.get_image_signed_url( "run-id", image_url=image_url, expires_in=3600, ) print(signed.data.signed_url) ``` ## Common Errors * `400 Invalid Supabase URL`: The `image_url` is not a valid `supabase://...` reference. * `400 Only run-images URLs are supported`: The URL points to a different storage bucket. * `403 Image URL is not present in this run's message history`: The URL was not read from the specified run. * `403 Image URL does not belong to the organization`: The storage path does not match the authenticated organization. * `404 Run not found`: The run ID does not exist in the authenticated organization. # Sessions and Chains Source: https://docs.cyberdesk.io/concepts/sessions-and-chains Reserve machines for multi-step workflows with exclusive access At its core, a **session** is a reservation of a single machine. While a session is active, that machine is dedicated to your session only — no unrelated runs will be scheduled onto it. This guarantees your multi-step automations run back-to-back on the same desktop without interference. ## What you get from a session * **Exclusive access** to one machine for the session's duration (strong scheduling guarantee) * **Deterministic sequencing**: "step 1 → step 2 → …" behavior with no opportunistic interleaving * **Shared state**: Files and desktop state persist across runs in the same session ## When to use sessions Sessions are essential when your automation requires multiple steps that must happen on the same machine without interruption: * **EHR workflows**: Log into Epic, navigate to a specific patient, extract their data, then upload documents to their chart — all with no interruptions from other runs * **Financial reporting**: Export monthly reports from your ERP system, transform the data in Excel, then re-import the processed results * **Document processing**: Download files from a web portal, process them with a local application, then upload the results back * **Any multi-step workflow**: Where state on the desktop (open applications, logged-in sessions, temporary files) must persist between steps ## Chains: The easiest way to use sessions **Chains** are a convenient way to create multiple runs that execute back-to-back in the same session. Instead of manually creating individual runs and managing their sequencing, you can define all your workflow steps upfront and let Cyberdesk handle the session management and execution order. ### Start a new session with a chain ```typescript theme={null} import { createCyberdeskClient, type WorkflowChainCreate } from 'cyberdesk' const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!) const chain: WorkflowChainCreate = { // Optional shared inputs applied to steps whose workflows declare those variables shared_inputs: { search_query: 'red panda facts' }, // Optional shared sensitive inputs available to all steps shared_sensitive_inputs: { api_key: 'shared-secret-key' }, // Attach files once at the beginning of the chain (applied to the first run) shared_file_inputs: [ // { filename: 'seed.txt', content: 'base64-...' } ], // Reserve a machine for the whole chain (either machine_id OR pool_ids) pool_ids: ['pool-with-chrome', 'customer-a'], keep_session_after_completion: false, steps: [ { workflow_id: 'step-1-workflow-id', session_alias: 'step1', inputs: { topic: 'red panda', }, sensitive_inputs: { username: 'user1', password: 'secret123' } }, { workflow_id: 'step-2-workflow-id', session_alias: 'step2', inputs: { // Use output of step1 as an input to step2 search_query: { $ref: 'step1.outputs.result' } }, sensitive_inputs: { security_token: 'step2-token' } } ] } const { data: chainResult, error } = await client.runs.chain(chain) if (error) throw new Error(String(error)) console.log('Session:', chainResult.session_id) console.log('Run IDs:', chainResult.run_ids) ``` ```python theme={null} from cyberdesk import CyberdeskClient, WorkflowChainCreate import os client = CyberdeskClient(os.environ['CYBERDESK_API_KEY']) chain = WorkflowChainCreate( # Optional shared inputs applied to steps whose workflows declare those variables shared_inputs={ "search_query": "red panda facts" }, # Optional shared sensitive inputs available to all steps shared_sensitive_inputs={ "api_key": "shared-secret-key" }, # Filter machines by pools; or use machine_id to target one machine pool_ids=["pool-with-chrome", "customer-a"], keep_session_after_completion=False, steps=[ { "workflow_id": "step-1-workflow-id", "session_alias": "step1", "inputs": { "topic": "red panda" }, # Step-specific sensitive inputs "sensitive_inputs": { "username": "user1", "password": "secret123" } }, { "workflow_id": "step-2-workflow-id", "session_alias": "step2", "inputs": { # Use output of step1 as an input to step2 "search_query": {"$ref": "step1.outputs.result"} }, "sensitive_inputs": { "security_token": "step2-token" } } ] ) resp = client.runs.chain_sync(chain) print("Session:", resp.data.session_id) print("Run IDs:", resp.data.run_ids) ``` **Key points:** * Provide `machine_id` to target a specific machine, or `pool_ids` to match any machine in **all** specified pools (intersection) * The chain always runs on one reserved session. If you omit `session_id`, the API creates one and reserves a machine before step 1 starts * `shared_inputs` are automatically filtered per workflow so each step only receives the variables it actually declares * `shared_sensitive_inputs` are available to all steps, while `sensitive_inputs` in individual steps provide step-specific sensitive values * `shared_file_inputs` are attached to the first run in the chain ## Passing data between steps with refs Once you have multiple workflows running in the same session, you'll often want to pass outputs from earlier steps as inputs to later ones. **Refs** make this seamless — simply reference a previous step's output: ```typescript theme={null} { "$ref": "step1.outputs.result" } ``` ```python theme={null} {"$ref": "step1.outputs.result"} ``` The path on the right points to a prior step's output field. Refs are resolved server-side within a session, so you don't need to manually poll and extract values. ### How refs interact with input schema validation If downstream workflows define `input_schema`, Cyberdesk validates ref usage before execution: * For refs that target earlier steps in the same chain request, Cyberdesk checks compatibility against the producing step's `output_schema`. * For refs that target runs that already exist in the session, Cyberdesk validates alias/path/type compatibility up front using available source metadata. * If referenced runs are queued/scheduling and don't have output yet, creation is allowed as long as compatibility checks pass. At execution time, ref resolution is strict: if an upstream optional field is missing in `output_data`, downstream required inputs will fail when the run executes. For full details on validation timing, error shapes, and `$`-prefixed sensitive keys, see [Input Validation](/concepts/input-validation). ### Nested path access You can access nested fields and array elements in refs: ```typescript theme={null} // Access a nested field { "$ref": "step1.outputs.result.query" } // Access an array element { "$ref": "step1.outputs.items[0].name" } // Combine both { "$ref": "step1.outputs.data.customers[0].email" } ``` ```python theme={null} # Access a nested field {"$ref": "step1.outputs.result.query"} # Access an array element {"$ref": "step1.outputs.items[0].name"} # Combine both {"$ref": "step1.outputs.data.customers[0].email"} ``` ### Refs inside structured inputs Refs can be used anywhere inside structured input objects. This allows you to build complex inputs by combining refs with literal values: ```typescript theme={null} steps: [ { workflow_id: 'step-1-workflow-id', session_alias: 'step1', inputs: { search_term: 'capybara' } }, { workflow_id: 'step-2-workflow-id', session_alias: 'step2', inputs: { // Build a structured input from multiple refs data: { query: { $ref: 'step1.outputs.result.query' }, context: { summary: { $ref: 'step1.outputs.result.summary' }, priority: 'high' // Mix refs with literal values } } } } ] ``` ```python theme={null} steps=[ { "workflow_id": "step-1-workflow-id", "session_alias": "step1", "inputs": {"search_term": "capybara"} }, { "workflow_id": "step-2-workflow-id", "session_alias": "step2", "inputs": { # Build a structured input from multiple refs "data": { "query": {"$ref": "step1.outputs.result.query"}, "context": { "summary": {"$ref": "step1.outputs.result.summary"}, "priority": "high" # Mix refs with literal values } } } } ] ``` For more on structured inputs and nested access in workflow prompts, see [Structured Inputs](/concepts/structured-inputs). ## Join an existing session If you already have a reserved session (e.g., created by a prior chain), you can reuse it: ```typescript theme={null} const { data: chainResult } = await client.runs.chain({ session_id: 'existing-session-uuid', steps: [ { workflow_id: 'wf-a', session_alias: 'warmup' }, { workflow_id: 'wf-b', session_alias: 'extract', inputs: { query: 'current patient' } }, ] }) ``` ```python theme={null} chain = WorkflowChainCreate( session_id="existing-session-uuid", steps=[ {"workflow_id": "wf-a", "session_alias": "warmup"}, {"workflow_id": "wf-b", "session_alias": "extract", "inputs": {"query": "current patient"}}, ] ) client.runs.chain_sync(chain) ``` This keeps the same reserved machine and any state/files already present on it. For chain creation, provide either `session_id` or `machine_id`/`pool_ids`, not both. ## Keep the session alive after the chain If you want to leave the reservation active for a follow-up chain or ad-hoc steps: ```typescript theme={null} await client.runs.chain({ pool_ids: ['customer-a'], keep_session_after_completion: true, steps: [ /* ... */ ] }) ``` ```python theme={null} client.runs.chain_sync(WorkflowChainCreate( pool_ids=["customer-a"], keep_session_after_completion=True, steps=[ ... ] )) ``` Later, you can start a new chain with that `session_id` to continue from where you left off. ## Ad-hoc sessions without a chain You don't have to use a chain to benefit from sessions. You can start a session with a single run and then submit additional runs that reference the same `session_id`. This is ideal when downstream steps depend on external conditions or when you want to decide at runtime which workflow to run next. ```typescript theme={null} // 1) Start a brand new session using a normal run const { data: warmup } = await client.runs.create({ workflow_id: 'login-workflow-id', pool_ids: ['customer-a'], start_session: true, // Reserve a machine and begin a session session_alias: 'step1', // Required if later runs will use $ref to this run's outputs input_values: { username: 'alice' } }) // Get the session to reuse and the reserved machine const sessionId = warmup.session_id! // 2) Run the next workflow in the same session (no other runs will interleave) const { data: step2 } = await client.runs.create({ workflow_id: 'search-workflow-id', session_id: sessionId, // Guarantees same machine & back-to-back scheduling input_values: { query: { $ref: 'step1.outputs.result' } } }) // 3) Final run that releases the session when complete const { data: final } = await client.runs.create({ workflow_id: 'cleanup-workflow-id', session_id: sessionId, release_session_after: true, // Release the session after this run completes input_values: { cleanup: 'true' } }) ``` ```python theme={null} # 1) Start a session and warm up the desktop warmup = client.runs.create_sync(RunCreate( workflow_id='login-workflow-id', pool_ids=['customer-a'], start_session=True, session_alias='step1', # Required if later runs will use $ref to this run's outputs input_values={'username': 'alice'} )).data session_id = warmup.session_id # 2) Add another run in the same session — scheduling remains exclusive client.runs.create_sync(RunCreate( workflow_id='search-workflow-id', session_id=session_id, input_values={'query': 'recent orders'} )) # 3) Final run that releases the session when complete client.runs.create_sync(RunCreate( workflow_id='cleanup-workflow-id', session_id=session_id, release_session_after=True, # Release the session after this run completes input_values={'cleanup': 'true'} )) ``` ## Automatic session release When creating individual runs in a session (not using chains), you can use `release_session_after: true` to automatically release the session when that run completes (regardless of success or failure): ```typescript theme={null} // This run will release the session after it completes const { data: finalRun } = await client.runs.create({ workflow_id: 'final-workflow-id', session_id: existingSessionId, release_session_after: true, input_values: { finalize: 'true' } }) ``` ```python theme={null} # This run will release the session after it completes final_run = client.runs.create_sync(RunCreate( workflow_id='final-workflow-id', session_id=existing_session_id, release_session_after=True, input_values={'finalize': 'true'} )) ``` This is useful as a convenience, so you don't have to decouple creating a session-ending run and actually ending the session. The session is released when the run completes, whether it succeeds, fails, or is cancelled. This ensures the session doesn't remain locked if something goes wrong. ## Detecting session completion via webhooks The `release_session_after` field on a run indicates whether this run released the session. This is useful for webhook consumers who need to know when all runs in a session are complete. **How it works:** * When you explicitly set `release_session_after: true` on a run, that field is stored * When using chains with `keep_session_after_completion: false` (the default), the last run automatically gets `release_session_after: true` * If a run errors or is cancelled and causes the session to be released, `release_session_after` is set to `true` on that run For webhook endpoint setup, signature verification, retries, and local testing, see [Webhooks Quickstart](/webhooks/quickstart). The examples below assume you have already verified a `run_complete` payload and want to treat a successful releasing run as the "session is done" signal. If you want to wait for a session to finish successfully and then gather every run's `output_data`, use this pattern: * `run_complete` is the wait signal * continue only when `run.status === "success"` * continue only when `run.release_session_after === true` * then list all runs in `run.session_id` and aggregate their outputs **Webhook example:** ```typescript theme={null} import { createCyberdeskClient, type RunResponse } from 'cyberdesk' const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!) async function listAllSessionRuns(sessionId: string): Promise { const allRuns: RunResponse[] = [] let skip = 0 const limit = 100 while (true) { const response = await client.runs.list({ session_id: sessionId, skip, limit, sort_mode: 'created_at_desc', fields: ['output_data'], }) if (response.error) { throw response.error } const page = response.data?.items ?? [] allRuns.push(...page) if (page.length < limit) { return allRuns } skip += limit } } async function onRunComplete(run: RunResponse) { if (run.status !== 'success') return if (run.release_session_after !== true) return if (!run.session_id) return const sessionRuns = await listAllSessionRuns(run.session_id) const allRunOutputData = sessionRuns.map((sessionRun) => ({ runId: sessionRun.id, outputData: sessionRun.output_data ?? null, })) console.log({ sessionId: run.session_id, allRunOutputData, }) } // After webhook verification: // const run: RunResponse = payload.run // await onRunComplete(run) ``` ```python theme={null} import os from cyberdesk import CyberdeskClient, RunField, RunListSortMode from openapi_client.cyberdesk_cloud_client.models.run_response import RunResponse from openapi_client.cyberdesk_cloud_client.types import Unset client = CyberdeskClient(os.environ["CYBERDESK_API_KEY"]) def serialize_output_data(output_data: object): if isinstance(output_data, Unset): return None to_dict = getattr(output_data, "to_dict", None) if callable(to_dict): return to_dict() return output_data async def list_all_session_runs(session_id: str) -> list[RunResponse]: all_runs: list[RunResponse] = [] skip = 0 limit = 100 while True: response = await client.runs.list( session_id=session_id, skip=skip, limit=limit, sort_mode=RunListSortMode.CREATED_AT_DESC, fields=[RunField.OUTPUT_DATA], ) if response.error: raise RuntimeError(f"Failed to list session runs: {response.error}") page = response.data.items if response.data else [] all_runs.extend(page) if len(page) < limit: return all_runs skip += limit async def on_run_complete(run: RunResponse): if run.status != "success": return if run.release_session_after is not True: return if not run.session_id: return session_runs = await list_all_session_runs(str(run.session_id)) all_run_output_data = [ { "run_id": str(session_run.id), "output_data": serialize_output_data(session_run.output_data), } for session_run in session_runs ] print({ "session_id": str(run.session_id), "all_run_output_data": all_run_output_data, }) # After webhook verification: # run: RunResponse = cast(RunResponse, evt.run) # await on_run_complete(run) ``` Use `release_session_after` to trigger downstream processing only after all runs in a session are complete — for example, aggregating results, sending notifications, or kicking off the next stage of your pipeline. If you also want to handle failed or cancelled sessions, remove the `run.status == "success"` guard and branch on the final status instead. ## Polling chain runs The chain API returns `run_ids` in creation order. You can poll them individually, or [receive a webhook when any of those runs complete](/webhooks/quickstart): ```typescript theme={null} const { data: chainRes } = await client.runs.chain(chain) for (const runId of chainRes.run_ids) { const run = await waitForRunCompletion(client, runId) console.log(run.status, run.output_data) } ``` ```python theme={null} chain = client.runs.chain_sync(...).data for run_id in chain.run_ids: completed = wait_for_run_completion_sync(client, run_id, 600) print(completed.status, completed.output_data) ``` ## Real-world patterns ### Login + Work (Exclusive) Reserve a session, log into a thick client once, then run 5 workflows in sequence. No other jobs will touch that machine mid-sequence. ### Search + Process with Refs Step 1 finds a record; Step 2 uses `{$ref: 'step1.outputs.id'}` to open/process; Step 3 posts results. All on the same desktop. ### Download → Transform → Export Files created by Step 1 are visible to Steps 2/3 because the session keeps the same working directory. ## Machine targeting If you provide a `machine_id` when creating a chain or run, `pool_ids` are ignored. Cyberdesk will only attempt the specified machine; if it's busy or unavailable, the run will wait until that machine is free (no fallback to other machines or pools). **Best practice**: Use `pool_ids` for flexibility — Cyberdesk will pick any available machine that matches all specified pools. Use `machine_id` only when you specifically need a particular machine (e.g., it has unique software or state). ## Next steps Full TypeScript SDK reference Full Python SDK reference Get notified when runs complete Create workflows in the dashboard # Structured Inputs Source: https://docs.cyberdesk.io/concepts/structured-inputs Pass complex, nested data structures as workflow inputs instead of flattening into dozens of variables When automating form-heavy workflows, you often need to pass many related fields. Instead of creating a separate variable for each field, you can pass structured JSON objects and access nested properties directly in your prompts. ## The Problem Consider a loan document with a "Section G" containing many fields. Without structured inputs, you'd need to define separate variables for each: ```text theme={null} {section_g_cost} {section_g_account_number} {section_g_customers_1_id} {section_g_customers_1_name} {section_g_customers_1_description} ... (many more) ``` This becomes unwieldy and error-prone. ## The Solution: Nested Access With structured inputs, you can pass a single JSON object and access nested properties directly: ```text theme={null} {section_g.cost} {section_g.account_number} {section_g.customers[0].id} {section_g.customers[0].name} {section_g.customers[1].description} ``` Then when creating a run, you simply pass: ```typescript theme={null} const { data: run } = await client.runs.create({ workflow_id: 'workflow-uuid', input_values: { section_g: { cost: 50000, account_number: 'ACC-12345', customers: [ { id: 'C001', name: 'John Doe', description: 'Primary applicant' }, { id: 'C002', name: 'Jane Doe', description: 'Co-applicant' } ] } } }); ``` ```python theme={null} run_data = RunCreate( workflow_id='workflow-uuid', input_values={ 'section_g': { 'cost': 50000, 'account_number': 'ACC-12345', 'customers': [ {'id': 'C001', 'name': 'John Doe', 'description': 'Primary applicant'}, {'id': 'C002', 'name': 'Jane Doe', 'description': 'Co-applicant'} ] } } ) ``` ## Syntax Reference ### Dot Notation Access object properties with `.`: ```text theme={null} {user.email} {config.settings.theme} ``` ### Bracket Notation (Arrays) Access array elements by index: ```text theme={null} {customers[0]} {items[2].name} ``` ### Bracket Notation (Special Keys) For keys with special characters, use bracket notation with quotes: ```text theme={null} {data['special-key']} {headers["Content-Type"]} ``` ### Combining Notations Chain any combination for deeply nested structures: ```text theme={null} {form.sections[0].fields['field-name'].value} {response.data.users[0].profile.settings['notification-preferences']} ``` ## Works Across All Variable Types Structured access works consistently for all three variable types: | Type | Syntax | Example | | --------- | -------------- | ------------------------------- | | Input | `{var.path}` | `{patient.demographics.dob}` | | Sensitive | `{$var.path}` | `{$credentials.api_key}` | | Runtime | `{{var.path}}` | `{{extracted_data.invoice_id}}` | Need these delimiters to appear literally instead of resolving as variables? Escape the opening delimiter: * `\{customer_name}` renders as literal `{customer_name}` * `\{{current_status}}` renders as literal `{{current_status}}` * `\{$support_pin}` renders as literal `{$support_pin}` This works in workflow prompt text and in input values you pass at run start. ### Sensitive Inputs Example ```typescript theme={null} // Workflow prompt uses: {$credentials.username} and {$credentials.password} const { data: run } = await client.runs.create({ workflow_id: 'workflow-uuid', sensitive_input_values: { credentials: { username: 'admin', password: 's3cr3t' } } }); ``` ### Runtime Values Example When a `focused_action` sets a runtime value as an object: ```text theme={null} "Use focused_action to extract the invoice details and save them as {{invoice}}" ``` You can then access nested properties in subsequent steps: ```text theme={null} "Enter the invoice number {{invoice.number}} in the search field" "Verify the amount matches {{invoice.line_items[0].amount}}" ``` ## Using Refs in Chains When running multiple workflows in a [session chain](/concepts/sessions-and-chains), you can use `$ref` to pass outputs from previous steps as inputs to later steps. **Refs work inside structured inputs**, allowing you to build complex input objects from prior outputs. ### Basic ref inside structured input ```typescript theme={null} steps: [ { workflow_id: 'extract-workflow', session_alias: 'extract', inputs: { document_id: '12345' } // This step outputs: { result: { customer: { name: 'John', email: 'john@example.com' } } } }, { workflow_id: 'process-workflow', session_alias: 'process', inputs: { // Use the entire extracted object customer_data: { $ref: 'extract.outputs.result.customer' } } } ] ``` ### Building structured inputs from multiple refs You can mix refs with literal values anywhere in the structure: ```typescript theme={null} inputs: { data: { // Ref to a nested field query: { $ref: 'step1.outputs.result.query' }, // Ref to an array element first_keyword: { $ref: 'step1.outputs.result.keywords[0]' }, context: { // Another ref summary: { $ref: 'step1.outputs.result.summary' }, // Literal value mixed in priority: 'high' } } } ``` Cyberdesk keeps `$ref` values as references when you create the run, validates them against the destination input schema when possible, and resolves them at execution time before the receiving workflow uses them. Your workflow prompt receives the actual values, not the `$ref` syntax. For more on refs and chains, see [Sessions and Chains](/concepts/sessions-and-chains#passing-data-between-steps-with-refs). ## Error Handling ### Type Mismatches (Fails the Run) If you try to access a nested property on a value that isn't an object or array, the run fails immediately with a clear error: ```text theme={null} // Prompt: {section_g.cost} // Input: {"section_g": "just a string"} // ❌ Error: Cannot access path '.cost' on input_values['section_g'] because it is a str, not an object or array ``` This catches configuration errors early, saving you time and usage costs. ### Missing Fields (Uses `__EMPTY__`) If a nested path simply doesn't exist, it's replaced with the `__EMPTY__` sentinel value (same behavior as regular empty inputs): ```text theme={null} // Prompt: {section_g.optional_field} // Input: {"section_g": {"cost": 500}} // Result: __EMPTY__ (field doesn't exist) ``` This allows workflows to gracefully handle optional nested fields. If you want to intentionally blank out a field that already has a value, use [`__CLEAR__`](/concepts/clear-values) instead of `__EMPTY__`. ### Array Out of Bounds Accessing an array index that doesn't exist is treated as missing (not a type error): ```text theme={null} // Prompt: {customers[5].name} // Input: {"customers": [{"name": "John"}]} // Result: __EMPTY__ (only 1 item in array) ``` ## Backward Compatibility Existing workflows continue to work exactly as before: * `{variable}` with a string value → replaced with the string * `{variable}` with an object/array value (no nested access) → JSON stringified ```typescript theme={null} // Prompt: {my_data} // Input: {"my_data": {"a": 1, "b": 2}} // Result: {"a": 1, "b": 2} (stringified JSON) ``` ## Best Practices Organize related inputs into logical objects (e.g., `patient`, `billing`, `shipping`) rather than dozens of flat variables. Define a standard schema for your inputs across workflows. This makes SDK integration easier and reduces errors. Cyberdesk validates type mismatches at run start. Structure your inputs correctly to catch errors before execution begins. Missing nested fields become `__EMPTY__`. Design your prompts to handle this gracefully for optional data. ## Real-World Example: Healthcare Form ```text theme={null} Fill out the patient intake form: **Demographics** - First Name: {patient.demographics.first_name} - Last Name: {patient.demographics.last_name} - DOB: {patient.demographics.date_of_birth} - SSN: {$patient.ssn} **Insurance** - Provider: {patient.insurance.provider} - Policy Number: {patient.insurance.policy_number} - Group ID: {patient.insurance.group_id} **Emergency Contacts** - Primary Contact: {patient.emergency_contacts[0].name} - Primary Phone: {patient.emergency_contacts[0].phone} - Secondary Contact: {patient.emergency_contacts[1].name} - Secondary Phone: {patient.emergency_contacts[1].phone} ``` ```typescript theme={null} const { data: run } = await client.runs.create({ workflow_id: 'patient-intake-workflow', input_values: { patient: { demographics: { first_name: 'John', last_name: 'Doe', date_of_birth: '1985-03-15' }, insurance: { provider: 'Blue Cross', policy_number: 'BC123456', group_id: 'GRP001' }, emergency_contacts: [ { name: 'Jane Doe', phone: '555-0101' }, { name: 'Bob Smith', phone: '555-0102' } ] } }, sensitive_input_values: { patient: { ssn: '123-45-6789' } } }); ``` ```python theme={null} run_data = RunCreate( workflow_id='patient-intake-workflow', input_values={ 'patient': { 'demographics': { 'first_name': 'John', 'last_name': 'Doe', 'date_of_birth': '1985-03-15' }, 'insurance': { 'provider': 'Blue Cross', 'policy_number': 'BC123456', 'group_id': 'GRP001' }, 'emergency_contacts': [ {'name': 'Jane Doe', 'phone': '555-0101'}, {'name': 'Bob Smith', 'phone': '555-0102'} ] } }, sensitive_input_values={ 'patient': { 'ssn': '123-45-6789' } } ) ``` This approach transforms 15+ separate variables into a clean, hierarchical structure that mirrors your actual data model. # Trajectories Source: https://docs.cyberdesk.io/concepts/trajectories Understanding how Cyberdesk records, validates, and replays workflow executions ## What Are Trajectories? Trajectories are reusable sequences of actions that Cyberdesk learns from workflow runs. Think of them as "learned paths" through your software that can be replayed deterministically to make future runs much faster and more reliable. Cyberdesk now separates **capture** from **generation**: * During runs, Cyberdesk may auto-save run-scoped trajectory candidates in the background. * Candidates become user-visible trajectories only after you click **Generate Trajectory** on the corresponding completed run. * Generated trajectories still require manual approval before replay. ## How Trajectories Work ### 1. Run Execution: Background Capture When you run a workflow (new or existing), Cyberdesk captures trajectory candidates for non-cached execution paths: ``` User triggers workflow → Non-cached steps execute → Actions recorded → Candidate saved to run ``` **What Gets Captured**: * Actions taken (click, type, etc.) * Screen state before each action (pre-check snapshot) * Agent's thought process for each step * Final results and observations * Display dimensions (screen resolution) * Input values used during the original run These run-scoped candidates are not replayable until they are generated/promoted. **Example Trajectory Step**: ```json theme={null} { "func_name": "click", "args": [850, 450], "kwargs": {}, "pre_check_snapshot": { "screenshot": "supabase://trajectory-images/organizations/org_123/workflows/wf_456/step_12.png", "last_agent_thought": "I need to click the Submit button to proceed", "custom_cache_detection_instructions": "The Submit button must be visible and enabled" }, "result": "Clicked at coordinates (850, 450)" } ``` ### 2. Generate from Run Details (Promote to Library) After a run completes: ``` Open run details → Click Generate Trajectory (if available) → Trajectory appears in Trajectories tab (unapproved) ``` At this stage, the trajectory is user-visible and reviewable, but still not used for replay until approved. If you see **No new trajectory** on a run, common reasons are: * The run was fully cached, so no new path needed to be captured. * The run is legacy data from before this capture model. * The run ended before a complete candidate trajectory was persisted. ### 3. Subsequent Runs: Replay with Validation When you run the same workflow again, Cyberdesk attempts to replay the trajectory: ``` Load approved, generated trajectories → Validate screen state → Replay if match → Complete in seconds ``` **Validation Process**: 1. **Load**: System loads approved, generated trajectories for this workflow and exact display resolution 2. **Filter**: Filters trajectories whose already-replayed prefix and current step still line up with the live run 3. **Compare**: Captures the current screen and compares it against candidate snapshots 4. **Decision**: The first passing candidate is used for replay. If an optional **main trajectory** is still in contention, Cyberdesk waits for that comparison and prefers it as the tiebreaker. 5. **Replay**: If match, executes the cached action 6. **Fallback**: If no match, falls back to agent for that step **Performance Impact**: * **Cache Hit**: Workflow completes in seconds (no AI reasoning needed) * **Cache Miss**: Falls back to recovery agent, which may either resume the trajectory or create a new one ### Main Trajectory Tiebreaker When a workflow has multiple approved/generated trajectories for the same resolution, Cyberdesk can optionally mark one of them as the **main trajectory**. * A workflow can have **zero or one** main trajectory. * Only **approved generated** trajectories can be marked main. * Switching the main checkbox on one trajectory clears it from the previous main automatically. * Clearing the checkbox leaves the workflow with **no main trajectory**, which is valid and preserves the previous fallback behavior. The main trajectory only matters when Cyberdesk would otherwise have to break a tie: * If **multiple** trajectory candidates get a cache hit at the same step, Cyberdesk prefers the main trajectory. * If **all** trajectory candidates miss at the same step, Cyberdesk prefers the main trajectory when building recovery context for the recovery agent. * If **no** trajectory is marked main, Cyberdesk behaves the same way it did before this feature. Choose a main trajectory when one path is your most stable or most desirable canonical path. If your workflow does not need that extra determinism, it is fine to leave every trajectory unchecked. ## Resume Trajectory: Smart Cache Recovery When a cache miss occurs, Cyberdesk doesn't immediately abandon the cached trajectory. Instead, the recovery agent has the ability to **resume the trajectory** after handling minor disruptions. ### How Resume Works When the screen state doesn't match the cached snapshot (cache miss), the recovery agent: 1. **Analyzes the situation** - Determines if this is a minor disruption or a major deviation 2. **Takes recovery actions** - Dismisses popups, closes dialogs, or fixes the screen state 3. **Decides whether to resume** - Calls `resume_trajectory` to continue the cached path, or proceeds with full recovery ``` Cache Miss → Recovery Agent → Quick Fix → resume_trajectory → Continue Cached Trajectory ↓ Major Issue → Full Recovery → New Trajectory Created ``` ### When Resume is Used The recovery agent calls `resume_trajectory` for: * **Popups and dialogs** - Unexpected alerts, cookie banners, notifications * **Minor UI shifts** - Elements that moved slightly but are still accessible * **Transient states** - Loading spinners that completed, tooltips that appeared The recovery agent proceeds with full recovery for: * **Major UI changes** - Different page loaded, workflow state changed significantly * **Blocked paths** - Required elements missing or inaccessible * **Uncertain situations** - When resuming might cause incorrect behavior ### How Resume is Called When the recovery agent determines the situation is recoverable, it calls `resume_trajectory()` to signal that the system should continue replaying the cached trajectory. The system will then retry the step that originally failed cache detection. ### Automatic Retry Logic If `resume_trajectory` is called but the next cache check still fails: * The system retries with the recovery agent again * The recovery prompt includes how many prior `resume_trajectory` attempts have already failed * The agent can then either try to restore the exact cached state again or proceed with full recovery instead ### Benefits of Resume * **Preserves the approved trajectory** - No new trajectory is created for minor disruptions * **Faster recovery** - Quick fixes complete in seconds vs. full re-recording * **Consistent execution** - Continues the validated path rather than an untested new one * **Recovery actions are not saved** - The popup dismissal or dialog close isn't added to the trajectory Recovery actions taken before `resume_trajectory` are logged for debugging but are not saved to the trajectory. The cached trajectory remains unchanged. ## The Approval Process **Critical**: Newly created trajectories are **NOT automatically approved**. They must be manually reviewed and approved before they can be used in future workflow executions. ### Why Approval is Required Trajectories capture exact coordinates, keyboard inputs, and action sequences. Before allowing automatic replay: * **Verify Correctness**: Ensure the recorded actions actually worked * **Check for Errors**: Confirm no mistakes were captured in the trajectory * **Review Observations**: Validate that focused actions extracted correct data * **Inspect Coordinates**: Ensure clicks hit the right UI elements **Safety First**: Unapproved trajectories sit in your trajectory library but are never used for replay, preventing potentially incorrect actions from running automatically. ### How to Approve Trajectories **In the Dashboard**: 1. Navigate to **Runs** (or the workflow's **Runs** section) and open a completed run. 2. Click **Generate Trajectory** on the run details panel (if available). 3. Navigate to **Workflows** → Select your workflow → **Trajectories** tab. 4. Review the trajectory: * Check each step's screenshot and action * Verify focused action observations * Inspect coordinates and inputs 5. Click the **Approval** toggle or checkbox. 6. Optionally mark one approved/generated trajectory as **Main** if you want deterministic tie-breaking. 7. Trajectory is now available for future runs. **Best Practice**: Generate and approve trajectories promptly after good runs to maximize cache hit rates. ## Editing Trajectories You can edit trajectories in the dashboard to fix coordinates, update observations, or refine cache detection. ### What You Can Edit **Step Actions**: * Coordinates for clicks and drags * Text to type * Scroll amounts * Screenshot zoom areas **Pre-Check Snapshots**: * Custom cache detection instructions * Expected screen states **Focused Action Observations**: * Extracted data and observations * Runtime variable assignments ### How to Edit 1. Open a trajectory in the trajectory viewer 2. Expand the step you want to edit 3. Click the edit icon on specific fields 4. Make your changes 5. Save the trajectory **Important**: Changes take effect immediately on approved trajectories. If you're making significant changes, consider unapproving the trajectory first, testing it, then re-approving. ### Adding Wait Steps You can add wait steps directly in the trajectory editor to introduce delays between actions. This is useful when: * The application needs time to process between steps * Animations or loading states need to complete * You're experiencing timing issues during replay **How to Add a Wait**: 1. Hover over any trajectory step 2. A **+ Wait** button appears at the bottom-right of the step 3. Click it to insert a 2-second wait after that step 4. The new wait step expands automatically so you can adjust the duration 5. Click **Save Changes** to persist **Cache Detection Disabled**: Steps added via UI cannot use cache detection because there's no reference screenshot. They will always replay deterministically with the specified duration. After adding waits to a trajectory, consider updating your workflow prompt to include the same waits. This ensures new trajectory recordings will also include the delays. ## Editing Focused Actions: Critical Workflow Update **⚠️ HIGHLY RECOMMENDED**: When you edit a `focused_action` observation or instruction in a trajectory, you should **also update the underlying workflow prompt** to match. ### Why This Matters When a cache miss occurs, the system falls back to running the AI agent using your workflow's main prompt. If the trajectory has diverged from the prompt, you'll get inconsistent behavior: **Scenario**: * **Trajectory**: `focused_action` extracts "patient\_mrn, date\_of\_birth, primary\_diagnosis" * **Workflow Prompt**: Says to extract "patient\_mrn and date\_of\_birth" only * **Cache Hit**: Works perfectly (uses trajectory) * **Cache Miss**: Falls back to prompt, extracts different fields, causes output schema mismatch ### The Right Approach When editing focused actions in trajectories: 1. **Edit the Trajectory**: * Update the focused action instruction * Modify the expected observation * Adjust runtime variables if needed 2. **Edit the Workflow Prompt**: * Update the same focused\_action instruction in the main prompt * Ensure consistency between trajectory and prompt * Test with cache disabled to verify prompt works 3. **Test Both Paths**: * Run with cache hit (uses trajectory) * Run with cache disabled (uses prompt) * Verify both produce same results **Example**: **If you change trajectory from**: ``` focused_action: "Extract patient MRN only" ``` **To**: ``` focused_action: "Extract patient MRN and date of birth" ``` **Also update workflow prompt from**: ``` "Use focused_action to extract the patient MRN" ``` **To**: ``` "Use focused_action to extract the patient MRN and date of birth" ``` **Pro Tip**: When making significant trajectory edits, temporarily disable cache detection to test the workflow with the updated prompt before re-enabling caching. ## Cache Detection: Enabling and Disabling Cache detection is active whenever Cyberdesk has an approved, generated trajectory for the workflow and current screen resolution. There is not currently a dedicated per-run "disable cache" toggle in the dashboard. ### When Cache Detection is Active (Default) Every workflow run attempts to use approved, generated trajectories: ``` Run Start → Load Approved Trajectories → Validate Each Step → Replay if Match → Agent if Miss ``` **Benefits**: * ⚡ Massive speed improvements (cache hits complete instantly) * 🎯 Consistent execution (same actions every time) ### Disabling Cache Detection To force fresh agent execution from the start of a workflow, temporarily unapprove the relevant trajectories in the dashboard. **Use Cases for Disabling**: * Testing workflow prompt changes * Debugging trajectory mismatches * Verifying prompt works without cache * Development and testing **Re-Enabling**: * Re-approve the trajectories you want Cyberdesk to consider again * You can also disable cache detection on individual trajectory steps inside the trajectory editor when only certain steps should always replay deterministically ## Cache Detection in Loops By default, recorded loop steps use cache detection. Control steps such as `end_loop_iteration` and `skip_loop_iteration` do not use cache detection, and UI-added steps also skip cache detection because they do not have reference screenshots. You can disable cache detection on individual loop steps in the trajectory editor to make them deterministic. If cache detection fails mid-loop, the recovery agent can: * `resume_trajectory` for minor mismatches * `end_loop_iteration` or `skip_loop_iteration` to continue the loop * `declare_task_failed` if the workflow is unsafe to recover See [Looping Tools](/workflow-prompting/looping-tools) for full details and examples. ## Custom Cache Detection Instructions To improve cache hit rates and reduce false negatives/positives, you can add [Custom Cache Detection Instructions](/concepts/custom-cache-detection) to trajectory steps. ### What They Do Custom instructions guide the cache validation AI by providing human context: ``` "The Submit button must be visible and enabled. Ignore the order number displayed, as it will differ between runs." ``` ### Where to Add Them 1. Open a trajectory in the trajectory viewer 2. Expand a step to view details 3. Find "Custom Cache Detection Instructions" in the Pre-check Snapshot 4. Add your guidance 5. Save changes ### When to Use Them Add custom instructions when: * ✅ A step frequently has false cache misses due to minor differences * ✅ Dynamic content (timestamps, IDs) causes unnecessary validation failures * ✅ Specific UI elements are critical while others are cosmetic * ✅ You need to specify tolerances for acceptable variations * ✅ Recovery agent needs context about validation criteria **Example**: ``` "The patient list must show at least 5 entries. Patient names will vary based on {search_term}. Focus on the list structure, not specific names." ``` ### Impact on Cache Hit Rates Well-written custom instructions can: * Reduce false negatives (rejecting valid matches) * Reduce false positives (accepting invalid matches) * Improve recovery agent performance * Provide valuable context for debugging See the full [Custom Cache Detection Instructions](/concepts/custom-cache-detection) guide for detailed examples and best practices. ## Trajectory Lifecycle ### Phase 1: Capture (Auto-Saved) * Workflow run executes non-cached steps * Actions are recorded in real-time * Candidate trajectory is saved and linked to the run * **Status**: Pending generation (`is_generated=false`), not visible in default trajectory list ### Phase 2: Generation (Manual Promote) * Open run details and click **Generate Trajectory** * Candidate is promoted into the trajectory library * **Status**: Generated but unapproved (`is_generated=true`, `is_approved=false`) ### Phase 3: Review * View trajectory in the workflow's **Trajectories** tab * Inspect each step and screenshot * Verify focused action observations * Check coordinates and inputs ### Phase 4: Approval * Approve if trajectory is correct * **Status**: Approved, ready for replay ### Phase 5: Usage * Future runs load this approved trajectory * Each step validated before replay * Cache hits = fast execution * Cache misses = recovery agent attempts to resume or records a new run-scoped candidate ### Phase 6: Maintenance * Edit as needed (coordinates, instructions) * Update workflow prompt if editing focused actions * Test with cache disabled after significant changes * Unapprove if major edits needed, re-approve after testing ## Multiple Trajectories Per Workflow A single workflow can have multiple approved, generated trajectories: **Why Multiple Trajectories?** * **Different Input Patterns**: Trajectory for "new patient" vs "existing patient" * **UI Variations**: Trajectory for each possible screen layout * **Resolution Differences**: Trajectory for 1920x1080 vs 1280x720 * **Branching Paths**: Different trajectories for different workflow branches **How Selection Works**: 1. System loads approved, generated trajectories for this workflow 2. Filters by display resolution (must match exactly) 3. Keeps only candidates whose current replay position and tool sequence still line up 4. Compares the current screen against candidate snapshots, in parallel when needed 5. Selects the first candidate that passes cache detection, with the optional main trajectory acting as the tiebreaker when it is still in contention 6. Replays selected trajectory step by step 7. Falls back to agent if no trajectory matches **Parallel Filtering**: * Multiple candidate comparisons can run simultaneously against the same captured screen * This keeps selection fast even when several approved trajectories share the same step * The main trajectory only affects ties; it does not force selection when it misses ## Best Practices ### 1. Approve Promptly Approve successful trajectories quickly to start benefiting from cache hits on subsequent runs. ### 2. Keep Prompts in Sync When editing trajectories (especially focused actions), update the workflow prompt to match: ``` Trajectory Edit: focused_action observation changed ↓ Workflow Prompt: Update focused_action instruction ↓ Test: Run with cache disabled ↓ Verify: Both paths produce same results ↓ Re-approve: Trajectory ready for use ``` ### 3. Use Custom Instructions Strategically Add custom instructions to steps that: * Frequently fail validation unnecessarily * Have dynamic content that's acceptable to ignore * Require nuanced validation logic ### 4. Test Before Approving Run the workflow at least once successfully before approving the trajectory: * Verify all actions completed correctly * Check output data matches expectations * Ensure focused actions extracted correct values * Inspect coordinates hit the right UI elements ### 5. Maintain Trajectory Hygiene * **Unapprove** trajectories that are outdated or incorrect * **Delete** trajectories that are no longer relevant * **Edit** trajectories when UI changes are minor (coordinate adjustments) * **Duplicate** trajectories to create a copy for experimentation (all images are copied to new storage paths) * **Re-record** when UI changes are major (new flow needed) ### 6. Monitor Cache Hit Rates Inspect cache behavior in recent runs using the run message history: * Look for cache hit, cache miss, and trajectory-selection messages * Check which steps frequently miss cache * Update custom instructions, edit the trajectory, or regenerate it when misses cluster around the same UI change ### 7. Consider Resolution Trajectories are resolution-specific: * Record trajectories at your most common resolution * If you use multiple resolutions, you'll need multiple trajectories * Display dimensions must match exactly for replay ## Common Scenarios ### Scenario 1: UI Element Moved **Problem**: Button moved from (850, 450) to (850, 480) **Solution**: 1. Open trajectory in viewer 2. Find the click step 3. Edit coordinates from (850, 450) to (850, 480) 4. Save trajectory 5. Test with cache enabled **Quick Fix**: Minor coordinate adjustments don't require prompt changes. ### Scenario 2: Focused Action Needs More Data **Problem**: Trajectory extracts "customer\_id" but you now need "customer\_id and email" **Solution**: 1. Edit trajectory focused\_action observation to include email 2. **IMPORTANT**: Update workflow prompt to match: ``` Before: "Use focused_action to extract customer_id" After: "Use focused_action to extract customer_id and email" ``` 3. Test with cache disabled to verify prompt works 4. Test with cache enabled to verify trajectory works 5. Re-approve if unapproved **Critical**: Prompt must match trajectory for consistent cache miss behavior. ### Scenario 3: Popup Causes Cache Miss (Resume Recovery) **Problem**: An unexpected popup (cookie banner, notification, etc.) causes cache detection to fail **What Happens Automatically**: 1. Cache detection fails due to the popup 2. Recovery agent is invoked with context about the expected screen state 3. Agent dismisses the popup 4. Agent calls `resume_trajectory()` to continue 5. Cache detection retries and succeeds 6. Workflow continues on the approved trajectory **Result**: The popup is handled without creating a new trajectory. The dismissal action is logged but not saved. If popups frequently cause cache misses, consider adding custom cache detection instructions to ignore known popup elements, or update your workflow to handle popups proactively. ### Scenario 4: Dynamic Content Causes False Misses **Problem**: Cache detection fails because timestamps/IDs differ **Solution**: 1. Open trajectory step 2. Add custom cache detection instruction: ``` "The form structure must match with 3 input fields. Ignore the order ID shown - it will differ per run." ``` 3. Save trajectory 4. Future runs will ignore order ID differences See [Custom Cache Detection Instructions](/concepts/custom-cache-detection) for more examples. ### Scenario 5: Workflow Prompt Changed Significantly **Problem**: You updated the workflow prompt with new steps or different logic **Solution**: 1. Unapprove all existing trajectories (they're now outdated) 2. Run workflow with cache disabled (force fresh recording) 3. Review new trajectory 4. Approve new trajectory 5. Old trajectories can be deleted or kept for reference **Don't Edit**: When prompts change significantly, recording a fresh trajectory is better than trying to edit the old one. ### Scenario 6: Want to Test Without Cache **Problem**: Need to verify prompt works independently of cache **Solution**: 1. Trigger run with all trajectories unapproved 2. Agent executes from scratch using workflow prompt 3. A new run-scoped trajectory candidate may be captured if non-cached steps executed 4. Open that run and click **Generate Trajectory** 5. Review and approve the generated trajectory if it's better **Use Case**: Regression testing, prompt validation, debugging ## Trajectory Data Structure The examples below are abbreviated for readability. Persisted trajectory rows also include metadata such as `func_hash`, `signature_hash`, `is_method`, `skip_cache_detection`, `post_check_snapshot`, and argument wrappers like `{ "static_value": ... }`. ### High-Level Structure ```json theme={null} { "id": "uuid", "workflow_id": "uuid", "is_approved": false, "dimensions": {"width": 1920, "height": 1080}, "original_input_values": {"patient_name": "John Doe"}, "trajectory_data": [ { "func_name": "click", "args": [x, y], "kwargs": {}, "pre_check_snapshot": { ... }, "result": "..." }, { "func_name": "type", "args": ["search text"], "kwargs": {}, "pre_check_snapshot": { ... }, "result": "..." }, ... ] } ``` ### Pre-Check Snapshot Captured before each action: ```json theme={null} { "screenshot": "supabase://trajectory-images/organizations/org_123/workflows/wf_456/step_12.png", "last_agent_thought": "I can see the login form...", "custom_cache_detection_instructions": "Login button must be visible", "coordinates": {"x": 850, "y": 450}, "step_signature_hash": 123456789 } ``` ### Focused Action Steps Special handling for dynamic observations: ```json theme={null} { "func_name": "focused_action", "args": ["Extract the patient MRN"], "kwargs": { "is_cached_action": true, "cached_thought": "I need to find the MRN field" }, "pre_check_snapshot": { ... }, "result": "Patient MRN: MRN12345" } ``` When editing focused actions, remember to update the workflow prompt to match! ## Performance Benefits ### Without Trajectories (Every Run Uses AI) ``` Run 1: AI agent from scratch (45 seconds) Run 2: AI agent from scratch (45 seconds) Run 3: AI agent from scratch (45 seconds) Average: 45 seconds per run ``` ### With Trajectories (Cache Hits) ``` Run 1: AI agent + record trajectory (45 seconds) → Trajectory approved Run 2: Replay trajectory (5 seconds) ✅ Cache hit Run 3: Replay trajectory (5 seconds) ✅ Cache hit Average: 18 seconds per run, improving to ~5 seconds as cache hits increase ``` **Typical Improvement**: 5-10x faster execution after trajectory approval ### Partial Cache Hits Even partial cache hits provide benefits: ``` Steps 1-5: Replay from trajectory (2 seconds) Step 6: Cache miss, use AI agent (8 seconds) Steps 7-10: Record new actions (continue with agent) Total: 10 seconds (vs 45 seconds from scratch) ``` ### Resume Recovery Performance When the recovery agent successfully resumes: ``` Steps 1-5: Replay from trajectory (2 seconds) Step 6: Cache miss, popup appears Recovery: Dismiss popup (1 second) resume_trajectory called Steps 6-10: Continue replay (2 seconds) Total: 5 seconds (trajectory preserved) ``` Resume recovery is faster than partial cache hits because no new trajectory is recorded. ## Trajectory Strategies ### Strategy 1: Single Golden Trajectory **Approach**: One approved trajectory per workflow **Best For**: * Highly deterministic workflows * Consistent UI layouts * Same input pattern every time * Minimal variation between runs **Pros**: Simple, easy to maintain **Cons**: Any variation causes cache miss ### Strategy 2: Multi-Path Trajectories **Approach**: Multiple approved trajectories for different scenarios **Best For**: * Workflows with branching logic * Different input patterns (new vs existing records) * UI variations (different screen layouts) * Different resolution targets **Pros**: Higher cache hit rates, handles variation **Cons**: More trajectories to maintain ### Strategy 3: Progressive Refinement **Approach**: Start with one trajectory, add more as variations are discovered **Best For**: * New workflows where patterns emerge over time * Workflows with occasional edge cases * Gradual optimization **Process**: 1. Approve first successful trajectory 2. Monitor cache miss patterns 3. Identify common variations 4. Record and approve trajectories for those variations 5. Cache hit rate improves over time ## Troubleshooting ### Low Cache Hit Rate **Symptoms**: Most runs fall back to agent instead of using cache **Possible Causes**: * No approved trajectories * Custom instructions too strict * UI changed since trajectory was recorded * Resolution mismatch * Input patterns vary significantly **Solutions**: * Approve successful trajectories * Add custom instructions to tolerate minor differences * Re-record trajectories after UI changes * Ensure consistent display resolution * Consider multiple trajectories for different input patterns ### Trajectory Approval Uncertainty **Question**: "Should I approve this trajectory?" **Checklist**: * ✅ Workflow completed successfully * ✅ Output data is correct * ✅ Focused actions extracted expected values * ✅ All clicks hit the right UI elements * ✅ No errors in the run logs * ✅ Screenshots show correct screens at each step If all checks pass → **Approve**\ If any concern → **Review more carefully** or run again to verify ### Resume Trajectory Not Working **Symptoms**: Recovery agent keeps creating new trajectories instead of resuming **Possible Causes**: * Major UI changes that can't be recovered with simple actions * Cache detection consistently failing even after recovery * Screen state too different from expected snapshot **Solutions**: * Check if the disruption is truly minor (popups, dialogs) vs major (different page, missing elements) * Add custom cache detection instructions to be more lenient * Re-record trajectory if the UI has changed significantly * Check logs to see if resume was attempted but cache detection kept failing If repeated `resume_trajectory` attempts keep failing, the recovery prompt will call that out and suggest taking over fully. If you're seeing this frequently, the cache detection instructions may need adjustment. ### Trajectory Editing vs Re-Recording **Edit Trajectory When**: * Minor coordinate adjustments (button moved slightly) * Updating custom cache detection instructions * Refining focused action observations (if prompt also updated) * Small corrections that don't change workflow logic **Re-Record Trajectory When**: * Major UI redesign * Workflow prompt changed significantly * Different action sequence needed * New steps added or removed * Complete workflow refactor **Rule of Thumb**: If you're changing more than 3-4 steps or the workflow logic has changed, re-record instead of editing. ## Advanced Topics ### Parameterized Trajectories Trajectories store the original input values that were present when the path was recorded: **Original Input Values**: ```json theme={null} { "patient_name": "John Doe", "date": "2024-01-15" } ``` Cyberdesk uses this context during cache detection and recovery, and surfaces it in the dashboard so you can see what the trajectory was originally based on. **What This Does Not Mean**: ```json theme={null} { "patient_name": "Jane Smith", "date": "2024-01-16" } ``` Different runtime values can help the system understand that dynamic content changed legitimately, but they do not automatically rewrite previously recorded click/type payloads inside the stored trajectory. If new inputs should change the recorded actions themselves, regenerate or edit the trajectory. ### Trajectory Versioning Trajectories remain editable after approval: * Saving edits updates the existing trajectory record and changes its `updated_at` timestamp * Old trajectory data is overwritten rather than stored as a separate built-in version history * Consider duplicating or unapproving before major edits ### Resolution Filtering Trajectories are automatically filtered by display resolution: * **Exact Match Required**: 1920x1080 trajectory won't work on 1280x720 * **Why**: Coordinates are absolute pixel positions * **Solution**: Record trajectories at each resolution you use ## Summary Trajectories are Cyberdesk's intelligent caching system that dramatically speeds up workflow execution: * **Capture**: Run-scoped trajectory candidates are auto-saved in the background * **Generation**: Use **Generate Trajectory** on a completed run to promote a candidate into the trajectory library * **Approval**: Required before generated trajectories can be used * **Validation**: AI compares screen states before each replay * **Resume Recovery**: Minor disruptions are handled automatically without creating new trajectories * **Editing**: Supported, but keep prompts in sync (especially for focused actions) * **Custom Instructions**: Enhance cache detection accuracy * **Cache Control**: Temporarily unapprove trajectories or disable cache detection on individual steps when you need fresh execution **Key Takeaway**: Capture happens automatically, but replayable trajectories are created intentionally. Generate from good runs, approve promptly, keep prompts synchronized when editing, and use custom instructions to optimize hit rates. For advanced cache validation, see [Custom Cache Detection Instructions](/concepts/custom-cache-detection). # Windows VM 101 Source: https://docs.cyberdesk.io/concepts/windows-vm-101 Set up a reliable Windows environment for desktop automation This guide describes the recommended Windows setup for the current RustDesk-based Cyberdriver. The old Python-agent setup with `--add-persistent-display` and `--black-screen-recovery` is documented under [Legacy Cyberdriver](/cyberdriver/legacy-cyberdriver). ## Goals A reliable Windows automation machine should: * stay reachable without an active RDP window * expose a stable 1024×768 desktop when possible * run Cyberdriver as a service * avoid unexpected restarts and sleep * support screenshots, clicks, typing, clipboard, files, and shell commands from Cyberdesk ## Recommended setup Use Windows Server 2022 or a supported Windows desktop environment. RDP is useful for initial setup, but Cyberdriver should continue working after RDP disconnects. Open PowerShell as Administrator and run the installer script from [Cyberdriver Quickstart](/cyberdriver/quickstart#windows-install). The script installs Cyberdriver as a Windows service, adds it to `PATH`, and verifies the CLI. During beta, Windows may show SmartScreen warnings; choose **More info → Run anyway**. Administrator is required for installation because service mode is what lets Cyberdriver start at boot and work at the Windows login screen. After installation, the Cyberdriver app itself can be opened normally. Open Cyberdriver, paste your organization API key into the **Cyberdesk tunnel** card, and click **Save**. Once connected, the machine appears in the Cyberdesk dashboard. From Cyberdesk, open the desktop's **Tools** dialog. Verify screenshot, click, type, clipboard, files, and PowerShell. If needed, use the System tab to set the display to 1024×768. Close the RDP window, wait briefly, then use Desktop Tools again. Cyberdriver 1.x beta relies on RustDesk's display stack and service mode rather than the legacy screenshot-based black-screen recovery loop. ## RDP and display notes Cyberdriver 1.x beta uses RustDesk's virtual display/display enumeration path. If RDP disconnect changes the active display, Cyberdriver should still be able to recover a usable display through the RustDesk service. If a machine shows a black screen after RDP disconnect: 1. Open Desktop Tools and check **System → Display dimensions**. 2. Use **Copy diagnostics** in Cyberdriver settings. 3. Reconnect once through Cyberdriver or RDP, then disconnect RDP and try again. 4. Contact support with diagnostics and logs. ## Power settings Disable sleep and aggressive screen lock policies for unattended automation. Keepalive can help with idle timers, but it is not a replacement for sensible OS power settings. ## Legacy setup The old Python Cyberdriver recommended: ```bash theme={null} cyberdriver join --secret YOUR_API_KEY --keepalive --black-screen-recovery --add-persistent-display ``` Do not use those flags with Cyberdriver 1.x beta. See [Legacy Cyberdriver](/cyberdriver/legacy-cyberdriver) if you are using the stable Python executable. # Workflow Duplication & Merge Source: https://docs.cyberdesk.io/concepts/workflow-duplication-and-merge Duplicate a workflow to branch, iterate, or merge changes back ## Why it’s useful * Iterate safely without touching the original workflow * Branch into a new workflow and riff from a solid baseline * Merge improvements back only when you’re confident * Optionally carry over trajectories for faster replay ## How to use it 1. Open a workflow, click the **three‑dot menu** in the top‑right, then click **Duplicate**. 2. Edit and test the copy. 3. If you want a new workflow, keep iterating on the copy and you’re done. 4. If you want to merge back, open the copy and click **Merge Workflow**. 5. Choose the target (usually the original), decide whether to copy generated trajectories, and confirm the merge. ## How it works * **Duplicate** creates a new workflow with the same prompt, schemas, post-run checks, settings, and tags. Prompt images are copied to new storage paths so the copy is independent. If you opt in, generated trajectories are duplicated onto the copy with fresh trajectory images and remapped prompt-image references. * **Merge** updates the target workflow’s prompt, schemas, post-run checks, settings, and tags from the source. The target name stays the same. This is a replace-to-match merge for workflow configuration, not a union. * If you duplicate a workflow, add new post-run checks on the copy, and merge it back, those new checks come over because the copy already contains the original checks plus your additions. Target-only post-run checks on the destination do not survive unless they also exist on the source. * Before the overwrite, merge saves the target’s current name, prompt, schemas, terminal command allowlist, and post-run checks in version history. If you opt in, generated trajectories are duplicated with new images. The source workflow is then deleted asynchronously. # Workflow Tags & Groups Source: https://docs.cyberdesk.io/concepts/workflow-tags Organize your workflows with tags and tag groups for better filtering, categorization, and team collaboration As your workflow library grows, finding and organizing them becomes essential. Workflow tags let you categorize, filter, and manage your workflows efficiently. Tag groups add another layer of organization with mutually exclusive labels. ## What Are Workflow Tags? Tags are labels you attach to workflows for organization and quick filtering. Each tag can have: * **Name** - A short, descriptive label (e.g., "Production", "Testing", "High Priority") * **Color** - Visual differentiation (red, orange, yellow, green, blue, purple, pink, gray) * **Emoji** - Optional icon for quick recognition * **Description** - Optional notes about when to use this tag ## Using Tags in the Dashboard ### Viewing Tags Tags appear prominently above your workflow table. Each tag shows: * Its emoji (if set) * Its name * A count of how many workflows use it ### Filtering by Tags Click any tag to filter the workflow table to only show workflows with that tag. You can select multiple tags to filter by all of them (AND logic) - only workflows that have **all** selected tags will appear. Use the clear-filters action to remove all selected tags and return to the full workflow list. ### Adding Tags to Workflows There are two ways to add tags to workflows: **Individual workflow:** 1. Open the workflow detail page 2. Use the tags section to add or remove tags **Bulk tagging:** 1. Select multiple workflows using the checkboxes 2. Click the "Add Tags" button that appears 3. Choose which tags to apply to all selected workflows ## Tag Groups Tag groups let you organize related tags and enforce mutual exclusivity. ### What Are Tag Groups? A tag group is a named collection of related tags where **only one tag from the group can be applied to a workflow at a time**. This is perfect for status-type categorizations. **Example: Status Group** * 🟢 Production * 🟡 Staging * 🔴 Development When you add "Production" to a workflow, any existing status tag (like "Staging") is automatically removed. ### When to Use Groups Track workflow lifecycle: Draft → Testing → Production → Deprecated Mark importance: Low → Medium → High → Critical Categorize by environment: Dev → Staging → Prod Assign to teams: Engineering → QA → Operations ### When NOT to Use Groups Don't use groups for tags that can coexist on the same workflow. For example: * Feature areas (a workflow might touch "Billing" AND "Notifications") * Clients (unless a workflow is truly client-specific) * Capabilities (a workflow might do "OCR" AND "Form Filling") These should be ungrouped tags so you can apply multiple. ## Creating Tags & Groups ### From the Dashboard Click **"Tag Actions"** above the workflow table to: * **New Tag** - Create a new tag (optionally in a group) * **New Group** - Create a new tag group * **Manage Tags** - Edit, delete, or reorganize existing tags ### Via the API ```typescript theme={null} // Create a tag group const { data: statusGroup } = await client.workflow_tag_groups.create({ name: "Status", emoji: "🚦" }); // Create tags in the group await client.workflow_tags.create({ name: "Production", color: "green", emoji: "🟢", group_id: statusGroup.id }); await client.workflow_tags.create({ name: "Development", color: "red", emoji: "🔴", group_id: statusGroup.id }); // Create an ungrouped tag await client.workflow_tags.create({ name: "Needs Review", color: "yellow", emoji: "👀" }); ``` ```python theme={null} # Create a tag group status_group = client.workflow_tag_groups.create( name="Status", emoji="🚦" ) # Create tags in the group client.workflow_tags.create( name="Production", color="green", emoji="🟢", group_id=status_group.id ) client.workflow_tags.create( name="Development", color="red", emoji="🔴", group_id=status_group.id ) # Create an ungrouped tag client.workflow_tags.create( name="Needs Review", color="yellow", emoji="👀" ) ``` ## Filtering Workflows by Tags (API) When listing workflows, pass `tag_ids` to filter: ```typescript theme={null} // Filter by single tag const { data: workflows } = await client.workflows.list({ tag_ids: "tag-uuid-1" }); // Filter by multiple tags (AND logic - must have ALL tags) const { data: workflows } = await client.workflows.list({ tag_ids: "tag-uuid-1,tag-uuid-2" }); // Include tag data in response const { data: workflows } = await client.workflows.list({ include_tags: true }); ``` ```python theme={null} # Filter by single tag workflows = client.workflows.list(tag_ids="tag-uuid-1") # Filter by multiple tags (AND logic - must have ALL tags) workflows = client.workflows.list(tag_ids="tag-uuid-1,tag-uuid-2") # Include tag data in response workflows = client.workflows.list(include_tags=True) ``` ## Drag-and-Drop Organization The tag bar supports drag-and-drop for intuitive organization: ### Reordering Tags Drag any tag left or right to change its position within its group (or among ungrouped tags). The order is saved automatically. ### Moving Tags Between Groups Drag a tag onto a different group's label to move it into that group. The tag will now be mutually exclusive with other tags in its new group. ### Ungrouping Tags Drag a grouped tag to the ungrouped area (outside any group) to remove it from its group. It will become a standalone tag that can coexist with any other tag. ### Reordering Groups Drag a group label left or right to change the display order of groups in the tag bar. ## Best Practices Begin with a few essential tags. You can always add more as patterns emerge from your workflow usage. Adopt a color convention (e.g., green = good/active, red = danger/deprecated) and stick to it across all tags. Any time a workflow can only be in one state at a time, use a tag group to enforce this. Emojis make tags instantly recognizable when scanning a long list. Pick distinctive ones. ## Example: Production Workflow Management Here's a complete tagging setup for managing production workflows: **Tag Groups:** * **Status** (🚦): Draft, Testing, Production, Deprecated * **Priority** (⚡): Low, Medium, High, Critical **Ungrouped Tags:** * 📋 Needs Review * 🔧 Maintenance * 📊 Generates Reports * 💳 Handles Payments * 🔐 Requires Auth This setup lets you: 1. Filter to "Production + Critical" to see urgent production workflows 2. Ensure a workflow is only ever in one status 3. Tag workflows with multiple capabilities (e.g., both "Generates Reports" and "Handles Payments") 4. Quickly spot workflows needing attention via the "Needs Review" tag ## API Reference For complete API documentation on tags and tag groups, see: * [Workflow Tags API](/api-reference/workflow-tags/list-tags) * [Workflow Tag Groups API](/api-reference/workflow-tag-groups/list-tag-groups) # Cyberdriver Reachability Checks Source: https://docs.cyberdesk.io/cyberdriver/automatic-machine-quarantine How Cyberdesk confirms and reports when Cyberdriver becomes unreachable during a run ## Overview Cyberdesk checks whether Cyberdriver is reachable at run startup and after an in-run Cyberdriver request exhausts its normal retries. These checks distinguish a sustained machine-side outage from a transient request failure. Automatic machine state quarantine is currently paused while reachability classification is tuned. A failed run-owned check does not change the desktop's connection status, availability, pool memberships, or Cyberdriver routing target. A genuine Cyberdriver WebSocket disconnect can still update connection state through the normal connection lifecycle. Cyberdesk confirms the outage with a lightweight display-dimensions request. * Mid-run confirmation first waits **2 seconds** so Cyberdriver can finish any abandoned request that timed out on the client. * It then makes up to **three** dimensions attempts. * Each attempt uses a **10-second** request timeout. * After a failed attempt, Cyberdesk waits **2 seconds**, then **4 seconds**, before the next attempt. The first successful response means the machine is reachable and stops the check. * Any unsuccessful response counts as a failed attempt, including HTTP 4xx. If all three attempts fail, Cyberdesk: * Ends the affected run with an `error` status and repair instructions. * Leaves the desktop's status, availability, pool memberships, and routing target unchanged. * Immediately sends a `downed_machines` webhook when **Downed machine alerts** is enabled for that desktop. The alert setting controls only webhook and Slack delivery. Reachability confirmation and normal run error handling still apply when alerts are disabled. ## Run Error The affected run explains what Cyberdesk detected: > Cyberdriver became unreachable on this machine after 3 display checks. Repair or restart Cyberdriver on the machine, then verify that it reconnects. Remaining runs in the same chain or session are cancelled through the normal failure flow. ## Slack And Webhook Alerts When **Downed machine alerts** is enabled, confirmed run-time detection immediately publishes the existing `downed_machines` event. Its `reason` identifies the Cyberdriver outage. The payload still contains the organization's full current set of monitored downed machines. The recurring monitor is a fallback, not a second notification. Cyberdesk records the alert transition before publishing, so a monitor check one second later sees the machine as already down and does not send a duplicate event. If immediate delivery fails, Cyberdesk releases the alert claim so the next monitor check can retry it. See [Downed Machines Webhook](/webhooks/downed-machines) to connect alerts to Slack or another destination. ## Respond To A Confirmed Failure 1. Open the affected Windows desktop and repair or restart Cyberdriver. 2. Verify that the desktop reconnects in Dashboard -> Desktops. 3. Confirm that basic desktop actions work. Cyberdesk also does not send a recovery webhook. A successful run-start dimensions check clears the internal alert state, allowing a later outage to notify you again. ## Scheduled Monitoring The recurring downed-machines monitor continues to check opted-in desktops even when no run is active. It uses the same maximum of three attempts, 10-second probe timeouts, and 2s/4s spacing between failures. The first successful dimensions response stops the check. Scheduled monitoring sends alerts but does not change desktop state or remove desktops from pools. # Clear All Cyberdriver Traces Source: https://docs.cyberdesk.io/cyberdriver/clear-cyberdriver-traces One-liner PowerShell script to fully remove Cyberdriver before a clean reinstall If Cyberdriver ever gets into a bad state, for example it says it is "already running", fails to start, or a self-update or reinstall leaves broken remnants behind, the fastest fix is to wipe every trace of it and then run the install script again. The one-liner below stops any running Cyberdriver processes and services, removes both the legacy Python folders (`.cyberdriver`) and the Cyberdriver 1.x beta install (Program Files, services, registry, shortcuts, scheduled tasks), cleans up the user `PATH`, and drops the Windows Defender exclusion. It is safe to run whether you installed legacy Cyberdriver, the 1.x beta, or both. Open PowerShell as Administrator (Administrator is needed to remove the 1.x beta service and machine-wide entries) and paste this single command: ```powershell theme={null} Get-Process -Name "Cyberdriver","cyberdriver" -EA SilentlyContinue | Stop-Process -Force -EA SilentlyContinue; Get-Service -Name "Cyberdriver","Cyberdriver Service" -EA SilentlyContinue | ForEach-Object { Stop-Service $_.Name -Force -EA SilentlyContinue; sc.exe delete $_.Name }; @("$env:USERPROFILE\.cyberdriver","$env:LOCALAPPDATA\.cyberdriver","$env:APPDATA\.cyberdriver","$env:ProgramFiles\Cyberdriver","${env:ProgramFiles(x86)}\Cyberdriver","$env:APPDATA\Cyberdriver","$env:APPDATA\Cyberdesk","$env:LOCALAPPDATA\Cyberdriver","$env:LOCALAPPDATA\Cyberdesk","$env:ProgramData\Cyberdriver","$env:ProgramData\Cyberdesk","C:\Windows\System32\config\systemprofile\AppData\Roaming\Cyberdriver","C:\Windows\System32\config\systemprofile\AppData\Roaming\Cyberdesk","C:\Windows\System32\config\systemprofile\AppData\Local\Cyberdriver","C:\Windows\System32\config\systemprofile\AppData\Local\Cyberdesk") | Where-Object { Test-Path $_ } | ForEach-Object { Remove-Item -LiteralPath $_ -Recurse -Force -EA SilentlyContinue }; foreach ($scope in 'User','Machine') { try { $p=[Environment]::GetEnvironmentVariable('Path',$scope); if($p){ [Environment]::SetEnvironmentVariable('Path', (($p -split ';' | Where-Object { $_ -and $_ -inotmatch '\\\.?cyberd(river|esk)' }) -join ';'), $scope) } } catch {} }; Remove-MpPreference -ExclusionPath "$env:LOCALAPPDATA\.cyberdriver\_pyinstaller" -EA SilentlyContinue; @("HKCU:\Software\Cyberdriver","HKCU:\Software\Cyberdesk","HKLM:\Software\Cyberdriver","HKLM:\Software\Cyberdesk","HKLM:\Software\WOW6432Node\Cyberdriver","HKLM:\Software\WOW6432Node\Cyberdesk","HKCU:\Software\Classes\cyberdriver","HKLM:\Software\Classes\cyberdriver") | Where-Object { Test-Path $_ } | ForEach-Object { Remove-Item $_ -Recurse -Force -EA SilentlyContinue }; Get-ChildItem "HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*","HKLM:\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*" -EA SilentlyContinue | Where-Object { $_.GetValue("DisplayName") -like "*Cyberdriver*" -or $_.GetValue("DisplayName") -like "*Cyberdesk*" } | ForEach-Object { Remove-Item $_.PSPath -Recurse -Force -EA SilentlyContinue }; Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs","$env:ProgramData\Microsoft\Windows\Start Menu\Programs" -Filter "*Cyberdriver*" -Recurse -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs","$env:ProgramData\Microsoft\Windows\Start Menu\Programs" -Filter "*Cyberdesk*" -Recurse -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:USERPROFILE\Desktop","$env:PUBLIC\Desktop" -Filter "*Cyberdriver*.lnk" -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:USERPROFILE\Desktop","$env:PUBLIC\Desktop" -Filter "*Cyberdesk*.lnk" -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ScheduledTask -TaskName "*Cyberdriver*" -EA SilentlyContinue | Unregister-ScheduledTask -Confirm:$false; Get-ScheduledTask -TaskName "*Cyberdesk*" -EA SilentlyContinue | Unregister-ScheduledTask -Confirm:$false; Write-Host "Cyberdriver cleanup complete! Restart if an 'already running' message persists, then run the install script again." -ForegroundColor Green ``` If the cleanup finishes but you still see an "already running" message, reboot the machine to release any lingering handles. Run the install script again from the [Cyberdriver Quickstart](/cyberdriver/quickstart). Use the [Legacy Cyberdriver](/cyberdriver/legacy-cyberdriver) installer for the stable Python client, or the 1.x beta installer if you need the Windows service path. This is a full uninstall. It removes saved configuration, including the machine fingerprint, so the machine will re-register in the Cyberdesk dashboard as a new desktop after you reinstall and rejoin. ## What the one-liner removes * Running `cyberdriver` / `Cyberdriver` processes. * The Cyberdriver 1.x beta Windows service (`Cyberdriver`, `Cyberdriver Service`). * Legacy Python folders: `%USERPROFILE%\.cyberdriver`, `%LOCALAPPDATA%\.cyberdriver`, `%APPDATA%\.cyberdriver`. * 1.x beta install and data folders under `Program Files`, `AppData`, `ProgramData`, and the system profile. * Cyberdriver entries from both the user and machine `PATH` (the legacy `.cyberdriver` folder and the 1.x beta `Program Files\Cyberdriver` install directory). * The Windows Defender exclusion for the legacy PyInstaller directory. * Registry keys under `HKCU`/`HKLM` for Cyberdriver and Cyberdesk, plus uninstall entries. * Start menu and desktop shortcuts. * Scheduled tasks created by Cyberdriver (including self-update restart tasks). # CLI and Machine Images Source: https://docs.cyberdesk.io/cyberdriver/cli Use Cyberdriver from the command line for AMIs, templates, and cloned fleets The Cyberdriver CLI is useful when you bake a Windows AMI or golden template, then clone many VMs from it. The main rule is simple: install Cyberdriver in the image, but connect each cloned VM with its own API key and identity when it first boots. ## Install on Windows Use the PowerShell installer script in [Cyberdriver Quickstart](/cyberdriver/quickstart#windows-install). That page is the source of truth for installing Cyberdriver, adding it to `PATH`, and verifying the CLI. The current Windows installer requires Administrator because it installs Cyberdriver as a system service. You can run the Cyberdriver app as a normal user after installation, but a fully non-admin Windows install is not currently supported by the public Cyberdriver MSI. ## Common commands After using the installer script above, restart PowerShell and `Cyberdriver.exe` should be available from any directory: ```powershell theme={null} Cyberdriver.exe --version Cyberdriver.exe status Cyberdriver.exe config-print Cyberdriver.exe logs --tail 65536 Cyberdriver.exe stop ``` `config-print` shows the current Cyberdesk environment, API base, Cyberdesk machine fingerprint, RustDesk peer ID, and whether the API key is configured. For scripts, resolve the executable from `PATH` instead of assuming a drive letter: ```powershell theme={null} $Cyberdriver = (Get-Command Cyberdriver.exe -ErrorAction Stop).Source ``` ## Join Cyberdesk Run this on the VM after it boots: ```powershell theme={null} $Cyberdriver = (Get-Command Cyberdriver.exe -ErrorAction Stop).Source & $Cyberdriver join --secret YOUR_API_KEY ``` For dev: ```powershell theme={null} & $Cyberdriver join --secret YOUR_API_KEY --env dev ``` For a custom API host: ```powershell theme={null} & $Cyberdriver join --secret YOUR_API_KEY --api-base wss://your-api.example.com ``` ## Build an AMI or Image Install the Cyberdriver MSI while preparing your base Windows image. Do not join Cyberdesk in the base image unless you generate a new identity before each clone starts. Configure your launch script, user data, or image init process to run: ```powershell theme={null} $Cyberdriver = (Get-Command Cyberdriver.exe -ErrorAction Stop).Source & $Cyberdriver join --secret $env:CYBERDESK_API_KEY --new-identity ``` `--new-identity` resets both pieces of machine identity: the Cyberdesk fingerprint and the RustDesk peer ID/keypair. Use this for AMIs, templates, and cloned hosts. Reinstalling or updating Cyberdriver without this flag preserves identity. Pass a name when the VM first joins: ```powershell theme={null} & $Cyberdriver join --secret $env:CYBERDESK_API_KEY --name "worker-$env:COMPUTERNAME" ``` Names are optional but make large fleets much easier to manage. After boot, check: ```powershell theme={null} & $Cyberdriver status & $Cyberdriver config-print ``` Then confirm the desktop appears in the Cyberdesk dashboard. ## Remote keepalive from a host If a host machine runs the remote desktop software for many VMs, you can register that host as the remote keepalive machine for a VM: ```powershell theme={null} & $Cyberdriver join --secret YOUR_API_KEY --register-as-keepalive-for ``` The same value can be configured in **Cyberdriver → Settings → Network → Remote keepalive for**. See [Keepalive](/cyberdriver/keepalive) for the full host + VM explanation. ## Useful flags | Flag | Use | | ----------------------------- | --------------------------------------------------------------------------------- | | `--secret` | Cyberdesk organization API key. | | `--name` | Optional dashboard display name. | | `--new-identity` | Generate a new Cyberdesk fingerprint and RustDesk peer ID/keypair before joining. | | `--env dev` | Point Cyberdriver at Cyberdesk development. | | `--api-base` | Custom Cyberdesk tunnel API base. | | `--register-as-keepalive-for` | Link this host as remote keepalive for a main machine ID. | ## Troubleshooting If cloned VMs overwrite each other in the dashboard, the image was cloned with persisted identity. Run: ```powershell theme={null} $Cyberdriver = (Get-Command Cyberdriver.exe -ErrorAction Stop).Source & $Cyberdriver join --secret YOUR_API_KEY --new-identity ``` If the VM does not appear connected, check logs: ```powershell theme={null} & $Cyberdriver logs --tail 65536 ``` If you use a host-level remote keepalive Cyberdriver, make sure the host and VM API keys belong to the same Cyberdesk organization. # Custom Hosts Source: https://docs.cyberdesk.io/cyberdriver/custom-host Route Cyberdriver traffic through a custom proxy Cyberdriver connects outbound to Cyberdesk over HTTPS/WebSocket. Some organizations need that traffic to go through a company-controlled domain for firewall allowlisting or traffic inspection. ## Requirements Your proxy must support long-lived WebSocket connections: * HTTP Upgrade headers * bidirectional streaming * long idle timeouts, ideally 24 hours * TLS to the public Cyberdesk endpoint ## Nginx example ```nginx theme={null} server { listen 443 ssl; server_name automation.yourcompany.com; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.pem; location / { proxy_pass https://api.cyberdesk.io; proxy_ssl_server_name on; proxy_set_header Host api.cyberdesk.io; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; proxy_read_timeout 86400s; proxy_send_timeout 86400s; } } ``` ## Configure Cyberdriver For CLI installs: ```powershell theme={null} $Cyberdriver = (Get-Command Cyberdriver.exe -ErrorAction Stop).Source & $Cyberdriver join --secret YOUR_API_KEY --api-base https://automation.yourcompany.com ``` For the Windows MSI flow, use the normal Cyberdriver settings unless the Cyberdesk team gives you a custom host value. # Desktop Tools Source: https://docs.cyberdesk.io/cyberdriver/desktop-tools Control a connected desktop from Cyberdesk Desktop Tools is the dashboard control surface for a connected Cyberdriver machine. It sends requests through Cyberdesk's authenticated tunnel to the local Cyberdriver service. ## Supported actions ### Display * Take screenshots * Read current display dimensions * Set Windows resolution to 1024×768 from the System tab when supported ### Mouse * Move the cursor * Left, right, middle, double, and triple click * Drag from a start coordinate to an end coordinate * Scroll up, down, left, or right ### Keyboard * Type text * Press key combinations such as `ctrl+c`, `alt+tab`, `enter`, and `esc` * Use quick buttons for common keys **Key syntax.** Use `+` to press keys **together** as one chord, e.g. `ctrl+a` or `alt+tab`. A chord can combine multiple modifiers (`ctrl`, `alt`, `shift`, `win`) but only one non-modifier key. Use **spaces** to press keys **one after another**, e.g. `down down down` taps Down three times, or `ctrl+c ctrl+v`. Joining repeated or multiple non-modifier keys with `+` (e.g. `down+down+down`) is invalid — use spaces instead. ### Clipboard * Copy selected text from the remote desktop and return it to Cyberdesk ### Files * List directories * Read files as base64 * Write or append files ### Shell * Run PowerShell commands on Windows * Reuse a PowerShell session when the endpoint supports it ## Coordinate model Cyberdriver normalizes screenshots and coordinates so that Desktop Tools clicks map to the visible screenshot. On Windows, Cyberdriver maps actions to display pixels and handles the translation internally. If clicks look offset, collect diagnostics and include the display dimensions plus screenshot size. ## Login screen access On Windows, install Cyberdriver as a service. Service mode lets Cyberdriver work before an interactive user logs in and enables access to the Windows login screen. # Diagnostics and Logs Source: https://docs.cyberdesk.io/cyberdriver/diagnostics Collect logs and debug state from Cyberdriver Diagnostics are critical during beta. Cyberdriver exposes local and remote diagnostics so support can understand what failed without guessing. ## From the Cyberdriver app Open **Settings → Network** and click **Copy diagnostics**. This copies a JSON payload with local configuration, service status, server targets, permissions, API key configured state, and other useful state. ## From the dashboard Cyberdesk can call: ```http theme={null} GET /internal/diagnostics ``` through the machine tunnel. The response includes: * Cyberdriver version * platform and hostname * Cyberdesk machine fingerprint * RustDesk peer ID * API/hbbs/hbbr server configuration * Cyberdesk tunnel API base * keepalive state and last activity time * display dimensions * log directory and latest log tail * virtual display platform additions on Windows ## From the CLI ```bash theme={null} cyberdriver logs ``` Useful options: ```bash theme={null} cyberdriver logs --tail 200000 cyberdriver logs --follow cyberdriver logs --path "C:\path\to\file.log" ``` ## What to send support When reporting a bug, include: 1. The diagnostics JSON 2. The latest log tail 3. Your OS version 4. Whether Cyberdriver is installed as a Windows service 5. The exact action that failed, such as screenshot, click, type, connect, or API key save # Display Reliability Source: https://docs.cyberdesk.io/cyberdriver/display-reliability How Cyberdriver handles RDP disconnects and virtual displays The legacy Python Cyberdriver had two separate features for display reliability: persistent display installation and black-screen recovery. Cyberdriver 1.x beta uses RustDesk's display stack instead. ## What changed in Cyberdriver 1.x beta * RustDesk can create and manage virtual displays. * Windows service mode can access the login screen. * The client defaults new Windows sessions toward 1024×768 and adaptive scaling. * Cyberdriver diagnostics expose Windows virtual-display platform additions so support can see what the display layer is doing. ## RDP disconnects When RDP disconnects, Windows may remove or change the active display. RustDesk's display service can enumerate displays and, when needed, create a headless virtual display. This is a better foundation than legacy screenshot-based black-screen detection. ## If you see a black screen 1. Open Desktop Tools and check display dimensions. 2. Use diagnostics to inspect `display_dimensions` and `platform_additions`. 3. Try reconnecting the remote session. 4. If needed, use the dashboard's Fix Black Screen tool for legacy machines. ## Persistent display flag The old `--add-persistent-display` flag is legacy-only. Do not use it for Cyberdriver 1.x beta unless support explicitly asks you to use an old Python agent. # Keepalive Source: https://docs.cyberdesk.io/cyberdriver/keepalive Keep local and remote desktop sessions active while idle Keepalive helps prevent idle locks and session timeouts while a machine is connected but not actively receiving work. ## How it works in Cyberdriver 1.x beta Cyberdriver tracks the last request received from Cyberdesk. If the machine is idle for the configured threshold, Cyberdriver performs a tiny cursor nudge and restores the cursor position. This is intentionally less disruptive than the legacy Python agent's click-and-type keepalive behavior. ## Enable or disable keepalive Open **Cyberdriver → Settings → Network** and toggle **Cyberdesk keepalive**. Keepalive is enabled by default unless disabled locally. ## Remote Keepalive (Host + VM) When automating a Windows VM managed by remote desktop software (RDP, Avatara, AnyDesk, and similar tools), the VM often locks or shuts off after inactivity. Because that idle timer is enforced by the remote desktop layer, running keepalive inside the VM may not help. Remote Keepalive solves this by running a second Cyberdriver on the **host** where the remote desktop software runs. The host Cyberdriver keeps the VM session active while the main Cyberdriver inside the VM is idle. This matters because it keeps the VM ready for work without repeatedly passing 2FA every time you want to run a workflow. ### Quick setup 1. Install and connect Cyberdriver inside the VM as usual. 2. Copy the VM's Cyberdesk machine ID from the Cyberdesk dashboard. 3. Install Cyberdriver on the host machine. 4. Open **Cyberdriver → Settings → Network** on the host. 5. Enter the VM's machine ID in **Remote keepalive for**. 6. Click **Save** and make sure **Cyberdesk keepalive** is enabled. You can also configure the host from the CLI: ```bash theme={null} cyberdriver join --secret YOUR_API_KEY --keepalive --register-as-keepalive-for ``` ### What happens under the hood * The host Cyberdriver links itself to your VM's Cyberdriver. * Cyberdesk enforces that both machines belong to the same organization. * Self-links are rejected. * The host Cyberdriver will not interfere while a workflow runs on the VM. * Remote activity resets the linked host keepalive machine's idle timer with a small random jitter around the keepalive threshold. * If the host Cyberdriver disconnects, the link is removed automatically. When it reconnects, the link is re-established. ## API coordination Cyberdesk coordinates remote keepalive internally through Cyberdriver's tunnel endpoints. You normally do not need to call these directly. Any `/computer/*` request also refreshes activity automatically. ## When to use it Use keepalive for: * long-running workflows * hosted Windows desktops that lock quickly * machines where remote desktop software enforces inactivity timers Keepalive is not a substitute for a healthy display/session. For RDP-related display loss, see [Display reliability](/cyberdriver/display-reliability). # Legacy Cyberdriver Source: https://docs.cyberdesk.io/cyberdriver/legacy-cyberdriver Stable Python-based Cyberdriver client This page documents the legacy Python Cyberdriver (`0.0.x`). It is currently the recommended stable install path for most desktop automation use cases, especially when you do not need Windows login-screen access. Use the [Cyberdriver 1.x beta](/cyberdriver/quickstart#beta-cyberdriver-1x-windows-install) only when you specifically need the newer Windows service path. ## What is legacy Cyberdriver? Legacy Cyberdriver is a small Python-based executable that connects your desktop to Cyberdesk, enabling AI-powered automation of desktop tasks. It provides a secure bridge between Cyberdesk's cloud infrastructure and your local Windows machine. ### Key features * **Local HTTP server** - Endpoints for display, keyboard, mouse, clipboard, file, and shell control. * **WebSocket tunnel** - Secure outbound connection to Cyberdesk Cloud. * **Works on physical and virtual machines** - Run on a local Windows computer or Windows VM. * **Stealth mode** - Runs invisibly in the background on Windows. * **Keepalive mode** - Prevents idle timeouts and session locks. * **Black screen recovery** - Automatically recovers from RDP display issues. * **Persistent virtual display** - Can start a virtual display that runs on the console session. * **Remote updates** - Update supported legacy versions from the dashboard. ### No firewall configuration required Legacy Cyberdriver does not require inbound firewall ports. It connects outbound to Cyberdesk Cloud through a secure reverse tunnel. * Works behind corporate firewalls. * No router configuration needed. * No exposed ports on your machine. * Secure, encrypted connection. ## Installation ### Windows PowerShell installation The easiest way to install legacy Cyberdriver on Windows is using the PowerShell installer. The script below pins the stable legacy download to `v0.0.41`. ```powershell theme={null} # Create tool directory $toolDir = "$env:USERPROFILE\.cyberdriver" New-Item -ItemType Directory -Force -Path $toolDir # Download cyberdriver try { Invoke-WebRequest -Uri "https://github.com/cyberdesk-hq/cyberdriver/releases/download/v0.0.41/cyberdriver.exe" -OutFile "$toolDir\cyberdriver.exe" -ErrorAction Stop } catch { Write-Host "ERROR: Failed to download Cyberdriver. If Cyberdriver is already running, run 'cyberdriver stop' first. Otherwise, check your internet connection and try again." -ForegroundColor Red return } # Verify installation if (Test-Path "$toolDir\cyberdriver.exe") { $fileSize = (Get-Item "$toolDir\cyberdriver.exe").Length if ($fileSize -gt 34MB) { # Add to PATH if not already there $userPath = [Environment]::GetEnvironmentVariable("Path", "User") if ($userPath -notlike "*$toolDir*") { [Environment]::SetEnvironmentVariable("Path", $userPath + ";" + $toolDir, "User") } Write-Host "Cyberdriver installed successfully! You may need to restart your terminal for PATH changes to take effect." } else { Write-Host "ERROR: Download appears incomplete (file too small). Please try again." -ForegroundColor Red } } else { Write-Host "ERROR: Download failed. Please try again." -ForegroundColor Red } ``` Close and reopen PowerShell for the PATH changes to take effect. Once restarted, you can connect to Cyberdesk Cloud. Legacy Cyberdriver automatically disables PowerShell's QuickEdit Mode on startup. This prevents the terminal from freezing if you accidentally click it while Cyberdriver is running. ## Getting started ### Connect to Cyberdesk Cloud To connect your desktop to Cyberdesk for remote automation: ```bash theme={null} cyberdriver join --secret YOUR_API_KEY ``` Replace `YOUR_API_KEY` with your API key from the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard). On Windows, legacy Cyberdriver runs in stealth mode by default, so `cyberdriver join` continues in the background with no visible window. You'll see a confirmation message, then your PowerShell prompt returns immediately. ### Stopping Cyberdriver To stop Cyberdriver: ```bash theme={null} cyberdriver stop ``` Since legacy Cyberdriver runs invisibly in the background, `cyberdriver stop` is the recommended way to shut it down. You can also end `cyberdriver.exe` in Task Manager. ### Naming a machine at join time Pass `--name ` to `cyberdriver join` to set a human-readable display name for the machine. The value lands on `Machine.name` in the dashboard and is searchable via the API. ```bash theme={null} cyberdriver join --secret YOUR_API_KEY --name "i-0abc123def456" ``` Behavior: * **First join** (new fingerprint): the machine is created with `name` set to your value. * **Subsequent joins** (same fingerprint, different `--name`): the existing machine's name is updated. * **Subsequent joins without `--name`**: the existing name is preserved. * **Validation:** printable ASCII only, max 128 characters, surrounding whitespace trimmed. * **Uniqueness is not enforced.** Two machines can share the same name. #### Parallel provisioning recipe When you boot many VMs from the same image, every fresh registration starts as `name: null` and the only differentiator is the Cyberdesk machine ID. Push a name you already know per VM and look it up afterwards. For example, use the cloud-provider instance ID, hostname, or workflow run ID. ```bash theme={null} cyberdriver join --secret "$CYBERDESK_API_KEY" --name "$INSTANCE_ID" ``` ```bash theme={null} curl -H "Authorization: Bearer $CYBERDESK_API_KEY" \ "https://api.cyberdesk.io/v1/machines?name=$INSTANCE_ID" ``` The response is a paginated list. Pick a unique-per-VM string to make this a one-shot lookup. With the Cyberdesk-issued machine ID in hand, you can target the VM for workflow runs, proxy requests, and automation. ## Common issues ### Cyberdriver can't click or interact with certain apps If Cyberdriver appears to be running but can't click on or interact with specific applications, especially legacy enterprise software or system utilities, this is typically a permissions issue. Solution: run Cyberdriver from an Administrator PowerShell terminal: 1. Right-click PowerShell and select **Run as Administrator**. 2. Navigate to your desired directory. 3. Run `cyberdriver join --secret YOUR_API_KEY`. Some desktop applications require admin privileges to receive input from other processes. For regular user-level applications, you can run Cyberdriver normally without admin privileges. ### Connection issues If Cyberdriver fails to connect or keeps disconnecting: 1. Verify your API key. 2. Check your internet connection. 3. Run `cyberdriver logs` and contact support with the output. ### Performance issues If automation seems slow or unresponsive: 1. Close unnecessary applications. 2. Check CPU usage. 3. Disable Windows animations to speed up window switching. ### TLS certificate issues Legacy Cyberdriver uses your system certificate store by default, which works automatically on most machines. It also bundles `certifi` as a fallback for machines missing standard root CAs. If you see TLS/SSL certificate errors: 1. Corporate networks with SSL inspection should work if your IT department's certificate is installed in the OS trust store. 2. Fresh Windows machines may be missing root certificates. Install Windows updates or contact support if `certifi` fallback does not resolve it. ### Space key not working in certain apps Some applications, particularly legacy enterprise apps, may not respond correctly to space key input. Try the experimental space flag: ```bash theme={null} cyberdriver join --secret YOUR_API_KEY --experimental-space ``` This sends space using a virtual key code (`VK_SPACE`) instead of the hardware scan code. ### Failed to load Python DLL during start or reinstall If you see an error like this: ```text theme={null} Failed to load Python DLL 'C:\Users\Administrator\AppData\Local\.cyberdriver\_pyinstaller\_MEI42282\python39.dll'. LoadLibrary: The specified module could not be found. ``` The installation is likely corrupted. Fully remove all Cyberdriver remnants, reboot, then reinstall. Run in PowerShell: ```powershell theme={null} $ErrorActionPreference = "Stop" # Stop running instances try { cyberdriver stop --force --timeout 3 | Out-Null } catch {} Get-Process cyberdriver -ErrorAction SilentlyContinue | Stop-Process -Force -ErrorAction SilentlyContinue # Stop updater/launcher leftovers if present Get-CimInstance Win32_Process -ErrorAction SilentlyContinue | Where-Object { $_.CommandLine -match "cyberdriver-updater|cyberdriver-update\.exe|launch-hidden\.ps1|launch-hidden\.vbs" } | ForEach-Object { Stop-Process -Id $_.ProcessId -Force -ErrorAction SilentlyContinue } # Remove any leftover scheduled restart tasks from self-update Get-ScheduledTask -TaskName "CyberdriverRestart_*" -ErrorAction SilentlyContinue | Unregister-ScheduledTask -Confirm:$false -ErrorAction SilentlyContinue # Remove all known Cyberdriver folders $paths = @( "$env:USERPROFILE\.cyberdriver", "$env:LOCALAPPDATA\.cyberdriver", "$env:APPDATA\.cyberdriver" ) foreach ($p in $paths) { if (Test-Path $p) { Remove-Item $p -Recurse -Force -ErrorAction SilentlyContinue } } # Remove old PATH entry for clean reinstall $toolDir = "$env:USERPROFILE\.cyberdriver" $userPath = [Environment]::GetEnvironmentVariable("Path", "User") if ($userPath) { $newPath = (($userPath -split ';') | Where-Object { $_ -and ($_ -ne $toolDir) }) -join ';' [Environment]::SetEnvironmentVariable("Path", $newPath, "User") } Write-Host "Cyberdriver cleanup complete. Reboot, then reinstall." ``` ### Clear all Cyberdriver traces (reset before reinstall) If Cyberdriver gets stuck, keeps saying it is "already running", or a reinstall leaves broken remnants behind, run the one-liner below in PowerShell to wipe every trace of it, then run the install script again. This is often the quickest fix. ```powershell theme={null} Get-Process -Name "Cyberdriver","cyberdriver" -EA SilentlyContinue | Stop-Process -Force -EA SilentlyContinue; Get-Service -Name "Cyberdriver","Cyberdriver Service" -EA SilentlyContinue | ForEach-Object { Stop-Service $_.Name -Force -EA SilentlyContinue; sc.exe delete $_.Name }; @("$env:USERPROFILE\.cyberdriver","$env:LOCALAPPDATA\.cyberdriver","$env:APPDATA\.cyberdriver","$env:ProgramFiles\Cyberdriver","${env:ProgramFiles(x86)}\Cyberdriver","$env:APPDATA\Cyberdriver","$env:APPDATA\Cyberdesk","$env:LOCALAPPDATA\Cyberdriver","$env:LOCALAPPDATA\Cyberdesk","$env:ProgramData\Cyberdriver","$env:ProgramData\Cyberdesk","C:\Windows\System32\config\systemprofile\AppData\Roaming\Cyberdriver","C:\Windows\System32\config\systemprofile\AppData\Roaming\Cyberdesk","C:\Windows\System32\config\systemprofile\AppData\Local\Cyberdriver","C:\Windows\System32\config\systemprofile\AppData\Local\Cyberdesk") | Where-Object { Test-Path $_ } | ForEach-Object { Remove-Item -LiteralPath $_ -Recurse -Force -EA SilentlyContinue }; foreach ($scope in 'User','Machine') { try { $p=[Environment]::GetEnvironmentVariable('Path',$scope); if($p){ [Environment]::SetEnvironmentVariable('Path', (($p -split ';' | Where-Object { $_ -and $_ -inotmatch '\\\.?cyberd(river|esk)' }) -join ';'), $scope) } } catch {} }; Remove-MpPreference -ExclusionPath "$env:LOCALAPPDATA\.cyberdriver\_pyinstaller" -EA SilentlyContinue; @("HKCU:\Software\Cyberdriver","HKCU:\Software\Cyberdesk","HKLM:\Software\Cyberdriver","HKLM:\Software\Cyberdesk","HKLM:\Software\WOW6432Node\Cyberdriver","HKLM:\Software\WOW6432Node\Cyberdesk","HKCU:\Software\Classes\cyberdriver","HKLM:\Software\Classes\cyberdriver") | Where-Object { Test-Path $_ } | ForEach-Object { Remove-Item $_ -Recurse -Force -EA SilentlyContinue }; Get-ChildItem "HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*","HKLM:\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*" -EA SilentlyContinue | Where-Object { $_.GetValue("DisplayName") -like "*Cyberdriver*" -or $_.GetValue("DisplayName") -like "*Cyberdesk*" } | ForEach-Object { Remove-Item $_.PSPath -Recurse -Force -EA SilentlyContinue }; Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs","$env:ProgramData\Microsoft\Windows\Start Menu\Programs" -Filter "*Cyberdriver*" -Recurse -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs","$env:ProgramData\Microsoft\Windows\Start Menu\Programs" -Filter "*Cyberdesk*" -Recurse -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:USERPROFILE\Desktop","$env:PUBLIC\Desktop" -Filter "*Cyberdriver*.lnk" -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:USERPROFILE\Desktop","$env:PUBLIC\Desktop" -Filter "*Cyberdesk*.lnk" -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ScheduledTask -TaskName "*Cyberdriver*" -EA SilentlyContinue | Unregister-ScheduledTask -Confirm:$false; Get-ScheduledTask -TaskName "*Cyberdesk*" -EA SilentlyContinue | Unregister-ScheduledTask -Confirm:$false; Write-Host "Cyberdriver cleanup complete! Restart if an 'already running' message persists, then run the install script again." -ForegroundColor Green ``` For a full breakdown of what this removes, see [Clear All Cyberdriver Traces](/cyberdriver/clear-cyberdriver-traces). ## Configuration Legacy Cyberdriver stores persistent configuration at: * `%LOCALAPPDATA%\.cyberdriver\config.json` The configuration file includes: * **Version**: Current Cyberdriver version. * **Fingerprint**: Unique Cyberdesk machine identifier. Connection details such as the current host, auth secret, and active tunnel state are runtime state, not fields stored in `config.json`. ## API endpoints When you run `cyberdriver join`, legacy Cyberdriver opens a local HTTP server on port `3000` by default. If that port is already in use, it falls forward to the next available local port. It also creates a secure reverse tunnel to Cyberdesk Cloud, allowing remote access without opening inbound ports. ### Display * `GET /computer/display/screenshot` - Capture screen. * Query params: `width`, `height`, `mode` (`exact`, `aspect_fit`, `aspect_fill`) * Returns: PNG image. * `GET /computer/display/dimensions` - Get screen dimensions. ### Keyboard * `POST /computer/input/keyboard/type` - Type text. * Body: `{"text": "Hello world"}` * `POST /computer/input/keyboard/key` - Send key combinations. * Body: `{"text": "ctrl+c"}` ### Clipboard * `POST /computer/copy_to_clipboard` - Send `Ctrl+C` and return clipboard contents keyed by the requested name. * Body: `{"text": "account_number"}` ### Mouse * `GET /computer/input/mouse/position` - Get cursor position. * `POST /computer/input/mouse/move` - Move cursor instantly. * `POST /computer/input/mouse/click` - Click mouse button. * `POST /computer/input/mouse/drag` - Drag from a start position to an end position. * `POST /computer/input/mouse/scroll` - Scroll mouse wheel vertically or horizontally. ### File system * `GET /computer/fs/list` - List directory contents. * `GET /computer/fs/read` - Read file contents. * `POST /computer/fs/write` - Write file contents. ### PowerShell * `POST /computer/shell/powershell/exec` - Execute PowerShell commands. * `POST /computer/shell/powershell/session` - Compatibility endpoint for create/destroy session IDs. ## Security considerations Legacy Cyberdriver provides remote access to your desktop. Keep your API key secret, use trusted machines, monitor dashboard activity, and update regularly for security fixes. ## Legacy feature details Legacy-specific features such as stealth mode, console protection, keepalive mode, black screen recovery, persistent virtual display, coordinate capture, and remote updates are kept here for compatibility. Some newer Cyberdriver docs describe the RustDesk-based `1.x` client and may not apply to this Python executable. If you are intentionally running legacy Cyberdriver and need exact behavior for one of these features, contact the founders at [founders@cyberdesk.io](mailto:founders@cyberdesk.io). ## Next steps Create and run your first workflow. Learn how to trigger runs programmatically. Create workflows and manage machines. Get help from the Cyberdesk team. # Cyberdriver Quickstart Source: https://docs.cyberdesk.io/cyberdriver/quickstart Install Cyberdriver and connect a desktop to Cyberdesk Cyberdriver is the desktop companion app that connects a machine to Cyberdesk. For most new installs, start with **Legacy Cyberdriver**. It is the stable option for desktop automation when you do not need Windows login-screen access. Use the [Cyberdriver 1.x beta](#beta-cyberdriver-1x-windows-install) only when you specifically need the newer Windows service path for boot-time startup or Windows login-screen control. ## Stable install: Legacy Cyberdriver Legacy Cyberdriver is the default stable install path for desktop automation today. It does not require inbound firewall ports and works well for machines where a user session is already available. Open PowerShell and run this installer script. It downloads legacy Cyberdriver `v0.0.41`, adds it to your user `PATH`, and verifies the download. ```powershell theme={null} # Create tool directory $toolDir = "$env:USERPROFILE\.cyberdriver" New-Item -ItemType Directory -Force -Path $toolDir # Download cyberdriver try { Invoke-WebRequest -Uri "https://github.com/cyberdesk-hq/cyberdriver/releases/download/v0.0.41/cyberdriver.exe" -OutFile "$toolDir\cyberdriver.exe" -ErrorAction Stop } catch { Write-Host "ERROR: Failed to download Cyberdriver. If Cyberdriver is already running, run 'cyberdriver stop' first. Otherwise, check your internet connection and try again." -ForegroundColor Red return } # Verify installation if (Test-Path "$toolDir\cyberdriver.exe") { $fileSize = (Get-Item "$toolDir\cyberdriver.exe").Length if ($fileSize -gt 34MB) { # Add to PATH if not already there $userPath = [Environment]::GetEnvironmentVariable("Path", "User") if ($userPath -notlike "*$toolDir*") { [Environment]::SetEnvironmentVariable("Path", $userPath + ";" + $toolDir, "User") } Write-Host "Cyberdriver installed successfully! You may need to restart your terminal for PATH changes to take effect." } else { Write-Host "ERROR: Download appears incomplete (file too small). Please try again." -ForegroundColor Red } } else { Write-Host "ERROR: Download failed. Please try again." -ForegroundColor Red } ``` Close and reopen PowerShell, then join your machine with an organization API key from the Cyberdesk dashboard. ```bash theme={null} cyberdriver join --secret YOUR_API_KEY ``` After the key is saved, the desktop should appear in **Cyberdesk -> Desktops**. From the Cyberdesk dashboard, open the desktop and click the **Desktop Tools** button. Verify that it connects and you can control the computer. Now that your desktop is connected, [create your first workflow and then create your first run](/quickstart). Let us know if you have any trouble. For details and troubleshooting, see [Legacy Cyberdriver](/cyberdriver/legacy-cyberdriver). ## \[BETA] Cyberdriver 1.x Windows install Cyberdriver 1.x beta is based on RustDesk and currently supported for Windows only. Use this beta path when you need Cyberdriver to run as a Windows service, start at boot, or control the Windows login screen. Administrator is required for the current Windows installer because Cyberdriver installs a system service. After installation, users can open and use the Cyberdriver app normally; they do not need to run the app as Administrator day to day. If you are upgrading from a Cyberdriver version older than `1.0.0`, open Windows Task Manager and end all running `Cyberdriver.exe` tasks before running the installer. Open PowerShell as Administrator and run this beta installer script. It downloads Cyberdriver `1.0.2`, installs Cyberdriver as a Windows service, adds Cyberdriver to `PATH`, verifies the CLI, and opens Cyberdriver. ```powershell theme={null} $ErrorActionPreference = "Stop" $MsiUrl = "https://github.com/cyberdesk-hq/cyberdriver-new/releases/download/v1.0.2/Cyberdriver-1.0.2-windows-x64.msi" $MsiPath = Join-Path $env:TEMP "Cyberdriver-1.0.2-windows-x64.msi" $InstallDir = Join-Path $env:ProgramFiles "Cyberdriver" $Cyberdriver = Join-Path $InstallDir "Cyberdriver.exe" function Test-IsAdmin { $identity = [Security.Principal.WindowsIdentity]::GetCurrent() $principal = New-Object Security.Principal.WindowsPrincipal($identity) return $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator) } if (-not (Test-IsAdmin)) { throw "Run PowerShell as Administrator, then rerun this installer." } try { Write-Host "Downloading Cyberdriver..." [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 Invoke-WebRequest -Uri $MsiUrl -OutFile $MsiPath -UseBasicParsing Write-Host "Installing Cyberdriver..." $install = Start-Process msiexec.exe -ArgumentList @("/i", "`"$MsiPath`"", "/qn", "/norestart") -Wait -PassThru if ($install.ExitCode -notin @(0, 3010)) { throw "Cyberdriver MSI install failed with exit code $($install.ExitCode)." } if (-not (Test-Path $Cyberdriver)) { throw "Cyberdriver.exe was not found at $Cyberdriver after install." } $machinePath = [Environment]::GetEnvironmentVariable("Path", "Machine") $pathParts = @($machinePath -split ";" | Where-Object { $_ }) $alreadyOnPath = $pathParts | Where-Object { $_.TrimEnd("\") -ieq $InstallDir.TrimEnd("\") } if (-not $alreadyOnPath) { [Environment]::SetEnvironmentVariable("Path", (($pathParts + $InstallDir) -join ";"), "Machine") } if (($env:Path -split ";") -notcontains $InstallDir) { $env:Path = "$env:Path;$InstallDir" } Write-Host "Verifying Cyberdriver CLI..." & $Cyberdriver --version Write-Host "Opening Cyberdriver..." Start-Process -FilePath $Cyberdriver Write-Host "Cyberdriver install complete. Restart PowerShell to pick up PATH in new shells." } finally { Remove-Item $MsiPath -ErrorAction SilentlyContinue } ``` If Windows warns that the installer is not commonly downloaded, choose **More info**, then **Run anyway**. The beta MSI is not yet code-signed. Open Cyberdriver from the Start menu. In the **Cyberdesk tunnel** card, paste an organization API key from the Cyberdesk dashboard and click **Save**. After the key is saved, the desktop should appear in **Cyberdesk → Desktops**. From the Cyberdesk dashboard, open the desktop and click the **Desktop Tools** button. Verify that it connects and you can control the computer. Cyberdriver will now persist on your machine, even across shutdowns. Now that your desktop is connected, [create your first workflow and then create your first run](/quickstart). Let us know if you have any trouble. ### Why Administrator Is Required For The Beta MSI The Cyberdriver 1.x beta Windows release is service-first. The MSI installs into `Program Files`, registers a Windows service, and adds Cyberdriver to the machine `PATH`, which requires Administrator. This is what enables: * Cyberdriver starting when the machine boots. * Remote access before a user logs in. * Windows login screen access. * More reliable unattended automation after shutdowns and restarts. A fully non-admin Windows setup is not supported by the current public MSI. Cyberdriver's upstream UI has a "Run without install" concept, but Cyberdesk does not currently ship or document a supported portable/non-admin Windows artifact. If we add one later, expect tradeoffs: it would only run while that user session is active, would not start at machine boot, would not control the Windows login screen, and may have weaker access to elevated/UAC contexts. ## Updating Cyberdriver 1.x beta You can update Cyberdriver `1.0.0+` from the Cyberdesk dashboard without manually opening RDP as long as the existing Cyberdriver service is healthy enough for Cyberdesk to connect. We are working on making this process even more seamless. For now, the dashboard update flow is the recommended way to update Cyberdriver `1.0.0+` without manually RDPing into the machine. In the Cyberdesk dashboard, open the desktop and click **Desktop Tools**. For Cyberdriver `1.0.0+`, this opens the live Cyberdriver Web stream by default. If an update is available, the right sidebar shows an **Update** button. Click it to start the guided update flow. Cyberdesk blocks the update if the desktop currently has a running or scheduling run. If no run is active, confirm that no run is about to start on that desktop. Cyberdesk runs the update through the existing Cyberdriver tunnel. If Windows shows an administrator prompt in the stream, approve it. The stream may briefly disconnect while the MSI updates or restarts the service. Cyberdesk waits up to 5 minutes for Cyberdriver to reconnect and report the new version. If verification fails, reopen Desktop Tools after the desktop reconnects and try the update again. ## Troubleshooting ### Clear all Cyberdriver traces and reinstall If Cyberdriver gets stuck, keeps saying it is "already running", or an install or update leaves broken remnants behind, the quickest fix is to wipe every trace of it and run the install script again. Open PowerShell as Administrator and paste this one-liner: ```powershell theme={null} Get-Process -Name "Cyberdriver","cyberdriver" -EA SilentlyContinue | Stop-Process -Force -EA SilentlyContinue; Get-Service -Name "Cyberdriver","Cyberdriver Service" -EA SilentlyContinue | ForEach-Object { Stop-Service $_.Name -Force -EA SilentlyContinue; sc.exe delete $_.Name }; @("$env:USERPROFILE\.cyberdriver","$env:LOCALAPPDATA\.cyberdriver","$env:APPDATA\.cyberdriver","$env:ProgramFiles\Cyberdriver","${env:ProgramFiles(x86)}\Cyberdriver","$env:APPDATA\Cyberdriver","$env:APPDATA\Cyberdesk","$env:LOCALAPPDATA\Cyberdriver","$env:LOCALAPPDATA\Cyberdesk","$env:ProgramData\Cyberdriver","$env:ProgramData\Cyberdesk","C:\Windows\System32\config\systemprofile\AppData\Roaming\Cyberdriver","C:\Windows\System32\config\systemprofile\AppData\Roaming\Cyberdesk","C:\Windows\System32\config\systemprofile\AppData\Local\Cyberdriver","C:\Windows\System32\config\systemprofile\AppData\Local\Cyberdesk") | Where-Object { Test-Path $_ } | ForEach-Object { Remove-Item -LiteralPath $_ -Recurse -Force -EA SilentlyContinue }; foreach ($scope in 'User','Machine') { try { $p=[Environment]::GetEnvironmentVariable('Path',$scope); if($p){ [Environment]::SetEnvironmentVariable('Path', (($p -split ';' | Where-Object { $_ -and $_ -inotmatch '\\\.?cyberd(river|esk)' }) -join ';'), $scope) } } catch {} }; Remove-MpPreference -ExclusionPath "$env:LOCALAPPDATA\.cyberdriver\_pyinstaller" -EA SilentlyContinue; @("HKCU:\Software\Cyberdriver","HKCU:\Software\Cyberdesk","HKLM:\Software\Cyberdriver","HKLM:\Software\Cyberdesk","HKLM:\Software\WOW6432Node\Cyberdriver","HKLM:\Software\WOW6432Node\Cyberdesk","HKCU:\Software\Classes\cyberdriver","HKLM:\Software\Classes\cyberdriver") | Where-Object { Test-Path $_ } | ForEach-Object { Remove-Item $_ -Recurse -Force -EA SilentlyContinue }; Get-ChildItem "HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*","HKLM:\Software\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*" -EA SilentlyContinue | Where-Object { $_.GetValue("DisplayName") -like "*Cyberdriver*" -or $_.GetValue("DisplayName") -like "*Cyberdesk*" } | ForEach-Object { Remove-Item $_.PSPath -Recurse -Force -EA SilentlyContinue }; Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs","$env:ProgramData\Microsoft\Windows\Start Menu\Programs" -Filter "*Cyberdriver*" -Recurse -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:APPDATA\Microsoft\Windows\Start Menu\Programs","$env:ProgramData\Microsoft\Windows\Start Menu\Programs" -Filter "*Cyberdesk*" -Recurse -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:USERPROFILE\Desktop","$env:PUBLIC\Desktop" -Filter "*Cyberdriver*.lnk" -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ChildItem "$env:USERPROFILE\Desktop","$env:PUBLIC\Desktop" -Filter "*Cyberdesk*.lnk" -EA SilentlyContinue | Remove-Item -Force -EA SilentlyContinue; Get-ScheduledTask -TaskName "*Cyberdriver*" -EA SilentlyContinue | Unregister-ScheduledTask -Confirm:$false; Get-ScheduledTask -TaskName "*Cyberdesk*" -EA SilentlyContinue | Unregister-ScheduledTask -Confirm:$false; Write-Host "Cyberdriver cleanup complete! Restart if an 'already running' message persists, then run the install script again." -ForegroundColor Green ``` For a full breakdown of what this removes, see [Clear All Cyberdriver Traces](/cyberdriver/clear-cyberdriver-traces). ### Corporate TLS Inspection and Firewalls Cyberdriver needs outbound access to Cyberdesk Cloud over HTTPS and secure WebSockets. Most enterprise networks work without inbound firewall changes because Cyberdriver initiates outbound connections, but corporate TLS inspection or strict proxy rules can still interrupt the tunnel. Cyberdriver validates TLS certificates. Cyberdesk Cloud uses publicly trusted certificates, and Cyberdriver uses public CA roots plus the Windows certificate store when available. If a firewall or proxy intercepts TLS and re-signs Cyberdesk traffic with a private enterprise CA, Cyberdriver may reject the connection if that CA is not trusted by the service context. Common symptoms include: * Cyberdriver diagnostics show `api_key_configured = true`. * The Cyberdriver service status is `ready`. * The Cyberdesk tunnel stays in `reconnecting`. * `last_error` mentions `invalid peer certificate: UnknownIssuer`. The preferred enterprise setup is to bypass TLS inspection for Cyberdesk Cloud traffic, especially the long-lived WebSocket tunnel. If inspection is mandatory, deploy the organization's private root CA into the Windows trusted certificate store in a way that is available to the Cyberdriver Windows service. Installing a certificate only for the interactive user may not be enough. Do not disable TLS verification for production. Any insecure or allow-insecure settings are intended only for localhost or development environments. Firewall and proxy rules should also allow long-lived WebSocket connections and should not aggressively terminate idle WebSocket traffic. Troubleshooting checklist: 1. Confirm the machine can reach Cyberdesk Cloud over HTTPS. 2. Confirm Cyberdesk WebSocket tunnel traffic is not blocked or intercepted. 3. Check whether the presented certificate issuer is a corporate proxy instead of a public CA. 4. If a corporate issuer is present, either bypass inspection for Cyberdesk or deploy the corporate root CA for the Cyberdriver service context. 5. Restart or reopen Cyberdriver and retry the connection. For exact allowlist requirements in a locked-down enterprise network, reach out to the Cyberdesk founders and we can help review the deployment. ## What should happen after setup Once connected: * The machine appears online in the Cyberdesk dashboard. * Other users in the same Cyberdesk organization can connect through Cyberdriver. * RustDesk peer IDs stay under the hood; the UI shows Cyberdesk machine names and IDs. * Desktop Tools can screenshot, click, type, scroll, use clipboard, access files, and run shell commands. ## Advanced topics Use screenshots, mouse, keyboard, files, clipboard, and shell commands from Cyberdesk. Collect logs and state when something goes wrong. Keep idle machines lightly active during long-running operations. Understand remote update and shutdown behavior. How Cyberdriver handles RDP disconnects and virtual displays. Route Cyberdriver through your own proxy domain. Stable Python-based client for most installs. Fully remove Cyberdriver with one PowerShell command before a clean reinstall. # Remote Management Source: https://docs.cyberdesk.io/cyberdriver/remote-management Shutdown and update Cyberdriver from Cyberdesk Cyberdesk can manage connected Cyberdriver machines through internal tunnel endpoints. ## Shutdown Remote shutdown is supported through: ```http theme={null} POST /internal/shutdown ``` This is intended to stop the Cyberdriver process or service-side tunnel after Cyberdesk has sent the request. The endpoint returns a response first, then exits shortly after so the response can flush through the tunnel. ## Updates Cyberdriver 1.x beta uses the RustDesk-based Windows MSI distribution path. Dashboard remote update for the stable Python agent remains capped to the legacy `0.0.x` line and should not downgrade a Cyberdriver 1.x beta install. Current launch recommendation: * Most customers use stable Legacy Cyberdriver unless Windows login-screen access is required. * Customers who need Windows login-screen access can use the Cyberdriver 1.x beta MSI. * Remote update polish can be added after launch once the signed Windows artifact flow is stable. ## Legacy auto-update cap The old dashboard auto-update flow is limited to legacy Python Cyberdriver versions up to `0.0.41`. Cyberdriver `1.0.0` and newer should be updated through the new MSI path. # Introduction Source: https://docs.cyberdesk.io/index Welcome to Cyberdesk Docs **Prerequisites**: Cyberdesk requires an active subscription. If you're not yet on a paid plan, [book a demo](https://cal.com/mahmoud-al-madi-klrs5s/cyberdesk-intro) to get started, or [join the self-serve waitlist](https://forms.gle/hF9iJzizL7PUxCxFA). ## Getting started Get up and running with Cyberdesk in minutes. Create your first workflow and run it in under 5 minutes. ## Install Cyberdriver Connect your desktop to Cyberdesk for remote automation. Install Cyberdriver to enable desktop automation on your machine. ## Use the SDK to trigger a run Execute workflows programmatically with our SDKs. Build automations with our type-safe TypeScript/JavaScript SDK. Integrate Cyberdesk into your Python applications with async support. ## Browse key concepts Learn how Cyberdesk caches successful workflows for 5-10x faster execution. Vision-based data extraction with flexible async processing modes. Dynamic observations and decisions that work in cached workflows. How observations transform into structured workflow output. ## Explore more Complete API documentation for all endpoints. Create and manage workflows in the web dashboard. # Quick Start Source: https://docs.cyberdesk.io/quickstart Create your first Cyberdesk workflow and run it in under 5 minutes This guide will walk you through creating your first workflow, installing Cyberdriver, and executing a run programmatically. ## Prerequisites * An active Cyberdesk subscription (if you're not on a paid plan, [book a demo](https://cal.com/mahmoud-al-madi-klrs5s/cyberdesk-intro)) * A Cyberdesk account ([sign up here](https://cyberdesk.io/register)) * Node.js 14+ or Python 3.10+ for SDK usage * Windows machine for desktop automation ## Step 1: Create a workflow in the dashboard Workflows define the tasks you want to automate. Let's create your first one. Go to the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard) and click on **Workflows** in the sidebar. Click the **New Workflow** button to open the workflow editor. Fill in the workflow details: * **Name** (optional): Give your workflow a descriptive name like "Extract Patient Data" * **Main Prompt**: This is the instruction that tells AI what to do. For example: ``` Navigate to the patient portal and extract the demographics information for patient ID: {patient_id}. Look for patient name {patient_first_name} {patient_last_name} to confirm you have the right record. ``` You can also add images to the prompt from the dashboard editor (click **Add Image**). Images help the agent understand tricky icons and UI elements in legacy apps. * **Input Schema** (optional): Define expected run inputs with JSON Schema. This validates the merged run payload (`input_values` + `sensitive_input_values` + machine/session values) before execution. In `input_schema`, sensitive root keys use a `$` prefix (for example, `"$api_key"`). * **Output Schema** (optional): Define the structure of data you want back using JSON Schema **Variables in Workflows:** * Use `{variable_name}` for input variables - values you pass when starting the workflow * Use `{{runtime_variable}}` for runtime variables - values discovered and set during execution by focused\_action * Use `{$variable}` for sensitive variables - pass values at run creation via `sensitive_input_values`. Sensitive values are stored in a secure vault during the run, never logged or sent to LLMs, resolved only during actual computer actions, and deleted immediately after the run completes. Example: `"Search for {patient_name} and save their ID as {{patient_id}} for later use"` **AI Assist Feature**: Instead of writing prompts manually, you can use our AI assist feature: 1. Type a natural language description of what you want to automate 2. Click **Apply** 3. AI will create a detailed prompt for you 4. You can send follow-up instructions to refine the prompt - AI uses the current field contents as context The same AI assist is available for both Input Schema and Output Schema fields to help you define JSON schemas quickly. Click **Create Workflow** to save. You'll be redirected to the workflow details page where you can find your workflow ID. ## Step 2: Install Cyberdriver Cyberdriver connects your desktop to Cyberdesk. Most users should start with the stable legacy installer. Use the Cyberdriver 1.x beta MSI only when you need Windows login-screen access, boot-time startup, or Windows service management. Open PowerShell and run the installer script from [Cyberdriver Quickstart](/cyberdriver/quickstart#stable-install-legacy-cyberdriver). The script downloads legacy Cyberdriver `v0.0.41`, adds it to your user `PATH`, and verifies the download. If you need the new Windows service path for login-screen control, use the [Cyberdriver 1.x beta installer](/cyberdriver/quickstart#beta-cyberdriver-1x-windows-install) instead. Close and reopen PowerShell, then join your machine with an organization API key from the dashboard. ```bash theme={null} cyberdriver join --secret YOUR_API_KEY ``` The desktop should appear online in **Cyberdesk -> Desktops**. Open the desktop in Cyberdesk and click **Desktop Tools**. Verify that it connects and you can control the computer. ## Step 3: Create a run via SDK Now let's execute your workflow programmatically using the SDK. ```bash theme={null} npm install cyberdesk ``` ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; async function runWorkflow() { const client = createCyberdeskClient('YOUR_API_KEY'); const { data: run, error } = await client.runs.create({ workflow_id: 'your-workflow-id', // From Step 1 machine_id: 'your-machine-id', // Optional: omit to auto-select an available machine input_values: { patient_id: '12345', patient_first_name: 'John', patient_last_name: 'Doe' } }); if (error || !run) { console.error('Failed to create run:', error); return; } console.log('Run created:', run.id); let currentRun = run; while (currentRun.status === 'scheduling' || currentRun.status === 'running') { await new Promise((resolve) => setTimeout(resolve, 5000)); const { data: updatedRun, error: refreshError } = await client.runs.get(currentRun.id); if (refreshError || !updatedRun) { console.error('Failed to refresh run:', refreshError); return; } currentRun = updatedRun; console.log('Status:', currentRun.status); } if (currentRun.status === 'success') { console.log('Results:', currentRun.output_data); } else { console.error('Run failed:', currentRun.error?.join(', ')); } } runWorkflow(); ``` ```bash theme={null} pip install cyberdesk ``` ```python theme={null} from cyberdesk import CyberdeskClient, RunCreate import asyncio async def run_workflow(): client = CyberdeskClient('YOUR_API_KEY') # Create a run run_data = RunCreate( workflow_id='your-workflow-id', # From Step 1 machine_id='your-machine-id', # Optional: omit to auto-select an available machine input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe' } ) response = await client.runs.create(run_data) if response.error: print(f'Failed to create run: {response.error}') return run = response.data print(f'Run created: {run.id}') while run.status in ['scheduling', 'running']: await asyncio.sleep(5) response = await client.runs.get(run.id) if response.error: print(f'Failed to refresh run: {response.error}') return run = response.data print(f'Status: {run.status}') if run.status == 'success': print('Results:', run.output_data) else: print('Run failed:', ', '.join(run.error or [])) asyncio.run(run_workflow()) ``` If your workflow defines an **Input Schema** and provided inputs do not match, the API returns a `422` validation error with path-level details so you can fix the payload before retrying. ## Step 4: View results in the dashboard After your run completes, you can view detailed information in the dashboard. Go to the [Runs page](https://cyberdesk.io/dashboard/runs) in your dashboard. Click on your run to see: * **Status**: Current state of the run * **Output Data**: The extracted/processed data based on your output schema * **Message History**: Complete conversation between AI and your desktop From the run details panel: * Click **Generate Trajectory** (when available) to promote that run's captured path into your workflow trajectory library Then navigate to your workflow's **Trajectories** tab to: * Review step-by-step actions with screenshots * **Approve trajectories** to enable fast cached execution on future runs Trajectories are Cyberdesk's intelligent caching system. Capture happens in the background during runs, generation makes a trajectory visible/reviewable, and approval enables replay. Learn more in [Trajectories 101](/concepts/trajectories). ## What's next? Deep dive into SDK features and advanced usage patterns Master workflow prompting with specialized tools and best practices Learn how trajectories speed up workflows by caching successful executions Explore all available API endpoints ## Learn More Vision-based extraction with async processing modes Dynamic observations and decisions in workflows Repeat workflow steps over arrays or counts efficiently How observations transform into structured output Optimize workflow performance with batch and run-scoped async # Python SDK Source: https://docs.cyberdesk.io/sdk-guides/python Complete guide for integrating Cyberdesk into your Python applications ## Quick Start Get up and running with Cyberdesk in under 5 minutes. This guide assumes you've already created workflows in the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard). ```bash pip theme={null} pip install cyberdesk ``` ```bash poetry theme={null} poetry add cyberdesk ``` ```bash pipenv theme={null} pipenv install cyberdesk ``` ```python theme={null} import asyncio from cyberdesk import CyberdeskClient, RunCreate async def main(): # Initialize the client client = CyberdeskClient('YOUR_API_KEY') # Create a run for your workflow run_data = RunCreate( workflow_id='your-workflow-id', machine_id='your-machine-id', input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe' } ) response = await client.runs.create(run_data) if response.error: print(f"Error creating run: {response.error}") return run = response.data # Wait for the run to complete while run.status in ['scheduling', 'running']: await asyncio.sleep(5) # Wait 5 seconds response = await client.runs.get(run.id) run = response.data # Get the output data if run.status == 'success': print('Patient data:', run.output_data) else: print('Run failed:', ', '.join(run.error or [])) # Run the async function asyncio.run(main()) ``` ```python theme={null} from cyberdesk import CyberdeskClient, RunCreate import time # Initialize the client client = CyberdeskClient('YOUR_API_KEY') # Create a run for your workflow run_data = RunCreate( workflow_id='your-workflow-id', machine_id='your-machine-id', input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe' } ) response = client.runs.create_sync(run_data) if response.error: print(f"Error creating run: {response.error}") else: run = response.data # Wait for the run to complete while run.status in ['scheduling', 'running']: time.sleep(5) # Wait 5 seconds response = client.runs.get_sync(run.id) run = response.data # Get the output data if run.status == 'success': print('Patient data:', run.output_data) else: print('Run failed:', ', '.join(run.error or [])) ``` Create and manage workflows in the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard). The dashboard supports rich, multimodal prompts — you can insert images alongside text to help the agent understand tricky UI elements. Use the SDK to execute runs against those workflows. ## Installation & Setup ### Prerequisites * Python 3.10 or higher * pip, poetry, or pipenv for package management ### Installation ```bash theme={null} pip install cyberdesk ``` ```bash theme={null} poetry add cyberdesk ``` ```bash theme={null} pipenv install cyberdesk ``` ### Virtual Environment Setup Always use a virtual environment to avoid dependency conflicts: ```bash theme={null} # Create virtual environment python -m venv venv # Activate it # On macOS/Linux: source venv/bin/activate # On Windows: venv\Scripts\activate # Install the SDK pip install cyberdesk ``` ## Authentication ### Creating a Client ```python theme={null} from cyberdesk import CyberdeskClient client = CyberdeskClient('YOUR_API_KEY') ``` ### Custom Base URL For self-hosted or enterprise deployments: ```python theme={null} client = CyberdeskClient('YOUR_API_KEY', base_url='https://api.your-domain.com') ``` ### Using Context Managers The client supports standard context managers for proper resource cleanup: ```python theme={null} with CyberdeskClient('YOUR_API_KEY') as client: response = client.runs.list_sync() # Client automatically cleans up when done ``` Never hardcode API keys in your source code. Use environment variables: ```python theme={null} import os from cyberdesk import CyberdeskClient client = CyberdeskClient(os.environ['CYBERDESK_API_KEY']) ``` ## Automatic Retries The SDK automatically retries failed requests with exponential backoff, following industry best practices similar to the Stripe SDK. This handles transient network issues and server errors gracefully without requiring any code changes. ### Default Behavior * **Retry count**: 3 retries (4 total attempts) * **Retryable errors**: Network failures, connection timeouts, DNS errors * **Retryable status codes**: 408, 429, 500, 502, 503, 504, and `409` only when `Idempotency-Status: in_progress` * **Non-retryable**: Most 4xx client errors * **Backoff strategy**: Exponential backoff with full jitter (250ms initial, 8s max) * **Retry-After header**: Respected when present ### Configuring Retries ```python theme={null} from cyberdesk import CyberdeskClient, RetryConfig import httpx # Default: 3 retries (4 total attempts) client = CyberdeskClient('YOUR_API_KEY') # More retries for unreliable networks client = CyberdeskClient('YOUR_API_KEY', retry=RetryConfig(max_retries=5)) # Disable retries entirely client = CyberdeskClient('YOUR_API_KEY', retry=RetryConfig(max_retries=0)) # With custom timeout configuration client = CyberdeskClient( 'YOUR_API_KEY', retry=RetryConfig(max_retries=3), timeout=httpx.Timeout(60.0, connect=10.0) ) ``` Retries only occur for network-level failures and specific HTTP status codes that indicate temporary server issues. Most client errors (4xx) are not retried because they indicate a problem with the request itself. ## Idempotency For write requests (POST, PUT, PATCH, DELETE), the SDK automatically adds an `Idempotency-Key` header. If a completed request is retried with the same key, the API replays the stored response. If the original request is still in progress, the API returns `409` with `Retry-After`. `5xx` responses are not replay-cached, so a later retry with the same key may execute again. ### Default Behavior * **Enabled by default**: All write requests automatically include an idempotency key * **Key generation**: Uses `uuid.uuid4()` to generate unique keys * **Server-side handling**: Completed responses are replayed for duplicate keys; in-progress duplicates return `409` ### Configuring Idempotency ```python theme={null} from cyberdesk import CyberdeskClient import uuid # Default: idempotency enabled with auto-generated keys client = CyberdeskClient('YOUR_API_KEY') # Disable idempotency (not recommended for production) client = CyberdeskClient('YOUR_API_KEY', idempotency_enabled=False) # Custom key generator (e.g., for deterministic keys) client = CyberdeskClient( 'YOUR_API_KEY', idempotency_key_generator=lambda: f"order-{order_id}-{uuid.uuid4()}" ) ``` You generally don't need to configure idempotency — the defaults work well. The SDK handles everything automatically, making completed write retries safe without extra plumbing in your app. ## Sync vs Async The Cyberdesk Python SDK provides both synchronous and asynchronous methods for all operations. ### When to Use Async Use async methods when: * Building web applications with async frameworks (FastAPI, aiohttp) * Making multiple concurrent API calls * Integrating with other async libraries * Building high-performance applications ### When to Use Sync Use sync methods when: * Writing simple scripts * Working in Jupyter notebooks * Integrating with sync-only frameworks (Django, Flask) * Learning or prototyping ### Method Naming Convention * Async methods: `client.resource.method()` * Sync methods: `client.resource.method_sync()` ```python theme={null} # Async response = await client.runs.create(data) # Sync response = client.runs.create_sync(data) ``` ## Working with Runs Runs are the primary way to execute workflows in Cyberdesk. Here's everything you need to know about managing runs through the SDK. ### Creating a Run ```python theme={null} from cyberdesk import RunCreate async def create_patient_data_run(): run_data = RunCreate( workflow_id='workflow-uuid', machine_id='machine-uuid', input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe', 'insurance': {'provider': 'Blue Cross', 'policy_number': 'BC123'} } ) response = await client.runs.create(run_data) if response.error: print(f"Failed to create run: {response.error}") else: print(f"Run created: {response.data.id}") return response.data ``` ```python theme={null} from cyberdesk import RunCreate run_data = RunCreate( workflow_id='workflow-uuid', machine_id='machine-uuid', input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe', 'insurance': {'provider': 'Blue Cross', 'policy_number': 'BC123'} } ) response = client.runs.create_sync(run_data) if response.error: print(f"Failed to create run: {response.error}") else: print(f"Run created: {response.data.id}") ``` Pass nested objects and access them in prompts with dot notation like `{insurance.provider}`. See [Structured Inputs](/concepts/structured-inputs). ### Prioritizing a Run Set `is_priority=True` when a run should be matched before normal queued runs: ```python theme={null} from cyberdesk import RunCreate run = client.runs.create_sync( RunCreate( workflow_id='workflow-uuid', is_priority=True, ) ) ``` The field defaults to `False`. `RunBulkCreate` and `WorkflowChainCreate` also accept `is_priority`; one value applies to every run created by that request. Priority affects the next eligible machine assignment only—it does not interrupt running work or let a later run bypass an earlier run in the same session or chain. In-place retries keep the run's existing priority unless the retry request supplies an `is_priority` override. Priority runs are processed before normal runs. A sustained stream of priority work can delay normal queued runs. See [Priority Runs](/concepts/priority-runs) for complete matching, session-ordering, and retry semantics. ### Creating a Run with Sensitive Input Values If your workflow prompt references sensitive variables using the `{$variable}` syntax (for example, `{$password}`), pass those values via `sensitive_input_values`. ```python theme={null} from cyberdesk import RunCreate run_data = RunCreate( workflow_id='workflow-uuid', machine_id='machine-uuid', input_values={ # non-sensitive inputs 'patient_id': '12345' }, sensitive_input_values={ # referenced in your prompt as {$password} 'password': 's3cr3tP@ss' } ) response = await client.runs.create(run_data) ``` ```python theme={null} from cyberdesk import RunCreate run_data = RunCreate( workflow_id='workflow-uuid', machine_id='machine-uuid', input_values={ 'patient_id': '12345' }, sensitive_input_values={ 'password': 's3cr3tP@ss' } ) response = client.runs.create_sync(run_data) ``` Sensitive inputs are stored in a secure third‑party secret vault (Basis Theory) only for the duration of the run. They are not logged in Cyberdesk, and they are not sent to any LLMs. The values are only resolved at the last moment during actual computer actions (e.g., when typing). After the run completes, these sensitive values are deleted from the vault. On the dashboard, sensitive inputs are never displayed and will not be prefilled when repeating a run. ### Creating a Run with Machine Pools You can specify pool requirements when creating a run. This ensures your run is executed on a machine that belongs to ALL specified pools. This is especially useful for: * Running workflows on customer-specific machines * Requiring machines with specific software installed * Organizing machines by location or capability ```python theme={null} from cyberdesk import RunCreate async def create_run_with_pools(): # Get pool IDs (typically from your configuration or database) customer_pool_id = 'pool-uuid-1' # e.g., "Customer A" pool excel_pool_id = 'pool-uuid-2' # e.g., "Has Excel" pool run_data = RunCreate( workflow_id='workflow-uuid', # Machine must be in BOTH pools (intersection, not union) pool_ids=[customer_pool_id, excel_pool_id], input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe' } ) response = await client.runs.create(run_data) if response.error: print(f"Failed to create run: {response.error}") else: print(f"Run created: {response.data.id}") print(f"Will execute on machine in pools: {run_data.pool_ids}") return response.data ``` ```python theme={null} from cyberdesk import RunCreate # Get pool IDs (typically from your configuration or database) customer_pool_id = 'pool-uuid-1' # e.g., "Customer A" pool excel_pool_id = 'pool-uuid-2' # e.g., "Has Excel" pool run_data = RunCreate( workflow_id='workflow-uuid', # Machine must be in BOTH pools (intersection, not union) pool_ids=[customer_pool_id, excel_pool_id], input_values={ 'patient_id': '12345', 'patient_first_name': 'John', 'patient_last_name': 'Doe' } ) response = client.runs.create_sync(run_data) if response.error: print(f"Failed to create run: {response.error}") else: print(f"Run created: {response.data.id}") print(f"Will execute on machine in pools: {run_data.pool_ids}") ``` **Pool Matching Logic:** When you specify multiple pools, Cyberdesk will only select machines that belong to **ALL** specified pools (intersection). For example, if you specify `["Customer A", "Has Excel"]`, only machines that are in both pools will be considered. If you provide a `machine_id` when creating a run, `pool_ids` are ignored. Cyberdesk will only attempt the specified machine; if it's busy or unavailable, the run will wait until that machine is free (no fallback to other machines or pools). **Creating and Managing Pools:** While you can manage pools via the SDK, we recommend using the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard) for a more intuitive experience: 1. Navigate to any machine in the dashboard 2. Click on the machine to view its details 3. Add the machine to existing pools or create new pools 4. Assign multiple pools to organize machines by customer, capability, or location Common pool strategies: * **By Customer**: "Customer A", "Customer B", etc. * **By Software**: "Has Excel", "Has Chrome", "Has Epic EHR" * **By Environment**: "Production", "Staging", "Development" * **By Location**: "US-East", "EU-West", etc. ### Creating a Run with File Inputs You can attach files to a run at creation. This is useful for workflows that need to process or manipulate files on the remote machine. ```python theme={null} import base64 from cyberdesk import RunCreate, FileInput async def create_run_with_file(): # Read and encode a file with open("path/to/your/file.txt", "rb") as f: content = base64.b64encode(f.read()).decode("utf-8") run_data = RunCreate( workflow_id='workflow-uuid', file_inputs=[ FileInput( filename="file.txt", content=content, target_path="C:/Users/Default/Desktop/file.txt", # Optional cleanup_imports_after_run=True # Optional ) ] ) response = await client.runs.create(run_data) if response.error: print(f"Failed to create run: {response.error}") else: print(f"Run created with file attachment: {response.data.id}") return response.data ``` ```python theme={null} import base64 from cyberdesk import RunCreate, FileInput # Read and encode a file with open("path/to/your/file.txt", "rb") as f: content = base64.b64encode(f.read()).decode("utf-8") run_data = RunCreate( workflow_id='workflow-uuid', file_inputs=[ FileInput( filename="file.txt", content=content, target_path="C:/Users/Default/Desktop/file.txt", # Optional cleanup_imports_after_run=True # Optional ) ] ) response = client.runs.create_sync(run_data) if response.error: print(f"Failed to create run: {response.error}") else: print(f"Run created with file attachment: {response.data.id}") ``` The name of the file, including its extension. The base64-encoded content of the file. The absolute path on the remote machine where the file should be saved. If not provided, it defaults to `~/CyberdeskTransfers/`. If `True`, the file will be deleted from the remote machine after the run completes (whether it succeeds or fails). Defaults to `False`. ### Listing Runs ```python theme={null} # List all runs response = await client.runs.list() runs = response.data.items # List with pagination response = await client.runs.list(skip=0, limit=20) # Filter by status from cyberdesk import RunStatus response = await client.runs.list(status=RunStatus.SUCCESS) # Filter by workflow response = await client.runs.list(workflow_id='workflow-uuid') # Multiple filters response = await client.runs.list( workflow_id='workflow-uuid', status=RunStatus.RUNNING, limit=10 ) ``` ```python theme={null} # List all runs response = client.runs.list_sync() runs = response.data.items # List with pagination response = client.runs.list_sync(skip=0, limit=20) # Filter by status from cyberdesk import RunStatus response = client.runs.list_sync(status=RunStatus.SUCCESS) # Filter by workflow response = client.runs.list_sync(workflow_id='workflow-uuid') ``` #### Faster lists with fields projection Use the optional `fields` parameter to return only selected fields per run. This reduces payload size and speeds up listing, especially when you do not need heavy JSON fields like `run_message_history`. Need to fetch screenshots referenced in `run_message_history`? See [Run Screenshots](/concepts/run-screenshots) for signed URL examples. * Base fields always included: `id`, `workflow_id`, `machine_id`, `status`, `created_at`. * Add more by passing `fields=[...]`. ```python theme={null} from cyberdesk.client import RunField # Minimal (base fields only) res = await client.runs.list() for item in res.data.items: print(item.id, item.status) # Include inputs only (still avoids run_message_history) res = await client.runs.list(fields=[RunField.INPUT_VALUES]) # Include a couple of specific fields res = await client.runs.list(fields=[RunField.INPUT_VALUES, RunField.SESSION_ID]) # Include attachments but skip history for speed res = await client.runs.list(fields=[RunField.INPUT_ATTACHMENT_IDS, RunField.OUTPUT_ATTACHMENT_IDS]) ``` ```python theme={null} from cyberdesk.client import RunField # Minimal (base fields only) res = client.runs.list_sync() # Include inputs only res = client.runs.list_sync(fields=[RunField.INPUT_VALUES]) # Include attachments (skipping run_message_history) res = client.runs.list_sync(fields=[RunField.INPUT_ATTACHMENT_IDS, RunField.OUTPUT_ATTACHMENT_IDS]) ``` If you need full records (including `run_message_history`), call `list()` without `fields`. ### Getting a Specific Run ```python theme={null} response = await client.runs.get('run-uuid') if response.data: run = response.data print(f"Run status: {run.status}") print(f"Output data: {run.output_data}") ``` ```python theme={null} response = client.runs.get_sync('run-uuid') if response.data: run = response.data print(f"Run status: {run.status}") print(f"Output data: {run.output_data}") ``` ### Updating a Run Run updates are typically handled automatically by the Cyberdesk system. Manual updates are rarely needed. ```python theme={null} from cyberdesk import RunUpdate, RunStatus update_data = RunUpdate(status=RunStatus.CANCELLED) response = await client.runs.update('run-uuid', update_data) ``` ```python theme={null} from cyberdesk import RunUpdate, RunStatus update_data = RunUpdate(status=RunStatus.CANCELLED) response = client.runs.update_sync('run-uuid', update_data) ``` ### Retrying a Run (same run\_id) Use retry when you want to re-run the exact same run id, clearing outputs and optionally providing fresh inputs/files. **Sensitive values:** always re-send `sensitive_input_values` on retry; secrets are deleted after each run. **File inputs:** if you set `cleanup_imports_after_run=True`, files are deleted from the remote machine after the run; include `file_inputs` again if you need a fresh copy or when no input attachments exist (providing `file_inputs` replaces prior input attachments). **Regular inputs:** only send `input_values` if you want to change them; otherwise the previous ones are reused. ```python theme={null} import os from cyberdesk import FileInput from cyberdesk.client import RunRetry retry_data = RunRetry( input_values={'query': 'new query'}, # optional sensitive_input_values={'password': os.environ['APP_PASSWORD']}, file_inputs=[ # providing file_inputs replaces prior input attachments FileInput(filename='input.pdf', content=base64_pdf) ], reuse_session=True, # default: keep existing session # session_id='existing-session-uuid', # release_session_after=True, # machine_id='specific-machine-uuid', # pool_ids=['pool-a', 'pool-b'], # used only when no machine_id is set ) response = await client.runs.retry('run-uuid', retry_data) if response.error: print('Failed to retry run:', response.error) ``` ```python theme={null} from cyberdesk import FileInput from cyberdesk.client import RunRetry retry_data = RunRetry( input_values={'query': 'new query'}, file_inputs=[FileInput(filename='input.pdf', content=base64_pdf)], reuse_session=True, ) response = client.runs.retry_sync('run-uuid', retry_data) if response.error: print('Failed to retry run:', response.error) ``` Behavior: * Retry is allowed only for terminal runs: success, task\_failed, error, or cancelled. * Outputs, history, and output attachments are always cleared. * Prior input attachments are kept unless you provide file\_inputs (then they are replaced). * If you provide sensitive\_input\_values, new secrets are created; otherwise sensitive aliases are cleared. * When a session\_id is present and the session is busy, immediate assignment is skipped and the retried run queues. * Avoid sending both machine\_id and pool\_ids on retry; if both are present, pool\_ids take precedence. ### Deleting a Run ```python theme={null} response = await client.runs.delete('run-uuid') if not response.error: print("Run deleted successfully") ``` ```python theme={null} response = client.runs.delete_sync('run-uuid') if not response.error: print("Run deleted successfully") ``` ### Polling for Run Completion Here's a robust pattern for waiting for runs to complete: ```python theme={null} import asyncio from datetime import datetime, timedelta async def wait_for_run_completion(client, run_id, timeout_seconds=300): """Wait for a run to complete with timeout.""" start_time = datetime.now() timeout = timedelta(seconds=timeout_seconds) while datetime.now() - start_time < timeout: response = await client.runs.get(run_id) if response.error: raise Exception(f"Failed to get run status: {response.error}") run = response.data if run.status == 'success': return run if run.status in ['error', 'task_failed', 'cancelled']: raise Exception(f"Run {run.status}: {', '.join(run.error or ['Unknown error'])}") await asyncio.sleep(5) # Poll every 5 seconds raise TimeoutError(f"Run timed out after {timeout_seconds} seconds") # Usage try: completed_run = await wait_for_run_completion(client, run.id) print("Output:", completed_run.output_data) except Exception as e: print(f"Run failed: {e}") ``` ```python theme={null} import time from datetime import datetime, timedelta def wait_for_run_completion_sync(client, run_id, timeout_seconds=300): """Wait for a run to complete with timeout.""" start_time = datetime.now() timeout = timedelta(seconds=timeout_seconds) while datetime.now() - start_time < timeout: response = client.runs.get_sync(run_id) if response.error: raise Exception(f"Failed to get run status: {response.error}") run = response.data if run.status == 'success': return run if run.status in ['error', 'task_failed', 'cancelled']: raise Exception(f"Run {run.status}: {', '.join(run.error or ['Unknown error'])}") time.sleep(5) # Poll every 5 seconds raise TimeoutError(f"Run timed out after {timeout_seconds} seconds") # Usage try: completed_run = wait_for_run_completion_sync(client, run.id) print("Output:", completed_run.output_data) except Exception as e: print(f"Run failed: {e}") ``` ## Working with File Attachments Manage files associated with your runs, such as input files uploaded at creation or output files generated by a workflow. ### Listing Run Attachments You can list all attachments for a specific run and filter them by type (`INPUT` or `OUTPUT`). ```python theme={null} from cyberdesk import AttachmentType # List all attachments for a run response = await client.run_attachments.list(run_id='run-uuid') attachments = response.data.items # List only output attachments response = await client.run_attachments.list( run_id='run-uuid', attachment_type=AttachmentType.OUTPUT ) output_files = response.data.items ``` ```python theme={null} from cyberdesk import AttachmentType # List all attachments for a run response = client.run_attachments.list_sync(run_id='run-uuid') attachments = response.data.items # List only input attachments response = client.run_attachments.list_sync( run_id='run-uuid', attachment_type=AttachmentType.INPUT ) input_files = response.data.items ``` ### Downloading an Attachment There are multiple ways to download attachments depending on your use case: #### Method 1: Get a Download URL Get a signed URL that triggers automatic download when accessed. This is perfect for web applications where you want to provide download links to users. ```python theme={null} # Get a download URL with custom expiration (default: 5 minutes) response = await client.run_attachments.get_download_url( 'attachment-uuid', expires_in=600 # 10 minutes ) if response.data: print(f"Download URL: {response.data.url}") print(f"Expires in: {response.data.expires_in} seconds") # You can use this URL in your web app or share it # The URL will trigger automatic download when accessed ``` ```python theme={null} # Get a download URL with custom expiration response = client.run_attachments.get_download_url_sync( 'attachment-uuid', expires_in=300 # 5 minutes (default) ) if response.data: print(f"Download URL: {response.data.url}") print(f"Expires in: {response.data.expires_in} seconds") ``` #### Method 2: Download Raw File Content Download the file content directly as bytes. Useful when you need to process the file in memory. ```python theme={null} # Get the attachment metadata first response = await client.run_attachments.get('attachment-uuid') attachment_info = response.data # Download the file content response = await client.run_attachments.download(attachment_info.id) if not response.error: # Save the file with open(attachment_info.filename, "wb") as f: f.write(response.data) print(f"Downloaded {attachment_info.filename}") ``` ```python theme={null} # Get the attachment metadata first response = client.run_attachments.get_sync('attachment-uuid') attachment_info = response.data # Download the file content response = client.run_attachments.download_sync(attachment_info.id) if not response.error: # Save the file with open(attachment_info.filename, "wb") as f: f.write(response.data) print(f"Downloaded {attachment_info.filename}") ``` #### Method 3: Save to File (Convenience Method) The SDK provides a convenience method that downloads and saves the file in one operation. ```python theme={null} # Save directly to a file response = await client.run_attachments.save_to_file( 'attachment-uuid', output_path='./downloads/' # Will use original filename ) if response.data: print(f"Saved to: {response.data['path']}") print(f"File size: {response.data['size']} bytes") # Or specify a custom filename response = await client.run_attachments.save_to_file( 'attachment-uuid', output_path='./downloads/custom-name.pdf' ) ``` ```python theme={null} # Save directly to a file response = client.run_attachments.save_to_file_sync( 'attachment-uuid', output_path='./downloads/' # Will use original filename ) if response.data: print(f"Saved to: {response.data['path']}") print(f"File size: {response.data['size']} bytes") ``` ### Example: Upload, Process, and Download Here's a full example of a workflow that processes a file. 1. **Workflow Prompt**: `"Take the file at ~/CyberdeskTransfers/report.txt, add a summary to the end of it, and mark it for export."` 2. **Workflow Setting**: `includes_file_exports` is set to `True`. ```python theme={null} import asyncio import base64 from cyberdesk import CyberdeskClient, RunCreate, FileInput, AttachmentType async def main(): with CyberdeskClient("YOUR_API_KEY") as client: # 1. Prepare and upload the input file report_content = "This is the initial report content." encoded_content = base64.b64encode(report_content.encode()).decode() run_data = RunCreate( workflow_id="your-file-processing-workflow-id", file_inputs=[ FileInput(filename="report.txt", content=encoded_content) ] ) response = await client.runs.create(run_data) run = response.data print(f"Run started: {run.id}") # 2. Wait for the run to complete completed_run = await wait_for_run_completion(client, run.id) print("Run finished with status:", completed_run.status) # 3. Find and download the output attachment if completed_run.status == 'success': response = await client.run_attachments.list( run_id=completed_run.id, attachment_type=AttachmentType.OUTPUT ) output_attachments = response.data.items if output_attachments: processed_report = output_attachments[0] # Option 1: Get a download URL (for web apps) url_response = await client.run_attachments.get_download_url(processed_report.id) if url_response.data: print(f"Download URL: {url_response.data.url}") print(f"Valid for: {url_response.data.expires_in} seconds") # Option 2: Download the processed file directly response = await client.run_attachments.download(processed_report.id) if not response.error: # Decode and print the content processed_content = response.data.decode() print("\n--- Processed Report ---") print(processed_content) print("------------------------") else: print(f"Failed to download processed file: {response.error}") else: print("No output files were generated.") # Assuming wait_for_run_completion is defined as in the previous examples asyncio.run(main()) ``` This example demonstrates the complete lifecycle: uploading a file with a run, executing a workflow that modifies it, and then retrieving the processed file from the run's output attachments. ## Sessions and Chained Runs At its core, a session is a reservation of a single machine. While a session is active, that machine is dedicated to your session only — no unrelated runs will be scheduled onto it. This guarantees your multi‑step automations run back‑to‑back on the same desktop without interference. What you get from a session: * Exclusive access to one machine for the session's duration (strong scheduling guarantee) * Deterministic "step 1 → step 2 → …" behavior with no opportunistic interleaving **Chains** are a convenient way to create multiple runs that execute back‑to‑back in the same session. Instead of manually creating individual runs and managing their sequencing, you can define all your workflow steps upfront and let Cyberdesk handle the session management and execution order. ### Passing data between steps with refs Once you have multiple workflows running in the same session, you'll often want to pass outputs from earlier steps as inputs to later ones. Refs make this seamless — simply reference a previous step's output using a JSON object: ```python theme={null} {"$ref": "step1.outputs.result"} ``` You can construct these as plain Python dicts when building chain steps. ### Start a new session and run a chain (best when you know the whole sequence) ```python theme={null} import os from cyberdesk import CyberdeskClient from cyberdesk.client import WorkflowChainCreate client = CyberdeskClient(os.environ['CYBERDESK_API_KEY']) chain = WorkflowChainCreate.from_dict({ "shared_inputs": { "search_query": "red panda facts" }, "shared_sensitive_inputs": { "api_key": "shared-secret-key" }, # Filter machines by pools; or use machine_id to target one machine "pool_ids": ["pool-with-chrome", "customer-a"], "keep_session_after_completion": False, "steps": [ { "workflow_id": "step-1-workflow-id", "session_alias": "step1", "inputs": { "topic": "red panda" }, # Step-specific sensitive inputs that override or extend shared_sensitive_inputs "sensitive_inputs": { "username": "user1", "password": "secret123" } }, { "workflow_id": "step-2-workflow-id", "session_alias": "step2", "inputs": { # Use output of step1 as an input to step2 "search_query": {"$ref": "step1.outputs.result"} }, # Step-specific sensitive inputs that override or extend shared_sensitive_inputs "sensitive_inputs": { "security_token": "step2-token" } } ] }) resp = client.runs.chain_sync(chain) print("session:", resp.data.session_id) print("run_ids:", resp.data.run_ids) ``` Notes: * Provide `machine_id` to target a specific machine, or `pool_ids` to match any machine in **all** specified pools (intersection). * The chain runs on one reserved session. If you omit `session_id`, the API creates one and reserves a machine before step 1. * `shared_inputs` are merged into each step, and step-level `inputs` override shared values when the same key appears in both places. * **`shared_sensitive_inputs`** are available to all steps in the chain. * **`sensitive_inputs`** in individual steps provide step-specific sensitive values that override or extend the shared ones. If the same key exists in both, the step-specific value takes precedence. * `shared_file_inputs` (if provided) are attached to the first run in the chain. ### Join an existing session ```python theme={null} from cyberdesk.client import WorkflowChainCreate chain = WorkflowChainCreate.from_dict({ "session_id": "existing-session-uuid", "steps": [ {"workflow_id": "wf-a", "session_alias": "warmup"}, {"workflow_id": "wf-b", "session_alias": "extract", "inputs": {"query": "current patient"}}, ] }) client.runs.chain_sync(chain) ``` For chains, provide either `session_id` or `machine_id`/`pool_ids`, not both. ### Keep the session alive after the chain ```python theme={null} from cyberdesk.client import WorkflowChainCreate client.runs.chain_sync(WorkflowChainCreate.from_dict({ "pool_ids": ["customer-a"], "keep_session_after_completion": True, "steps": [ ... ] })) ``` Later you can start another chain with `session_id` to continue work on the same machine. ### Ad‑hoc sessions without a chain (start with a single run, then add more) You can start a session with a normal run and then submit additional runs referencing the same `session_id` — useful when downstream steps are conditional or discovered at runtime. ```python theme={null} # 1) Start a session and warm up the desktop warmup = client.runs.create_sync(RunCreate( workflow_id='login-workflow-id', pool_ids=['customer-a'], start_session=True, input_values={'username': 'alice'} )).data session_id = warmup.session_id # 2) Add another run in the same session — scheduling remains exclusive client.runs.create_sync(RunCreate( workflow_id='search-workflow-id', session_id=session_id, input_values={'query': 'recent orders'} )) # 3) Final run that releases the session when complete client.runs.create_sync(RunCreate( workflow_id='cleanup-workflow-id', session_id=session_id, release_session_after=True, # Release the session after this run completes input_values={'cleanup': 'true'} )) ``` ### Automatic session release with release\_session\_after When creating individual runs in a session (not using chains), you can use `release_session_after=True` to automatically release the session when that run completes (regardless of success or failure): ```python theme={null} # This run will release the session after it completes final_run = client.runs.create_sync(RunCreate( workflow_id='final-workflow-id', session_id=existing_session_id, release_session_after=True, input_values={'finalize': 'true'} )) ``` This is useful mainly as a convenience, so you don't have to decouple creating a session ending run and actually ending the session. Note: The session is released when the run completes, whether it succeeds, fails, or is cancelled. This ensures the session doesn't remain locked if something goes wrong. ### Detecting session completion via webhooks The `release_session_after` field on a run indicates whether this run released the session. You can use this in your webhook handler to detect when all runs in a session are complete: ```python theme={null} # In your webhook handler for "run_complete" events if event["run"].get("release_session_after") is True: # This run released the session - all runs in this session are done print(f"Session {event['run']['session_id']} was released by run {event['run']['id']}") print(f"Final status: {event['run']['status']}") # 'success', 'task_failed', 'error', or 'cancelled' ``` This field is automatically set to `True` when: * You explicitly set `release_session_after=True` on a run * A chain completes with `keep_session_after_completion=False` (the last run gets this flag) * A run errors, task-fails, or is cancelled and causes the session to be released See [Detecting session completion via webhooks](/concepts/sessions-and-chains#detecting-session-completion-via-webhooks) for more details. ### Polling chain runs The chain API returns run\_ids in creation order; you can poll them individually, or [receive a webhook when any of those runs complete](/webhooks/quickstart) ```python theme={null} chain = client.runs.chain_sync(...).data for run_id in chain.run_ids: completed = wait_for_run_completion_sync(client, run_id, 600) print(completed.status, completed.output_data) ``` ### Real‑world cases that require sessions * **EHR workflows**: Log into Epic, navigate to a specific patient, extract their data, then upload documents to their chart — all with no interruptions from other miscellaneous runs. * **Financial reporting**: Export monthly reports from your ERP system, transform the data in Excel, then re‑import the processed results — all back‑to‑back without interference. * **Document processing**: Download files from a web portal, process them with a local application, then upload the results back — ensuring no other runs interfere with your workflow. ### Bulk Creating Runs with Pools When creating multiple runs in bulk, you can also specify pool requirements. All runs will be distributed across machines that match the pool criteria. If you provide a `machine_id` in a bulk run request, `pool_ids` are ignored for those runs. Each run will only target the specified machine; if it is busy, the run will wait for that machine rather than falling back to other machines or pools. ```python theme={null} from cyberdesk import RunBulkCreate async def bulk_create_with_pools(): # Create 100 runs that require machines in specific pools bulk_data = RunBulkCreate( workflow_id='workflow-uuid', count=100, pool_ids=['customer-a-pool-id', 'excel-pool-id'], input_values={ 'task_type': 'data_extraction', 'priority': 'high' } ) response = await client.runs.bulk_create(bulk_data) if response.data: print(f"Created {len(response.data.created_runs)} runs") print(f"Failed: {response.data.failed_count}") # Each run will execute on machines that match all specified pools when available ``` ```python theme={null} from cyberdesk import RunBulkCreate bulk_data = RunBulkCreate( workflow_id='workflow-uuid', count=100, pool_ids=['customer-a-pool-id', 'excel-pool-id'], input_values={ 'task_type': 'data_extraction', 'priority': 'high' } ) response = client.runs.bulk_create_sync(bulk_data) if response.data: print(f"Created {len(response.data.created_runs)} runs") print(f"Failed: {response.data.failed_count}") # Each run will execute on machines that match all specified pools when available ``` **Bulk Run Assignment:** When bulk creating runs with pool requirements, Cyberdesk attempts to assign each run to any available machine that meets all specified pools. If no matching machine is available, runs remain in scheduling until one is free. No specific load balancing guarantees are made. ## Real-World Example: Healthcare Integration Here's a complete example of retrieving patient data from an Epic EHR system using Cyberdesk: ```python theme={null} from fastapi import FastAPI, HTTPException, Body from cyberdesk import CyberdeskClient, RunCreate import os app = FastAPI() client = CyberdeskClient(os.environ['CYBERDESK_API_KEY']) @app.post("/patients/lookup") async def get_patient_data( patient_id: str = Body(...), patient_first_name: str = Body(...), patient_last_name: str = Body(...) ): """Retrieve patient data from Epic EHR.""" try: # Create a run to fetch patient data run_data = RunCreate( workflow_id='550e8400-e29b-41d4-a716-446655440000', # Your Epic workflow ID machine_id='550e8400-e29b-41d4-a716-446655440001', # Your Epic machine ID input_values={ 'patient_id': patient_id, 'patient_first_name': patient_first_name, 'patient_last_name': patient_last_name } ) response = await client.runs.create(run_data) if response.error: raise HTTPException(status_code=500, detail=f"Failed to create run: {response.error}") run = response.data print(f"Fetching data for patient {patient_first_name} {patient_last_name} ({patient_id})...") # Wait for completion (2 minute timeout) completed_run = await wait_for_run_completion(client, run.id, 120) # Process the patient data patient_data = completed_run.output_data return { 'patientId': patient_id, 'demographics': patient_data['demographics'], 'medications': patient_data['medications'], 'vitals': patient_data['recentVitals'], 'lastUpdated': patient_data['lastUpdated'] } except TimeoutError: raise HTTPException(status_code=504, detail="Request timed out") except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` ```python theme={null} # views.py from django.http import JsonResponse from django.views import View from cyberdesk import CyberdeskClient, RunCreate import os import json client = CyberdeskClient(os.environ['CYBERDESK_API_KEY']) class PatientLookupView(View): def post(self, request): """Retrieve patient data from Epic EHR.""" try: data = json.loads(request.body) patient_id = data['patient_id'] patient_first_name = data['patient_first_name'] patient_last_name = data['patient_last_name'] # Create a run to fetch patient data run_data = RunCreate( workflow_id='550e8400-e29b-41d4-a716-446655440000', # Your Epic workflow ID machine_id='550e8400-e29b-41d4-a716-446655440001', # Your Epic machine ID input_values={ 'patient_id': patient_id, 'patient_first_name': patient_first_name, 'patient_last_name': patient_last_name } ) response = client.runs.create_sync(run_data) if response.error: return JsonResponse({'error': f"Failed to create run: {response.error}"}, status=500) run = response.data print(f"Fetching data for patient {patient_first_name} {patient_last_name} ({patient_id})...") # Wait for completion completed_run = wait_for_run_completion_sync(client, run.id, 120) # Process the patient data patient_data = completed_run.output_data return JsonResponse({ 'patientId': patient_id, 'demographics': patient_data['demographics'], 'medications': patient_data['medications'], 'vitals': patient_data['recentVitals'], 'lastUpdated': patient_data['lastUpdated'] }) except TimeoutError: return JsonResponse({'error': 'Request timed out'}, status=504) except Exception as e: return JsonResponse({'error': str(e)}, status=500) ``` ```python theme={null} #!/usr/bin/env python3 """ Script to extract patient data from Epic EHR and save to CSV. """ import csv import asyncio from datetime import datetime from cyberdesk import CyberdeskClient, RunCreate import os async def extract_patient_data(client, patient_list): """Extract data for multiple patients.""" results = [] for patient in patient_list: print(f"Processing patient {patient['first_name']} {patient['last_name']} ({patient['id']})...") try: # Create run for patient data run_data = RunCreate( workflow_id='550e8400-e29b-41d4-a716-446655440000', # Your Epic workflow ID machine_id='550e8400-e29b-41d4-a716-446655440001', # Your Epic machine ID input_values={ 'patient_id': patient['id'], 'patient_first_name': patient['first_name'], 'patient_last_name': patient['last_name'] } ) response = await client.runs.create(run_data) if response.error: print(f"Error for patient {patient['id']}: {response.error}") continue # Wait for completion completed_run = await wait_for_run_completion(client, response.data.id) patient_data = completed_run.output_data results.append({ 'patient_id': patient['id'], 'first_name': patient['first_name'], 'last_name': patient['last_name'], 'dob': patient_data.get('demographics', {}).get('dateOfBirth'), 'mrn': patient_data.get('demographics', {}).get('mrn'), 'phone': patient_data.get('demographics', {}).get('phone'), 'email': patient_data.get('demographics', {}).get('email'), 'medication_count': len(patient_data.get('medications', [])), 'extracted_at': datetime.now().isoformat() }) except Exception as e: print(f"Failed to process patient {patient['id']}: {e}") return results async def main(): # Read patient list from CSV patient_list = [] with open('patients.csv', 'r') as f: reader = csv.DictReader(f) for row in reader: patient_list.append({ 'id': row['patient_id'], 'first_name': row['first_name'], 'last_name': row['last_name'] }) with CyberdeskClient(os.environ['CYBERDESK_API_KEY']) as client: results = await extract_patient_data(client, patient_list) # Save to CSV with open('patient_data_export.csv', 'w', newline='') as f: if results: writer = csv.DictWriter(f, fieldnames=results[0].keys()) writer.writeheader() writer.writerows(results) print(f"Exported {len(results)} patient records to patient_data_export.csv") if __name__ == "__main__": asyncio.run(main()) ``` ## Other SDK Resources **Important:** While the SDK provides full CRUD operations for all Cyberdesk resources, we strongly recommend using the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard) for managing these resources. The dashboard provides a more intuitive interface for: * Creating and editing workflows * Managing machines * Viewing connections * Analyzing trajectories The SDK methods below are provided for advanced use cases and automation scenarios. ```python theme={null} from cyberdesk import PoolCreate, PoolUpdate, MachinePoolUpdate # List pools response = await client.pools.list() pools = response.data.items # Create a pool pool_data = PoolCreate( name='Customer A', description='All machines for Customer A' ) response = await client.pools.create(pool_data) # Get a pool (with optional machine list) response = await client.pools.get('pool-id', include_machines=True) # Update a pool update_data = PoolUpdate(description='Updated description') response = await client.pools.update('pool-id', update_data) # Add machines to a pool from cyberdesk import MachinePoolAssignment assignment_data = MachinePoolAssignment( machine_ids=['machine-1', 'machine-2'] ) response = await client.pools.add_machines('pool-id', assignment_data) # Update a machine's pools pool_update = MachinePoolUpdate( pool_ids=['pool-1', 'pool-2', 'pool-3'] ) response = await client.machines.update_pools('machine-id', pool_update) # Delete a pool response = await client.pools.delete('pool-id') ``` ```python theme={null} from cyberdesk import PoolCreate, PoolUpdate, MachinePoolUpdate # List pools response = client.pools.list_sync() pools = response.data.items # Create a pool pool_data = PoolCreate( name='Customer A', description='All machines for Customer A' ) response = client.pools.create_sync(pool_data) # Get a pool response = client.pools.get_sync('pool-id', include_machines=True) # Update a pool update_data = PoolUpdate(description='Updated description') response = client.pools.update_sync('pool-id', update_data) # Add machines to a pool from cyberdesk import MachinePoolAssignment assignment_data = MachinePoolAssignment( machine_ids=['machine-1', 'machine-2'] ) response = client.pools.add_machines_sync('pool-id', assignment_data) # Update a machine's pools pool_update = MachinePoolUpdate( pool_ids=['pool-1', 'pool-2', 'pool-3'] ) response = client.machines.update_pools_sync('machine-id', pool_update) # Delete a pool response = client.pools.delete_sync('pool-id') ``` ```python theme={null} from cyberdesk import MachineCreate, MachineUpdate # List machines response = await client.machines.list() machines = response.data.items # Create a machine machine_data = MachineCreate( name='Epic EHR Machine', description='Production Epic environment' ) response = await client.machines.create(machine_data) # Get a machine response = await client.machines.get('machine-id') machine = response.data # Update a machine update_data = MachineUpdate(name='Updated Name') response = await client.machines.update('machine-id', update_data) # Delete a machine response = await client.machines.delete('machine-id') ``` ```python theme={null} from cyberdesk import MachineCreate, MachineUpdate # List machines response = client.machines.list_sync() machines = response.data.items # Create a machine machine_data = MachineCreate( name='Epic EHR Machine', description='Production Epic environment' ) response = client.machines.create_sync(machine_data) # Get a machine response = client.machines.get_sync('machine-id') # Update a machine update_data = MachineUpdate(name='Updated Name') response = client.machines.update_sync('machine-id', update_data) # Delete a machine response = client.machines.delete_sync('machine-id') ``` ```python theme={null} from cyberdesk import WorkflowCreate, WorkflowUpdate # List workflows response = await client.workflows.list() # Create a workflow workflow_data = WorkflowCreate( name='Patient Data Extraction', description='Extracts patient demographics and medications', main_prompt='Navigate to patient chart and extract data' ) response = await client.workflows.create(workflow_data) # Get a workflow response = await client.workflows.get('workflow-id') # Update a workflow update_data = WorkflowUpdate(description='Updated description') response = await client.workflows.update('workflow-id', update_data) # Delete a workflow response = await client.workflows.delete('workflow-id') ``` ```python theme={null} # List workflows response = client.workflows.list_sync() # Create a workflow workflow_data = WorkflowCreate( name='Patient Data Extraction', description='Extracts patient demographics and medications', main_prompt='Navigate to patient chart and extract data' ) response = client.workflows.create_sync(workflow_data) # Get a workflow response = client.workflows.get_sync('workflow-id') # Update a workflow update_data = WorkflowUpdate(description='Updated description') response = client.workflows.update_sync('workflow-id', update_data) # Delete a workflow response = client.workflows.delete_sync('workflow-id') ``` Upload and manage images for use in workflow prompts. The returned `supabase_url` can be embedded directly in workflow prompt HTML. ```python theme={null} # Upload an image from a file path response = await client.workflows.upload_prompt_image(file_path='./screenshot.png') if response.data: print(f"Supabase URL: {response.data.supabase_url}") print(f"Signed URL (for preview): {response.data.signed_url}") # Use the supabase_url in your workflow prompt HTML: # Screenshot # Upload from raw bytes with open('./image.png', 'rb') as f: image_bytes = f.read() response = await client.workflows.upload_prompt_image( file_content=image_bytes, filename='my-image.png', content_type='image/png' ) # List all prompt images response = await client.workflows.list_prompt_images() for img in response.data: print(f"{img.filename}: {img.supabase_url}") # Get a fresh signed URL for an existing image response = await client.workflows.get_prompt_image_signed_url( path='org_xxx/prompt-assets/my-image.png' ) print(f"Signed URL: {response.data.signed_url}") print(f"Expires in: {response.data.expires_in} seconds") # Delete an image response = await client.workflows.delete_prompt_image( path='org_xxx/prompt-assets/my-image.png' ) ``` ```python theme={null} # Upload an image from a file path response = client.workflows.upload_prompt_image_sync(file_path='./screenshot.png') if response.data: print(f"Supabase URL: {response.data.supabase_url}") # Use in workflow prompt: Screenshot # List all prompt images response = client.workflows.list_prompt_images_sync() for img in response.data: print(f"{img.filename}: {img.supabase_url}") # Get a fresh signed URL response = client.workflows.get_prompt_image_signed_url_sync( path='org_xxx/prompt-assets/my-image.png' ) # Delete an image response = client.workflows.delete_prompt_image_sync( path='org_xxx/prompt-assets/my-image.png' ) ``` **Using prompt images in workflows:** After uploading, copy the `supabase_url` and use it in your workflow's `main_prompt` HTML: ```html theme={null}

Click on the button shown in this screenshot:

Button to click

Then proceed to fill out the form.

``` Cyberdesk automatically resolves these URLs when running workflows, displaying the images to the AI agent.
```python theme={null} from cyberdesk import ConnectionCreate, ConnectionStatus # List connections response = await client.connections.list() # Create a connection connection_data = ConnectionCreate(machine_id='machine-id') response = await client.connections.create(connection_data) # Filter by machine and status response = await client.connections.list( machine_id='machine-id', status=ConnectionStatus.CONNECTED ) ``` ```python theme={null} # List connections response = client.connections.list_sync() # Create a connection connection_data = ConnectionCreate(machine_id='machine-id') response = client.connections.create_sync(connection_data) # Filter by machine and status response = client.connections.list_sync( machine_id='machine-id', status='connected' ) ``` ```python theme={null} from cyberdesk import TrajectoryCreate, TrajectoryUpdate # List trajectories response = await client.trajectories.list() # Get a trajectory response = await client.trajectories.get('trajectory-id') # Get latest trajectory for a workflow response = await client.trajectories.get_latest_for_workflow('workflow-id') # Create a trajectory trajectory_data = TrajectoryCreate( workflow_id='workflow-id', steps=[] ) response = await client.trajectories.create(trajectory_data) # Update a trajectory update_data = TrajectoryUpdate(steps=[]) response = await client.trajectories.update('trajectory-id', update_data) # Duplicate a trajectory (creates a copy with fresh image copies) response = await client.trajectories.duplicate('trajectory-id') # Delete a trajectory response = await client.trajectories.delete('trajectory-id') ``` ```python theme={null} # List trajectories response = client.trajectories.list_sync() # Get a trajectory response = client.trajectories.get_sync('trajectory-id') # Get latest trajectory for a workflow response = client.trajectories.get_latest_for_workflow_sync('workflow-id') # Create a trajectory trajectory_data = TrajectoryCreate( workflow_id='workflow-id', steps=[] ) response = client.trajectories.create_sync(trajectory_data) # Update a trajectory update_data = TrajectoryUpdate(steps=[]) response = client.trajectories.update_sync('trajectory-id', update_data) # Duplicate a trajectory (creates a copy with fresh image copies) response = client.trajectories.duplicate_sync('trajectory-id') # Delete a trajectory response = client.trajectories.delete_sync('trajectory-id') ``` Organize your workflows with tags. Tags support emojis, colors, and optional grouping for mutual exclusivity. ```python theme={null} from cyberdesk import WorkflowTagCreate, WorkflowTagUpdate # List all tags (with workflow counts) response = await client.workflow_tags.list() for tag in response.data: print(f"{tag.emoji or ''} {tag.name}: {tag.workflow_count} workflows") # Create a tag response = await client.workflow_tags.create( name="Production", emoji="🚀", color="green", description="Production-ready workflows" ) # Create a tag in a group (for mutual exclusivity) response = await client.workflow_tags.create( name="High Priority", emoji="🔴", group_id="priority-group-id" ) # Get a specific tag response = await client.workflow_tags.get('tag-id') # Update a tag response = await client.workflow_tags.update('tag-id', name="Updated Name", emoji="✨") # Archive a tag (soft delete - keeps on existing workflows) response = await client.workflow_tags.archive('tag-id') # Unarchive a tag response = await client.workflow_tags.unarchive('tag-id') # Delete a tag (hard delete) response = await client.workflow_tags.delete('tag-id') # Reorder tags (for drag-and-drop UI) response = await client.workflow_tags.reorder(['tag-3', 'tag-1', 'tag-2']) # Add tags to a workflow response = await client.workflow_tags.add_to_workflow( workflow_id='workflow-id', tag_ids=['tag-1', 'tag-2'] ) # Remove a tag from a workflow response = await client.workflow_tags.remove_from_workflow('workflow-id', 'tag-id') # Get all tags for a workflow response = await client.workflow_tags.get_for_workflow('workflow-id') # Bulk add tags to multiple workflows response = await client.workflow_tags.bulk_add_to_workflows( workflow_ids=['wf-1', 'wf-2', 'wf-3'], tag_ids=['production-tag-id'] ) ``` ```python theme={null} from cyberdesk import WorkflowTagCreate, WorkflowTagUpdate # List all tags (with workflow counts) response = client.workflow_tags.list_sync() for tag in response.data: print(f"{tag.emoji or ''} {tag.name}: {tag.workflow_count} workflows") # Create a tag response = client.workflow_tags.create_sync( name="Production", emoji="🚀", color="green" ) # Add tags to a workflow response = client.workflow_tags.add_to_workflow_sync('workflow-id', ['tag-1', 'tag-2']) # Archive/unarchive tags response = client.workflow_tags.archive_sync('tag-id') response = client.workflow_tags.unarchive_sync('tag-id') # Reorder tags response = client.workflow_tags.reorder_sync(['tag-3', 'tag-1', 'tag-2']) ``` **Mutual Exclusivity:** When a tag belongs to a group, adding it to a workflow automatically removes any other tag from the same group. This is useful for status-like tags (e.g., "Draft" vs "Published"). Group tags for organization and mutual exclusivity. Only one tag from a group can be assigned to a workflow at a time. ```python theme={null} from cyberdesk import WorkflowTagGroupCreate, WorkflowTagGroupUpdate # List all tag groups response = await client.workflow_tag_groups.list() for group in response.data: print(f"{group.emoji or ''} {group.name}") # Create a tag group response = await client.workflow_tag_groups.create( name="Priority", emoji="🔥", color="red", description="Priority levels - only one per workflow" ) # Get a specific group response = await client.workflow_tag_groups.get('group-id') # Update a group response = await client.workflow_tag_groups.update('group-id', name="Updated Priority") # Delete a group (tags become ungrouped, not deleted) response = await client.workflow_tag_groups.delete('group-id') # Reorder groups (for drag-and-drop UI) response = await client.workflow_tag_groups.reorder(['group-2', 'group-1', 'group-3']) ``` ```python theme={null} from cyberdesk import WorkflowTagGroupCreate # List all tag groups response = client.workflow_tag_groups.list_sync() # Create a tag group response = client.workflow_tag_groups.create_sync( name="Priority", emoji="🔥", color="red" ) # Reorder groups response = client.workflow_tag_groups.reorder_sync(['group-2', 'group-1', 'group-3']) ``` ```python theme={null} from cyberdesk import ModelConfigurationCreate, ModelConfigurationUpdate # List all model configurations (system defaults + org-owned) response = await client.model_configurations.list() configs = response.data # Create a custom model configuration config_data = ModelConfigurationCreate( name='My GPT-4o', provider='openai', model_id='gpt-4o', api_key=os.environ['OPENAI_API_KEY'], # Stored securely description='Custom OpenAI config with our API key' ) response = await client.model_configurations.create(config_data) # Get a specific configuration response = await client.model_configurations.get('config-id') # Update a configuration update_data = ModelConfigurationUpdate(name='Updated Name') response = await client.model_configurations.update('config-id', update_data) # Delete a configuration response = await client.model_configurations.delete('config-id') ``` ```python theme={null} from cyberdesk import ModelConfigurationCreate, ModelConfigurationUpdate # List all model configurations response = client.model_configurations.list_sync() # Create a custom model configuration config_data = ModelConfigurationCreate( name='My GPT-4o', provider='openai', model_id='gpt-4o', api_key=os.environ['OPENAI_API_KEY'], description='Custom OpenAI config with our API key' ) response = client.model_configurations.create_sync(config_data) # Get a specific configuration response = client.model_configurations.get_sync('config-id') # Update a configuration update_data = ModelConfigurationUpdate(name='Updated Name') response = client.model_configurations.update_sync('config-id', update_data) # Delete a configuration response = client.model_configurations.delete_sync('config-id') ``` ```python theme={null} from datetime import datetime from cyberdesk import UsageMode # Aggregate usage data for a date range response = await client.usage.aggregate( from_date=datetime(2025, 1, 1), to_date=datetime(2025, 1, 31), mode=UsageMode.SIMULATED # or UsageMode.BILLED for Stripe billing ) if response.data: usage = response.data print(f"Runs: {usage.runs_counted}") print(f"Agentic steps: {usage.total_agentic_steps}") print(f"Cached steps: {usage.total_cached_steps}") ``` ```python theme={null} from datetime import datetime from cyberdesk import UsageMode # Aggregate usage data for a date range response = client.usage.aggregate_sync( from_date=datetime(2025, 1, 1), to_date=datetime(2025, 1, 31), mode=UsageMode.SIMULATED ) if response.data: usage = response.data print(f"Runs: {usage.runs_counted}") print(f"Agentic steps: {usage.total_agentic_steps}") print(f"Cached steps: {usage.total_cached_steps}") ``` See [Usage-Based Billing](/additional-details/usage-based-billing#programmatic-usage-data) for more details.
## Error Handling All SDK methods return an `ApiResponse` object with `data` and `error` attributes: ```python theme={null} response = await client.runs.create(run_data) if response.error: status_code = getattr(response.error, 'status_code', None) if status_code == 401: raise Exception("Invalid API key") raise response.error validation_errors = getattr(response.data, 'detail', None) if validation_errors: print("Validation failed:", validation_errors) else: print(f"Run created: {response.data.id}") ``` ### Common Error Types * **Unexpected HTTP status / transport errors**: surfaced in `response.error` as exceptions. For HTTP failures, check `getattr(response.error, "status_code", None)`. * **Validation errors (`422`)**: returned in `response.data.detail`, not `response.error`. * **Successful responses**: returned in `response.data`. ### Exception Handling Pattern ```python theme={null} import logging logger = logging.getLogger(__name__) async def safe_run_creation(client, run_data): response = await client.runs.create(run_data) if response.error: logger.error("API error: %s", response.error) raise response.error validation_errors = getattr(response.data, "detail", None) if validation_errors: raise ValueError(f"Validation failed: {validation_errors}") return response.data ``` ## Type Hints and IDE Support The SDK provides comprehensive type hints for better IDE support: ```python theme={null} from cyberdesk import ( CyberdeskClient, RunResponse, RunStatus, MachineStatus, ConnectionStatus ) from typing import Optional def process_run(run: RunResponse) -> Optional[dict]: """Process a completed run.""" if run.status == RunStatus.SUCCESS: # IDE knows output_data is available return run.output_data elif run.status == RunStatus.TASK_FAILED: print(f"Run failed: {', '.join(run.error or [])}") return None else: print(f"Run status: {run.status}") return None ``` ## Working with Jupyter Notebooks The SDK works seamlessly in Jupyter notebooks: ```python theme={null} # In Jupyter, use sync methods or nest async code from cyberdesk import CyberdeskClient import pandas as pd client = CyberdeskClient('YOUR_API_KEY') # Get recent runs response = client.runs.list_sync(limit=10) runs = response.data.items # Convert to DataFrame for analysis df = pd.DataFrame([ { 'id': run.id, 'workflow_id': run.workflow_id, 'status': run.status, 'created_at': run.created_at, 'duration': (run.ended_at - run.created_at).total_seconds() if run.ended_at else None } for run in runs ]) # Analyze run performance df.groupby('status').agg({ 'id': 'count', 'duration': 'mean' }) ``` ## Best Practices Store API keys and workflow IDs in environment variables, never in code. The SDK automatically retries on transient failures with exponential backoff. Adjust `retry=RetryConfig(...)` if needed. Set reasonable timeouts for run completion based on your workflow complexity. Keep detailed logs of run IDs and statuses for debugging and audit trails. Leverage type hints for better IDE support and fewer runtime errors. Use context managers or explicitly close clients to free resources. ## Performance Optimization ### Concurrent Operations When working with multiple operations, use asyncio for better performance: ```python theme={null} import asyncio import os from cyberdesk import CyberdeskClient async def process_multiple_patients(patient_ids): """Process multiple patients concurrently.""" with CyberdeskClient(os.environ['CYBERDESK_API_KEY']) as client: # Create runs concurrently tasks = [ create_patient_run(client, patient_id) for patient_id in patient_ids ] runs = await asyncio.gather(*tasks) # Wait for all runs to complete results = await asyncio.gather(*[ wait_for_run_completion(client, run.id) for run in runs if run ]) return results # Process 10 patients in parallel results = asyncio.run(process_multiple_patients(patient_ids[:10])) ``` ### Connection Pooling The SDK automatically manages connection pooling for optimal performance. No additional configuration is needed. ## Next Steps Explore the complete API documentation Create and manage workflows in the dashboard Browse more code examples and use cases # TypeScript SDK Source: https://docs.cyberdesk.io/sdk-guides/typescript Complete guide for integrating Cyberdesk into your TypeScript/JavaScript applications ## Quick Start Get up and running with Cyberdesk in under 5 minutes. This guide assumes you've already created workflows in the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard). ```bash npm theme={null} npm install cyberdesk ``` ```bash yarn theme={null} yarn add cyberdesk ``` ```bash pnpm theme={null} pnpm add cyberdesk ``` ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; // Initialize the client const client = createCyberdeskClient('YOUR_API_KEY'); // Create a run for your workflow const { data: run } = await client.runs.create({ workflow_id: 'your-workflow-id', machine_id: 'your-machine-id', input_values: { patient_id: '12345', patient_first_name: 'John', patient_last_name: 'Doe' } }); // Wait for the run to complete let updatedRun = run; let status = updatedRun.status; while (status === 'scheduling' || status === 'running') { await new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5 seconds const { data: nextRun } = await client.runs.get(run.id); updatedRun = nextRun; status = updatedRun.status; } // Get the output data if (status === 'success') { console.log('Patient data:', updatedRun.output_data); } else { console.error('Run failed:', updatedRun.error?.join(', ')); } ``` We recommend creating and managing workflows through the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard). The dashboard editor supports rich, multimodal prompts — you can add screenshots or UI snippets directly into your prompt to guide the agent. The SDK is optimized for executing runs against your existing workflows. ## Installation & Setup ### Prerequisites * Node.js 18+ or another runtime with `fetch` support * TypeScript 4.0 or higher (for TypeScript projects) ### Installation ```bash theme={null} npm install cyberdesk ``` ```bash theme={null} yarn add cyberdesk ``` ```bash theme={null} pnpm add cyberdesk ``` ### TypeScript Configuration The SDK includes TypeScript definitions out of the box. For the best experience, ensure your `tsconfig.json` includes: ```json tsconfig.json theme={null} { "compilerOptions": { "target": "ES2020", "module": "commonjs", "strict": true, "esModuleInterop": true, "skipLibCheck": true } } ``` ## Authentication ### Creating a Client ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; const client = createCyberdeskClient('YOUR_API_KEY'); ``` ### Custom Base URL For self-hosted or enterprise deployments: ```typescript theme={null} const client = createCyberdeskClient('YOUR_API_KEY', 'https://api.your-domain.com'); ``` Never hardcode API keys in your source code. Use environment variables: ```typescript theme={null} const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!); ``` ## Automatic Retries The SDK automatically retries failed requests with exponential backoff, following industry best practices similar to the Stripe SDK. This handles transient network issues and server errors gracefully without requiring any code changes. ### Default Behavior * **Retry count**: 3 retries (4 total attempts) * **Retryable errors**: Network failures, fetch errors, connection timeouts * **Retryable status codes**: 408, 429, 500, 502, 503, 504, and `409` when the API returns `Idempotency-Status: in_progress` * **Non-retryable**: 4xx client errors (except 408, 409, 429) * **Backoff strategy**: Exponential backoff with full jitter (250ms initial, 8s max) * **Retry-After header**: Respected when present ### Configuring Retries ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; // Default: 3 retries (4 total attempts) const client = createCyberdeskClient('YOUR_API_KEY'); // More retries for unreliable networks const client = createCyberdeskClient('YOUR_API_KEY', undefined, { retry: { maxRetries: 5 } }); // Disable retries entirely const client = createCyberdeskClient('YOUR_API_KEY', undefined, { retry: { maxRetries: 0 } }); // Custom base URL with retries const client = createCyberdeskClient( 'YOUR_API_KEY', 'https://api.your-domain.com', { retry: { maxRetries: 3 } } ); ``` Retries only occur for network-level failures and specific HTTP status codes that indicate temporary server issues. Client errors (4xx) are not retried because they indicate a problem with the request itself. ## Idempotency For write requests (POST, PUT, PATCH, DELETE), the SDK automatically adds an `Idempotency-Key` header. This ensures that if a request is retried due to a network timeout or server error, it won't be processed twice — you'll get the same response as the original request. ### Default Behavior * **Enabled by default**: All write requests automatically include an idempotency key * **Key generation**: Uses `crypto.randomUUID()` (with fallback for older environments) * **Server-side handling**: The API stores the response and replays it for duplicate keys ### Configuring Idempotency ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; // Default: idempotency enabled with auto-generated keys const client = createCyberdeskClient('YOUR_API_KEY'); // Disable idempotency (not recommended for production) const client = createCyberdeskClient('YOUR_API_KEY', undefined, { idempotency: { enabled: false } }); // Custom key generator (e.g., for deterministic keys) const client = createCyberdeskClient('YOUR_API_KEY', undefined, { idempotency: { generateKey: () => `order-${orderId}-${Date.now()}` } }); ``` You generally don't need to configure idempotency — the defaults work well. The SDK handles everything automatically, making retries safe even for operations that create resources. ## Working with Runs Runs are the primary way to execute workflows in Cyberdesk. Here's everything you need to know about managing runs through the SDK. ### Creating a Run ```typescript theme={null} const { data: run, error } = await client.runs.create({ workflow_id: 'workflow-uuid', machine_id: 'machine-uuid', input_values: { patient_id: '12345', patient_first_name: 'John', patient_last_name: 'Doe', insurance: { provider: 'Blue Cross', policy_number: 'BC123' } } }); if (error) { console.error('Failed to create run:', error); } else { console.log('Run created:', run.id); } ``` Pass nested objects and access them in prompts with dot notation like `{insurance.provider}`. See [Structured Inputs](/concepts/structured-inputs). ### Prioritizing a Run Set `is_priority: true` when a run should be matched before normal queued runs: ```typescript theme={null} const { data: run, error } = await client.runs.create({ workflow_id: 'workflow-uuid', is_priority: true }); ``` The field defaults to `false`. Bulk creation and chain creation also accept `is_priority`; one value applies to every run created by that request. Priority affects the next eligible machine assignment only—it does not interrupt running work or let a later run bypass an earlier run in the same session or chain. In-place retries keep the run's existing priority unless the retry request supplies an `is_priority` override. Priority runs are processed before normal runs. A sustained stream of priority work can delay normal queued runs. See [Priority Runs](/concepts/priority-runs) for complete matching, session-ordering, and retry semantics. ### Creating a Run with Sensitive Input Values If your workflow prompt references sensitive variables using the `{$variable}` syntax (for example, `{$password}`), you can pass those values separately via `sensitive_input_values`. ```typescript theme={null} const { data: run, error } = await client.runs.create({ workflow_id: 'workflow-uuid', machine_id: 'machine-uuid', input_values: { // non-sensitive inputs patient_id: '12345' }, sensitive_input_values: { // sensitive inputs referenced in your prompt as {$password} password: 's3cr3tP@ss' } }); ``` Sensitive inputs are stored in a secure third‑party secret vault (Basis Theory) only for the duration of the run. They are not logged in Cyberdesk, and they are not sent to any LLMs. The values are only resolved at the last moment during actual computer actions (e.g., when typing). After the run completes, these sensitive values are deleted from the vault. On the dashboard, sensitive inputs are never displayed and will not be prefilled when repeating a run. ### Creating a Run with Machine Pools You can specify pool requirements when creating a run. This ensures your run is executed on a machine that belongs to ALL specified pools. This is especially useful for: * Running workflows on customer-specific machines * Requiring machines with specific software installed * Organizing machines by location or capability ```typescript theme={null} // Get pool IDs (typically from your configuration or database) const customerPoolId = 'pool-uuid-1'; // e.g., "Customer A" pool const excelPoolId = 'pool-uuid-2'; // e.g., "Has Excel" pool const { data: run, error } = await client.runs.create({ workflow_id: 'workflow-uuid', // Machine must be in BOTH pools (intersection, not union) pool_ids: [customerPoolId, excelPoolId], input_values: { patient_id: '12345', patient_first_name: 'John', patient_last_name: 'Doe' } }); if (error) { console.error('Failed to create run:', error); } else { console.log('Run created:', run.id); console.log('Will execute on machine in pools:', [customerPoolId, excelPoolId]); } ``` **Pool Matching Logic:** When you specify multiple pools, Cyberdesk will only select machines that belong to **ALL** specified pools (intersection). For example, if you specify `["Customer A", "Has Excel"]`, only machines that are in both pools will be considered. If you provide a `machine_id` when creating a run, `pool_ids` are ignored. Cyberdesk will only attempt the specified machine; if it's busy or unavailable, the run will wait until that machine is free (no fallback to other machines or pools). **Creating and Managing Pools:** While you can manage pools via the SDK, we recommend using the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard) for a more intuitive experience: 1. Navigate to any machine in the dashboard 2. Click on the machine to view its details 3. Add the machine to existing pools or create new pools 4. Assign multiple pools to organize machines by customer, capability, or location Common pool strategies: * **By Customer**: "Customer A", "Customer B", etc. * **By Software**: "Has Excel", "Has Chrome", "Has Epic EHR" * **By Environment**: "Production", "Staging", "Development" * **By Location**: "US-East", "EU-West", etc. ### Creating a Run with File Inputs You can attach files to a run at creation. This is useful for workflows that need to process or manipulate files on the remote machine. ```typescript theme={null} import { promises as fs } from 'fs'; import type { FileInput } from 'cyberdesk'; // Read a file and convert it to base64 const fileBuffer = await fs.readFile('path/to/your/file.txt'); const content = fileBuffer.toString('base64'); const fileInputs: FileInput[] = [ { filename: 'file.txt', content: content, target_path: 'C:/Users/Default/Desktop/file.txt', // Optional cleanup_imports_after_run: true // Optional } ]; const { data: run, error } = await client.runs.create({ workflow_id: 'workflow-uuid', file_inputs: fileInputs }); if (error) { console.error('Failed to create run:', error); } else { console.log('Run created with file attachment:', run.id); } ``` The name of the file, including its extension. The base64-encoded content of the file. The absolute path on the remote machine where the file should be saved. If not provided, it defaults to `~/CyberdeskTransfers/`. If `true`, the file will be deleted from the remote machine after the run completes (whether it succeeds or fails). Defaults to `false`. ### Listing Runs ```typescript theme={null} // List all runs const { data: runs } = await client.runs.list(); // List with pagination const { data: paginatedRuns } = await client.runs.list({ skip: 0, limit: 20 }); // Filter by status const { data: completedRuns } = await client.runs.list({ status: 'success' }); // Filter by workflow const { data: workflowRuns } = await client.runs.list({ workflow_id: 'workflow-uuid' }); ``` #### Faster lists with fields projection Return only selected fields using the `fields` option. This avoids large payloads (like `run_message_history`) and speeds up responses. Need to fetch screenshots referenced in `run_message_history`? See [Run Screenshots](/concepts/run-screenshots) for signed URL examples. * Base fields always included: `id`, `workflow_id`, `machine_id`, `status`, `created_at`. * Add more by passing the `fields` array. ```typescript theme={null} // Minimal (base fields only) const { data: minimal } = await client.runs.list(); // Include inputs only const { data: inputsOnly } = await client.runs.list({ fields: ['input_values'] // or [RunField.input_values] if using enum from SDK }); // Include a couple specific fields const { data: some } = await client.runs.list({ fields: ['input_values', 'session_id'] }); // Include attachments but skip history for speed const { data: noHistory } = await client.runs.list({ fields: ['input_attachment_ids', 'output_attachment_ids'] }); ``` ### Getting a Specific Run ```typescript theme={null} const { data: run, error } = await client.runs.get('run-uuid'); if (run) { console.log('Run status:', run.status); console.log('Output data:', run.output_data); } ``` ### Updating a Run Run updates are typically handled automatically by the Cyberdesk system. Manual updates are rarely needed. ```typescript theme={null} const { data: updatedRun } = await client.runs.update('run-uuid', { status: 'cancelled' }); ``` ### Deleting a Run ```typescript theme={null} const { error } = await client.runs.delete('run-uuid'); if (!error) { console.log('Run deleted successfully'); } ``` ### Retrying a Run (same run\_id) Use retry when you want to re-run the exact same run id, clearing outputs and optionally providing fresh inputs/files. **Sensitive values:** always re-send `sensitive_input_values` on retry; secrets are deleted after each run. **File inputs:** if you set `cleanup_imports_after_run`, files are deleted from the remote machine after the run; include `file_inputs` again if you need a fresh copy or when no input attachments exist (providing `file_inputs` replaces prior input attachments). **Regular inputs:** only send `input_values` if you want to change them; otherwise the previous ones are reused. ```typescript theme={null} // Replace inputs/files/sensitive values as needed; keeps same run_id const { data: retried, error } = await client.runs.retry('run-uuid', { // optional overrides input_values: { query: 'new query' }, sensitive_input_values: { password: process.env.APP_PASSWORD! }, file_inputs: [ // providing file_inputs replaces prior input attachments { filename: 'input.pdf', content: base64Pdf } ], // session controls (all optional) reuse_session: true, // default: keep existing session // session_id: 'existing-session-uuid', // release_session_after: true, // machine selection (optional) // machine_id: 'specific-machine-uuid', // pool_ids: ['pool-a', 'pool-b'], // used only when no machine_id is set }); if (error) { // Active runs (scheduling/running) cannot be retried console.error('Failed to retry run:', error); } ``` Behavior: * Retry is allowed only for terminal runs: success, task\_failed, error, or cancelled. * Outputs, history, and output attachments are always cleared. * Prior input attachments are kept unless you provide file\_inputs (then they are replaced). * If you provide sensitive\_input\_values, new secrets are created; otherwise sensitive aliases are cleared. * When a session\_id is present and the session is busy, immediate assignment is skipped and the retried run queues. * When machine\_id is provided, pool\_ids are ignored. ### Polling for Run Completion Here's a robust pattern for waiting for runs to complete: ```typescript theme={null} async function waitForRunCompletion(client: any, runId: string, timeoutMs = 300000) { const startTime = Date.now(); const pollInterval = 5000; // 5 seconds while (Date.now() - startTime < timeoutMs) { const { data: run, error } = await client.runs.get(runId); if (error) { throw new Error(`Failed to get run status: ${error}`); } if (run.status === 'success') { return run; } if ( run.status === 'error' || run.status === 'cancelled' || run.status === 'task_failed' ) { throw new Error(`Run ${run.status}: ${run.error?.join(', ') || 'Unknown error'}`); } await new Promise(resolve => setTimeout(resolve, pollInterval)); } throw new Error('Run timed out'); } // Usage try { const completedRun = await waitForRunCompletion(client, run.id); console.log('Output:', completedRun.output_data); } catch (error) { console.error('Run failed:', error); } ``` ## Working with File Attachments Manage files associated with your runs, such as input files uploaded at creation or output files generated by a workflow. ### Listing Run Attachments You can list all attachments for a specific run and filter them by type (`input` or `output`). ```typescript theme={null} // List all attachments for a run const { data: attachments } = await client.run_attachments.list({ run_id: 'run-uuid' }); // List only output attachments const { data: outputFiles } = await client.run_attachments.list({ run_id: 'run-uuid', attachment_type: 'output' }); ``` ### Downloading an Attachment There are two ways to download attachments depending on your use case: #### Method 1: Get a Download URL Get a signed URL that triggers automatic download when accessed. Perfect for web applications where you want to provide download links to users. ```typescript theme={null} // Get a download URL with custom expiration (default: 5 minutes) const { data } = await client.run_attachments.getDownloadUrl( 'attachment-uuid', 600 // 10 minutes ); if (data) { console.log(`Download URL: ${data.url}`); console.log(`Expires in: ${data.expires_in} seconds`); // You can use this URL in your web app // For example, in a React component: // Download File } ``` #### Method 2: Download Raw File Content Download the file content directly. The SDK will return the raw data which you can then save to a file or process further. ```typescript theme={null} import { promises as fs } from 'fs'; // Get the attachment metadata first const { data: attachmentInfo } = await client.run_attachments.get('attachment-uuid'); if (attachmentInfo) { // Download the file content const { data: fileData, error } = await client.run_attachments.download(attachmentInfo.id); if (fileData) { // For Node.js: Save to file const buffer = Buffer.from(fileData); await fs.writeFile(attachmentInfo.filename, buffer); console.log(`Downloaded ${attachmentInfo.filename}`); // For browsers: Create a Blob and download // const blob = new Blob([fileData]); // const url = URL.createObjectURL(blob); // const a = document.createElement('a'); // a.href = url; // a.download = attachmentInfo.filename; // a.click(); } } ``` ### Example: Upload, Process, and Download Here's a full example of a workflow that processes a file. 1. **Workflow Prompt**: `"Take the file at ~/CyberdeskTransfers/report.txt, add a summary to the end of it, and mark it for export."` 2. **Workflow Setting**: `includes_file_exports` is set to `true`. ```typescript theme={null} import { createCyberdeskClient, FileInput } from 'cyberdesk'; import { promises as fs } from 'fs'; async function main() { const client = createCyberdeskClient('YOUR_API_KEY'); // 1. Prepare and upload the input file const reportContent = "This is the initial report content."; const encodedContent = Buffer.from(reportContent).toString('base64'); const { data: run } = await client.runs.create({ workflow_id: "your-file-processing-workflow-id", file_inputs: [{ filename: "report.txt", content: encodedContent }] }); console.log(`Run started: ${run.id}`); // 2. Wait for the run to complete const completedRun = await waitForRunCompletion(client, run.id); console.log(`Run finished with status: ${completedRun.status}`); // 3. Find and download the output attachment if (completedRun.status === 'success') { const { data: outputAttachments } = await client.run_attachments.list({ run_id: completedRun.id, attachment_type: 'output' }); if (outputAttachments?.items?.length) { const processedReport = outputAttachments.items[0]; // Option 1: Get a download URL (for web apps) const { data: urlData } = await client.run_attachments.getDownloadUrl(processedReport.id); if (urlData) { console.log(`Download URL: ${urlData.url}`); console.log(`Valid for: ${urlData.expires_in} seconds`); } // Option 2: Download the processed file directly const { data: fileData } = await client.run_attachments.download(processedReport.id); if (fileData) { const processedContent = new TextDecoder().decode(fileData); console.log("\n--- Processed Report ---"); console.log(processedContent); console.log("------------------------"); } } else { console.log("No output files were generated."); } } } // Assuming waitForRunCompletion is defined as in the previous examples main(); ``` This example demonstrates the complete lifecycle: uploading a file with a run, executing a workflow that modifies it, and then retrieving the processed file from the run's output attachments. ### Bulk Creating Runs with Pools When creating multiple runs in bulk, you can also specify pool requirements. All runs will be distributed across machines that match the pool criteria. ```typescript theme={null} // Create 100 runs that require machines in specific pools const { data: result, error } = await client.runs.bulkCreate({ workflow_id: 'workflow-uuid', count: 100, pool_ids: ['customer-a-pool-id', 'excel-pool-id'], input_values: { task_type: 'data_extraction', priority: 'high' } }); if (result) { console.log(`Created ${result.created_runs.length} runs`); console.log(`Failed: ${result.failed_count}`); // All runs will execute on machines in both specified pools } ``` **Bulk Run Assignment:** When bulk creating runs with pool requirements, Cyberdesk attempts to assign each run to any available machine that meets the pool criteria. If no matching machine is available, runs remain in scheduling until one is free. No specific load balancing guarantees are made. ## Sessions and Chained Runs At its core, a session is a reservation of a single machine. While a session is active, that machine is dedicated to your session only — no unrelated runs will be scheduled onto it. This guarantees your multi‑step automations run back‑to‑back on the same desktop without interference. What you get from a session: * Exclusive access to one machine for the session's duration (strong scheduling guarantee) * Deterministic "step 1 → step 2 → …" behavior with no opportunistic interleaving **Chains** are a convenient way to create multiple runs that execute back‑to‑back in the same session. Instead of manually creating individual runs and managing their sequencing, you can define all your workflow steps upfront and let Cyberdesk handle the session management and execution order. ### Real‑world cases that require sessions * **EHR workflows**: Log into Epic, navigate to a specific patient, extract their data, then upload documents to their chart — all with no interruptions from other miscellaneous runs. * **Financial reporting**: Export monthly reports from your ERP system, transform the data in Excel, then re‑import the processed results — all back‑to‑back without interference. * **Document processing**: Download files from a web portal, process them with a local application, then upload the results back — ensuring no other runs interfere with your workflow. ### Passing data between steps with refs Once you have multiple workflows running in the same session, you'll often want to pass outputs from earlier steps as inputs to later ones. Refs make this seamless — simply reference a previous step's output using a JSON object: ```ts theme={null} { "$ref": "step1.outputs.result" } ``` The SDK type for this shape is `RefValue` (exported), but a plain object with a top‑level `$ref` string also works. The path on the right points to a prior step’s output field. ### Start a new session and run a chain (best when you know the whole sequence) ```ts theme={null} import { createCyberdeskClient, type WorkflowChainCreate } from 'cyberdesk' const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!) const chain: WorkflowChainCreate = { // Optional shared inputs are applied only to steps whose workflows declare those variables shared_inputs: { search_query: 'red panda facts' }, // Optional shared sensitive inputs available to all steps shared_sensitive_inputs: { api_key: 'shared-secret-key' }, // Attach files once at the beginning of the chain (applied to the first run) shared_file_inputs: [ // { filename: 'seed.txt', content: 'base64-...' } ], // Reserve a machine for the whole chain (either machine_id OR pool_ids) pool_ids: ['pool-with-chrome', 'customer-a'], keep_session_after_completion: false, steps: [ { workflow_id: 'step-1-workflow-id', session_alias: 'step1', inputs: { topic: 'red panda', }, sensitive_inputs: { username: 'user1', // Step-specific sensitive input password: 'secret123' } }, { workflow_id: 'step-2-workflow-id', session_alias: 'step2', inputs: { // Use output of step1 as an input to step2 search_query: { $ref: 'step1.outputs.result' } }, sensitive_inputs: { security_token: 'step2-token' // Step-specific sensitive input } } ] } const { data: chainResult, error } = await client.runs.chain(chain) if (error) throw new Error(String(error)) console.log('Session:', chainResult.session_id) console.log('Run IDs:', chainResult.run_ids) ``` Notes: * Provide `machine_id` to target a specific machine, or `pool_ids` to let Cyberdesk choose any machine that belongs to **all** specified pools (intersection). * The chain always runs on one reserved session. If you omit `session_id`, the API creates one for you and reserves a machine before step 1 starts. * `shared_inputs` are automatically filtered per workflow so each step only receives the variables it actually declares. * `shared_sensitive_inputs` are available to all steps, while `sensitive_inputs` in individual steps provide step-specific sensitive values. * `shared_file_inputs` are attached to the first run in the chain. ### Join an existing session If you already have a reserved session (e.g., created by a prior chain), you can reuse it: ```ts theme={null} const { data: chainResult } = await client.runs.chain({ session_id: 'existing-session-uuid', steps: [ { workflow_id: 'wf-a', session_alias: 'warmup' }, { workflow_id: 'wf-b', session_alias: 'extract', inputs: { query: 'current patient' } }, ] }) ``` This keeps the same reserved machine and any state/files already present on it. `machine_id`/`pool_ids` are ignored when `session_id` is provided. ### Keep the session alive after the chain If you want to leave the reservation active for a follow‑up chain or ad‑hoc steps: ```ts theme={null} await client.runs.chain({ pool_ids: ['customer-a'], keep_session_after_completion: true, steps: [ /* ... */ ] }) ``` Later, you can start a new chain with that `session_id` to continue from where you left off. ### Ad‑hoc sessions without a chain (start with a single run, then add more) You don't have to use a chain to benefit from sessions. You can start a session with a single run and then submit additional runs that reference the same `session_id`. ```ts theme={null} // 1) Start a brand new session using a normal run const { data: warmup } = await client.runs.create({ workflow_id: 'login-workflow-id', pool_ids: ['customer-a'], start_session: true, // Reserve a machine and begin a session input_values: { username: 'alice' } }) // Get the session to reuse and the reserved machine const sessionId = warmup.session_id! // 2) Run the next workflow in the same session (no other runs will interleave) const { data: step2 } = await client.runs.create({ workflow_id: 'search-workflow-id', session_id: sessionId, // Guarantees same machine & back‑to‑back scheduling input_values: { query: { $ref: 'step1.outputs.result' } // Refs are resolved server‑side within a session } }) // 3) Final run that releases the session when complete const { data: final } = await client.runs.create({ workflow_id: 'cleanup-workflow-id', session_id: sessionId, release_session_after: true, // Release the session after this run completes input_values: { cleanup: 'true' } }) ``` This approach is ideal when the next steps depend on external conditions (e.g., decide at runtime which workflow to run next) or when you want to keep the session open for a while and feed runs one at a time. ### Automatic session release with release\_session\_after When creating individual runs in a session (not using chains), you can use `release_session_after: true` to automatically release the session when that run completes (regardless of success or failure): ```ts theme={null} // This run will release the session after it completes const { data: finalRun } = await client.runs.create({ workflow_id: 'final-workflow-id', session_id: existingSessionId, release_session_after: true, input_values: { finalize: 'true' } }) ``` This is useful mainly as a convenience, so you don't have to decouple creating a session ending run and actually ending the session. Note: The session is released when the run completes, whether it succeeds, fails, or is cancelled. This ensures the session doesn't remain locked if something goes wrong. ### Detecting session completion via webhooks The `release_session_after` field on a run indicates whether this run released the session. You can use this in your webhook handler to detect when all runs in a session are complete: ```typescript theme={null} // In your webhook handler for "run_complete" events if (event.run.release_session_after === true) { // This run released the session - all runs in this session are done console.log(`Session ${event.run.session_id} was released by run ${event.run.id}`) console.log(`Final status: ${event.run.status}`) // 'success', 'error', or 'cancelled' } ``` This field is automatically set to `true` when: * You explicitly set `release_session_after: true` on a run * A chain completes with `keep_session_after_completion: false` (the last run gets this flag) * A run errors or is cancelled and causes the session to be released See [Detecting session completion via webhooks](/concepts/sessions-and-chains#detecting-session-completion-via-webhooks) for more details. ### Polling chain runs The chain API returns run\_ids in creation order; you can poll them individually, or [receive a webhook when any of those runs complete](/webhooks/quickstart) ```ts theme={null} const { data: chainRes } = await client.runs.chain(chain) for (const runId of chainRes.run_ids) { const run = await waitForRunCompletion(client, runId) console.log(run.status, run.output_data) } ``` ### Real‑world patterns * **Login + Work (Exclusive)**: Reserve a session, log into a thick client once, then run 5 workflows in sequence. No other jobs will touch that machine mid‑sequence. * **Search + Process with Refs**: Step 1 finds a record; Step 2 uses `{$ref: 'step1.outputs.id'}` to open/process; Step 3 posts results. All on the same desktop. * **Download → Transform → Export**: Files created by Step 1 are visible to Steps 2/3 because the session keeps the same working directory. If you provide a `machine_id` in a bulk run request, `pool_ids` are ignored for those runs. Each run will only target the specified machine; if it is busy, the run will wait for that machine rather than falling back to other machines or pools. ## Real-World Example: Healthcare Integration Here's a complete example of retrieving patient data from an Epic EHR system using Cyberdesk: ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; async function getPatientData(patientId: string, firstName: string, lastName: string) { const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!); try { // Create a run to fetch patient data const { data: run, error } = await client.runs.create({ workflow_id: '550e8400-e29b-41d4-a716-446655440000', // Your Epic workflow ID machine_id: '550e8400-e29b-41d4-a716-446655440001', // Your Epic machine ID input_values: { patient_id: patientId, patient_first_name: firstName, patient_last_name: lastName } }); if (error) { throw new Error(`Failed to create run: ${error}`); } console.log(`Fetching data for patient ${firstName} ${lastName} (${patientId})...`); // Wait for completion const completedRun = await waitForRunCompletion(client, run.id, 120000); // 2 minute timeout // Process the patient data const patientData = completedRun.output_data; return { patientId: patientId, demographics: patientData.demographics, medications: patientData.medications, vitals: patientData.recentVitals, lastUpdated: patientData.lastUpdated }; } catch (error) { console.error('Error fetching patient data:', error); throw error; } } // Express.js route example app.post('/api/patients/lookup', async (req, res) => { try { const { patient_id, first_name, last_name } = req.body; const patientData = await getPatientData(patient_id, first_name, last_name); res.json(patientData); } catch (error) { res.status(500).json({ error: 'Failed to fetch patient data' }); } }); ``` ## Other SDK Resources **Important:** While the SDK provides full CRUD operations for all Cyberdesk resources, we strongly recommend using the [Cyberdesk Dashboard](https://cyberdesk.io/dashboard) for managing these resources. The dashboard provides a more intuitive interface for: * Creating and editing workflows * Managing machines * Viewing connections * Analyzing trajectories The SDK methods below are provided for advanced use cases and automation scenarios. ```typescript theme={null} import type { PoolCreate, PoolUpdate, MachinePoolUpdate } from 'cyberdesk'; // List pools const { data: pools } = await client.pools.list(); // Create a pool const { data: pool } = await client.pools.create({ name: 'Customer A', description: 'All machines for Customer A' }); // Get a pool (with optional machine list) const { data: poolWithMachines } = await client.pools.get('pool-id', true); // Update a pool const { data: updated } = await client.pools.update('pool-id', { description: 'Updated description' }); // Add machines to a pool const { data: updatedPool } = await client.pools.addMachines('pool-id', { machine_ids: ['machine-1', 'machine-2'] }); // Remove machines from a pool await client.pools.removeMachines('pool-id', { machine_ids: ['machine-1'] }); // Get pools for a machine const { data: machinePools } = await client.machines.getPools('machine-id'); // Update a machine's pools const { data: machine } = await client.machines.updatePools('machine-id', { pool_ids: ['pool-1', 'pool-2', 'pool-3'] }); // Delete a pool await client.pools.delete('pool-id'); ``` ```typescript theme={null} // List machines const { data: machines } = await client.machines.list(); // Create a machine const { data: machine } = await client.machines.create({ name: 'Epic EHR Machine', description: 'Production Epic environment' }); // Get a machine const { data: machine } = await client.machines.get('machine-id'); // Update a machine const { data: updated } = await client.machines.update('machine-id', { name: 'Updated Name' }); // Delete a machine await client.machines.delete('machine-id'); ``` ```typescript theme={null} // List workflows const { data: workflows } = await client.workflows.list(); // Create a workflow const { data: workflow } = await client.workflows.create({ name: 'Patient Data Extraction', description: 'Extracts patient demographics and medications', main_prompt: 'Navigate to patient chart and extract data' }); // Get a workflow const { data: workflow } = await client.workflows.get('workflow-id'); // Update a workflow const { data: updated } = await client.workflows.update('workflow-id', { description: 'Updated description' }); // Delete a workflow await client.workflows.delete('workflow-id'); ``` Upload and manage images for use in workflow prompts. The returned `supabase_url` can be embedded directly in workflow prompt HTML. ```typescript theme={null} import { uploadWorkflowPromptImageV1WorkflowsPromptImagePost, listWorkflowPromptImagesV1WorkflowsPromptImagesGet, getWorkflowPromptImageSignedUrlV1WorkflowsPromptImageSignedUrlGet, deleteWorkflowPromptImageV1WorkflowsPromptImageDelete } from 'cyberdesk/client/sdk.gen'; // Upload an image const fileInput = document.querySelector('input[type="file"]') as HTMLInputElement; const file = fileInput.files?.[0]; if (file) { const { data } = await uploadWorkflowPromptImageV1WorkflowsPromptImagePost({ client: client._client, // Access the underlying client body: { file } }); if (data) { console.log(`Supabase URL: ${data.supabase_url}`); console.log(`Signed URL (for preview): ${data.signed_url}`); // Use the supabase_url in your workflow prompt HTML: // Screenshot } } // List all prompt images const { data: images } = await listWorkflowPromptImagesV1WorkflowsPromptImagesGet({ client: client._client }); for (const img of images || []) { console.log(`${img.filename}: ${img.supabase_url}`); } // Get a fresh signed URL for an existing image const { data: signedData } = await getWorkflowPromptImageSignedUrlV1WorkflowsPromptImageSignedUrlGet({ client: client._client, query: { path: 'org_xxx/prompt-assets/my-image.png' } }); if (signedData) { console.log(`Signed URL: ${signedData.signed_url}`); console.log(`Expires in: ${signedData.expires_in} seconds`); } // Delete an image await deleteWorkflowPromptImageV1WorkflowsPromptImageDelete({ client: client._client, query: { path: 'org_xxx/prompt-assets/my-image.png' } }); ``` **Using prompt images in workflows:** After uploading, copy the `supabase_url` and use it in your workflow's `main_prompt` HTML: ```html theme={null}

Click on the button shown in this screenshot:

Button to click

Then proceed to fill out the form.

``` Cyberdesk automatically resolves these URLs when running workflows, displaying the images to the AI agent.
```typescript theme={null} // List connections const { data: connections } = await client.connections.list(); // Create a connection const { data: connection } = await client.connections.create({ machine_id: 'machine-id' }); // Filter by machine const { data: machineConnections } = await client.connections.list({ machine_id: 'machine-id', status: 'active' }); ``` ```typescript theme={null} // List trajectories const { data: trajectories } = await client.trajectories.list(); // Get a trajectory const { data: trajectory } = await client.trajectories.get('trajectory-id'); // Get latest trajectory for a workflow const { data: latest } = await client.trajectories.getLatestForWorkflow('workflow-id'); // Create a trajectory const { data: trajectory } = await client.trajectories.create({ workflow_id: 'workflow-id', steps: [] }); // Update a trajectory const { data: updated } = await client.trajectories.update('trajectory-id', { steps: [/* updated steps */] }); // Duplicate a trajectory (creates a copy with fresh image copies) const { data: duplicated } = await client.trajectories.duplicate('trajectory-id'); // Delete a trajectory await client.trajectories.delete('trajectory-id'); ``` Organize your workflows with tags. Tags support emojis, colors, and optional grouping for mutual exclusivity. ```typescript theme={null} import type { WorkflowTagCreate, WorkflowTagResponse } from 'cyberdesk'; // List all tags (with workflow counts) const { data: tags } = await client.workflow_tags.list(); tags?.forEach(tag => console.log(`${tag.emoji || ''} ${tag.name}: ${tag.workflow_count} workflows`)); // Include archived tags const { data: allTags } = await client.workflow_tags.list({ include_archived: true }); // Create a tag const { data: tag } = await client.workflow_tags.create({ name: "Production", emoji: "🚀", color: "green", description: "Production-ready workflows" }); // Create a tag in a group (for mutual exclusivity) const { data: priorityTag } = await client.workflow_tags.create({ name: "High Priority", emoji: "🔴", group_id: "priority-group-id" }); // Get a specific tag const { data: tag } = await client.workflow_tags.get('tag-id'); // Update a tag const { data: updated } = await client.workflow_tags.update('tag-id', { name: "Updated Name", emoji: "✨" }); // Archive a tag (soft delete - keeps on existing workflows) const { data: archived } = await client.workflow_tags.archive('tag-id'); // Unarchive a tag const { data: unarchived } = await client.workflow_tags.unarchive('tag-id'); // Delete a tag (hard delete) await client.workflow_tags.delete('tag-id'); // Reorder tags (for drag-and-drop UI) await client.workflow_tags.reorder(['tag-3', 'tag-1', 'tag-2']); // Add tags to a workflow const { data: addedTags } = await client.workflow_tags.addToWorkflow( 'workflow-id', ['tag-1', 'tag-2'] ); // Remove a tag from a workflow await client.workflow_tags.removeFromWorkflow('workflow-id', 'tag-id'); // Get all tags for a workflow const { data: workflowTags } = await client.workflow_tags.getForWorkflow('workflow-id'); // Bulk add tags to multiple workflows await client.workflow_tags.bulkAddToWorkflows( ['wf-1', 'wf-2', 'wf-3'], ['production-tag-id'] ); ``` **Mutual Exclusivity:** When a tag belongs to a group, adding it to a workflow automatically removes any other tag from the same group. This is useful for status-like tags (e.g., "Draft" vs "Published"). Group tags for organization and mutual exclusivity. Only one tag from a group can be assigned to a workflow at a time. ```typescript theme={null} import type { WorkflowTagGroupCreate, WorkflowTagGroupResponse } from 'cyberdesk'; // List all tag groups const { data: groups } = await client.workflow_tag_groups.list(); groups?.forEach(group => console.log(`${group.emoji || ''} ${group.name}`)); // Create a tag group const { data: group } = await client.workflow_tag_groups.create({ name: "Priority", emoji: "🔥", color: "red", description: "Priority levels - only one per workflow" }); // Get a specific group const { data: group } = await client.workflow_tag_groups.get('group-id'); // Update a group const { data: updated } = await client.workflow_tag_groups.update('group-id', { name: "Updated Priority" }); // Delete a group (tags become ungrouped, not deleted) await client.workflow_tag_groups.delete('group-id'); // Reorder groups (for drag-and-drop UI) await client.workflow_tag_groups.reorder(['group-2', 'group-1', 'group-3']); ``` ```typescript theme={null} import type { ModelConfigurationCreate } from 'cyberdesk'; // List all model configurations (system defaults + org-owned) const { data: configs } = await client.model_configurations.list(); // Create a custom model configuration const { data: config } = await client.model_configurations.create({ name: 'My GPT-4o', provider: 'openai', model: 'gpt-4o', api_key: process.env.OPENAI_API_KEY, // Stored securely description: 'Custom OpenAI config with our API key' }); // Get a specific configuration const { data: config } = await client.model_configurations.get('config-id'); // Update a configuration const { data: updated } = await client.model_configurations.update('config-id', { name: 'Updated Name' }); // Delete a configuration await client.model_configurations.delete('config-id'); ``` ```typescript theme={null} // Aggregate usage data for a date range const { data: usage } = await client.usage.aggregate({ from_date: new Date('2025-01-01'), to_date: new Date('2025-01-31'), mode: 'simulated' // or 'billed' for customers on Stripe billing }); if (usage) { console.log(`Runs: ${usage.runs_counted}`); console.log(`Agentic steps: ${usage.total_agentic_steps}`); console.log(`Cached steps: ${usage.total_cached_steps}`); } ``` See [Usage-Based Billing](/additional-details/usage-based-billing#programmatic-usage-data) for more details.
## Error Handling All SDK methods return an object with `data` and `error` properties: ```typescript theme={null} const { data, error } = await client.runs.create({ workflow_id: 'workflow-id', machine_id: 'machine-id' }); if (error) { // Handle error console.error('Error details:', error); } else { // Use data console.log('Run created:', data.id); } ``` ### Common Error Types Invalid input parameters ```typescript theme={null} { error: { message: "Validation failed", details: { workflow_id: "Invalid UUID format" } } } ``` Invalid or missing API key ```typescript theme={null} { error: { message: "Authentication failed", status: 401 } } ``` Too many requests ```typescript theme={null} { error: { message: "Rate limit exceeded", status: 429, retryAfter: 60 } } ``` ## TypeScript Types The SDK exports all types for better IDE support: ```typescript theme={null} import type { MachineResponse, PoolResponse, PoolCreate, PoolUpdate, MachinePoolUpdate, WorkflowResponse, RunResponse, RunStatus, MachineStatus, ConnectionStatus } from 'cyberdesk'; // Use types in your code function handleRun(run: RunResponse) { if (run.status === 'success') { // TypeScript knows output_data exists console.log(run.output_data); } } ``` ## Best Practices Store API keys and workflow IDs in environment variables, never in code. The SDK automatically retries on transient failures with exponential backoff. Adjust `maxRetries` if needed. Set reasonable timeouts for run completion based on your workflow complexity. Keep detailed logs of run IDs and statuses for debugging and audit trails. ## Next Steps Explore the complete API documentation Create and manage workflows in the dashboard Browse more code examples and use cases # Detailed Webhook Guide Source: https://docs.cyberdesk.io/webhooks/detailed-guide End-to-end guidance for implementing Cyberdesk webhooks ## Introduction Webhooks are HTTPS POST requests sent from Cyberdesk to your service when events happen (e.g., a workflow run completes). They eliminate polling and let you react in real-time. The high-level steps: 1. Activate Webhooks in the Dashboard (creates a Svix Application for your org). 2. Add an endpoint URL and subscribe to `run_complete`. 3. Implement your endpoint to verify signatures and process events. 4. Test and monitor via the embedded Webhooks portal. ## Events and Event Types Cyberdesk webhooks are organized by event types (e.g., `run_complete`). You can browse the full, always-up-to-date list and their schemas in the Event Catalog inside the Webhooks portal. As of today, `run_complete` is the most commonly used type and fires when a workflow run reaches a terminal state (`success`, `error`, `task_failed`, or `cancelled`). Its payload includes the completed Run object, excluding `run_message_history` to keep webhook deliveries small and reliable. If you need message history, retrieve the run later through the API or SDK. ## Adding an Endpoint 1. Go to Dashboard → Webhooks. 2. Click "Add Endpoint" and enter your publicly reachable HTTPS URL. 3. Select the `run_complete` event type (or accept all for initial testing). 4. Copy the generated endpoint signing secret (`whsec_...`). Keep separate endpoints/secrets for each environment (dev, staging, prod). ## Testing Endpoints Use the "Testing" tools in the Webhooks portal to send example events to your endpoint. Inspect payloads, responses, and message attempts under Logs/Activity. You can replay messages individually or recover all failed messages since a timestamp. We’re building a managed Webhooks SDK with built‑in verification and typing. If you want this prioritized, please let the team know. ## Verifying Signatures Always verify signatures to ensure messages originate from Cyberdesk. We use Svix headers and signing format. Headers sent with every webhook: * `svix-id` – unique message id (use for idempotency) * `svix-timestamp` – Unix timestamp * `svix-signature` – signature over the raw request body ### Install dependencies ```bash theme={null} npm install cyberdesk svix # or yarn add cyberdesk svix # or pnpm add cyberdesk svix ``` ```bash theme={null} pip install cyberdesk svix # or poetry add cyberdesk svix # or pipenv install cyberdesk svix ``` ### Endpoint handlers ```ts theme={null} import express from "express"; import { Webhook, WebhookVerificationError } from "svix"; import { createCyberdeskClient } from "cyberdesk"; import type { RunCompletedEvent, RunResponse } from "cyberdesk"; const app = express(); app.use(express.raw({ type: "application/json" })); function assertRunCompletedEvent(x: unknown): asserts x is RunCompletedEvent { if (!x || (x as any).event_type !== "run_complete" || !(x as any).run) { throw new Error("Invalid run_complete payload"); } } app.post("/webhooks/cyberdesk", (req, res) => { const secret = process.env.SVIX_WEBHOOK_SECRET!; const wh = new Webhook(secret); const headers = { "svix-id": req.header("svix-id")!, "svix-timestamp": req.header("svix-timestamp")!, "svix-signature": req.header("svix-signature")!, }; try { const payload = wh.verify(req.body, headers); assertRunCompletedEvent(payload); const run: RunResponse = payload.run; // fully typed // process run res.status(200).end(); } catch (err) { if (err instanceof WebhookVerificationError) return res.status(400).end(); res.status(500).end(); } }); ``` ```python theme={null} from fastapi import FastAPI, Request, HTTPException from svix.webhooks import Webhook, WebhookVerificationError from openapi_client.cyberdesk_cloud_client.models.run_completed_event import RunCompletedEvent from openapi_client.cyberdesk_cloud_client.models.run_response import RunResponse from typing import cast import os app = FastAPI() @app.post("/webhooks/cyberdesk") async def cyberdesk_webhook(request: Request): secret = os.environ["SVIX_WEBHOOK_SECRET"] wh = Webhook(secret) payload = await request.body() headers = { "svix-id": request.headers.get("svix-id"), "svix-timestamp": request.headers.get("svix-timestamp"), "svix-signature": request.headers.get("svix-signature"), } try: data = wh.verify(payload, headers) evt = RunCompletedEvent.from_dict(data) run: RunResponse = cast(RunResponse, evt.run) # process run return {"ok": True} except WebhookVerificationError: raise HTTPException(status_code=400, detail="Invalid signature") ``` Verify against the exact raw request body. Do not re-stringify JSON before verification. ## Retry Mechanism Cyberdesk (via Svix) retries failed deliveries using exponential backoff: * Immediately * 5 seconds * 5 minutes * 30 minutes * 2 hours * 5 hours * 10 hours * 10 hours A response with HTTP 2xx is considered success. Avoid long processing in the webhook handler—enqueue work and return 200 quickly to prevent timeouts. Manual retries: from the portal you can resend single messages or recover all failed messages since a given time. ## Troubleshooting & Failure Recovery Common pitfalls: * Not using raw body for signature verification * Using the wrong endpoint secret (each endpoint has its own secret) * Returning non-2xx status for successful processing * Handler timeouts (do heavy work asynchronously) Failure recovery: * Re-enable disabled endpoints in the portal * Replay failed messages individually or recover all since a timestamp ## Using run\_complete Data `run_complete` includes the full `RunResponse`. Useful patterns: * Trigger the next workflow in your pipeline * Update your job table with `run.status` and `output_data` * Persist `output_attachment_ids` and fetch/download files as needed * Correlate `input_values` to your original request (e.g., patient\_id) * Understand sensitive inputs: plaintext secrets are never included in the payload. `run.sensitive_input_aliases` maps each sensitive input key (for example, `password`) to the secure secret identifier used during execution, so you can audit which sensitive variables were referenced without exposing their values. ### Example: TypeScript handler ```ts theme={null} import { createCyberdeskClient } from "cyberdesk"; import type { RunCompletedEvent, RunResponse } from "cyberdesk"; if (payload.event_type === "run_complete") { const run: RunResponse = (payload as RunCompletedEvent).run; await db.runs.upsert({ id: run.id, status: run.status, output: run.output_data }); // kick off a follow-up Cyberdesk run const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!); await client.runs.create({ workflow_id: process.env.NEXT_WORKFLOW_ID! }); if (run.output_attachment_ids?.length) { // display download links using Cyberdesk attachment APIs } } ``` ### Example: Python handler ```python theme={null} import os from cyberdesk import CyberdeskClient, RunCreate from openapi_client.cyberdesk_cloud_client.models.run_completed_event import RunCompletedEvent from openapi_client.cyberdesk_cloud_client.models.run_response import RunResponse from typing import cast evt = RunCompletedEvent.from_dict(data) run: RunResponse = cast(RunResponse, evt.run) save_status(run.id, run.status, run.output_data) client = CyberdeskClient(os.environ["CYBERDESK_API_KEY"]) client.runs.create_sync(RunCreate(workflow_id=os.environ["NEXT_WORKFLOW_ID"])) ``` ### React to Post-run Check failures `run_complete` is the best place to react to Post-run Checks, because the event fires only after those checks finish. The result data lives on `run.post_run_checks`. Important details: * `run.post_run_checks` is an array, not an object keyed by name * each item includes `name`, `status`, `error_message`, `messages`, and `matched_filenames` * if you want to access a specific check by name, keep names stable and unique in the workflow editor ```ts theme={null} function getFailedPostRunChecks(run: RunResponse) { return (run.post_run_checks ?? []).filter((check) => check.status !== "success"); } function getCheckByName(run: RunResponse, name: string) { return (run.post_run_checks ?? []).find((check) => check.name === name); } app.post("/webhooks/cyberdesk", (req, res) => { const secret = process.env.SVIX_WEBHOOK_SECRET!; const wh = new Webhook(secret); const headers = { "svix-id": req.header("svix-id")!, "svix-timestamp": req.header("svix-timestamp")!, "svix-signature": req.header("svix-signature")!, }; try { const payload = wh.verify(req.body, headers); assertRunCompletedEvent(payload); const run: RunResponse = payload.run; const failedChecks = getFailedPostRunChecks(run); const invoiceCheck = getCheckByName(run, "Invoice PDF exists"); if (failedChecks.length > 0) { console.log("failed checks:", failedChecks.map((check) => ({ name: check.name, status: check.status, error: check.error_message, messages: check.messages, matched: check.matched_filenames, }))); } if (invoiceCheck && invoiceCheck.status !== "success") { console.log("invoice check did not pass:", invoiceCheck); } res.status(200).json({ ok: true }); } catch (err) { res.status(400).json({ error: "Invalid signature" }); } }); ``` ```python theme={null} def get_failed_post_run_checks(run: RunResponse): return [check for check in (run.post_run_checks or []) if getattr(check.status, "value", check.status) != "success"] def get_check_by_name(run: RunResponse, name: str): return next((check for check in (run.post_run_checks or []) if check.name == name), None) @app.post("/webhooks/cyberdesk") async def cyberdesk_webhook(request: Request): secret = os.environ["SVIX_WEBHOOK_SECRET"] wh = Webhook(secret) payload = await request.body() headers = { "svix-id": request.headers.get("svix-id"), "svix-timestamp": request.headers.get("svix-timestamp"), "svix-signature": request.headers.get("svix-signature"), } try: data = wh.verify(payload, headers) evt = RunCompletedEvent.from_dict(data) run: RunResponse = cast(RunResponse, evt.run) failed_checks = get_failed_post_run_checks(run) invoice_check = get_check_by_name(run, "Invoice PDF exists") if failed_checks: print([ { "name": check.name, "status": getattr(check.status, "value", check.status), "error": check.error_message, "messages": check.messages, "matched": check.matched_filenames, } for check in failed_checks ]) if invoice_check and getattr(invoice_check.status, "value", invoice_check.status) != "success": print("invoice check did not pass", invoice_check) return {"ok": True} except WebhookVerificationError: raise HTTPException(status_code=400, detail="Invalid signature") ``` ## Advanced example: dynamic workflow chain webhook Some production integrations build the entire Cyberdesk chain up front, but the steps included in that chain depend on the initial request. For example, a charge-entry integration might always run an encounter setup workflow, skip remove/update/add workflows when their input arrays are empty, and always finish with a finalization workflow. In this pattern: * Create the chain with `client.runs.chain(...)` from your normal POST endpoint. * Store the returned `run_ids` in your database with their step order. * In your `run_complete` webhook, branch on every terminal status. * Treat non-success statuses before the final step as early chain stops; later runs may never emit webhooks. * When the final run has `release_session_after: true`, clean up any local locks or resources tied to that Cyberdesk session. ```python theme={null} import os from typing import Any, cast from cyberdesk import CyberdeskClient from openapi_client.cyberdesk_cloud_client.models.workflow_chain_create import ( WorkflowChainCreate, ) from fastapi import BackgroundTasks, FastAPI, HTTPException, Request from openapi_client.cyberdesk_cloud_client.models.run_completed_event import ( RunCompletedEvent, ) from openapi_client.cyberdesk_cloud_client.models.run_response import RunResponse from svix.webhooks import Webhook, WebhookVerificationError app = FastAPI() client = CyberdeskClient(os.environ["CYBERDESK_API_KEY"]) # Replace this with durable database tables in production. chains_by_run_id: dict[str, dict[str, Any]] = {} processed_messages: set[str] = set() def build_steps(batch: dict[str, Any]) -> list[dict[str, Any]]: steps = [ { "workflow_id": os.environ["WORKFLOW_1_ID"], "session_alias": "encounter", "inputs": { "FIN": batch["FIN"], "icd_codes": batch["icd_codes"], "notes": batch["notes"], }, } ] if batch.get("charges_to_remove"): steps.append({ "workflow_id": os.environ["WORKFLOW_2_ID"], "session_alias": "remove_charges", "inputs": {"charges_to_remove": batch["charges_to_remove"]}, }) if batch.get("charges_to_update"): steps.append({ "workflow_id": os.environ["WORKFLOW_3_ID"], "session_alias": "update_charges", "inputs": {"charges_to_update": batch["charges_to_update"]}, }) if batch.get("charges_to_add"): steps.append({ "workflow_id": os.environ["WORKFLOW_4_ID"], "session_alias": "add_charges", "inputs": {"charges_to_add": batch["charges_to_add"]}, }) steps.append({ "workflow_id": os.environ["WORKFLOW_5_ID"], "session_alias": "finalize", "inputs": {}, }) return steps @app.post("/charge-batches") def start_charge_batch(batch: dict[str, Any]): steps = build_steps(batch) chain = WorkflowChainCreate.from_dict({ "steps": steps, "keep_session_after_completion": False, }) response = client.runs.chain_sync(chain) if response.error or response.data is None: raise HTTPException(status_code=502, detail=str(response.error)) run_ids = response.data.run_ids for index, run_id in enumerate(run_ids): chains_by_run_id[run_id] = { "session_id": response.data.session_id, "step_number": index + 1, "total_steps": len(run_ids), "workflow_id": steps[index]["workflow_id"], } return { "session_id": response.data.session_id, "run_ids": run_ids, "included_steps": [step["session_alias"] for step in steps], } async def handle_run_terminal(run: RunResponse) -> None: record = chains_by_run_id.get(str(run.id)) if not record: return is_final_step = record["step_number"] == record["total_steps"] if not is_final_step and run.status != "success": # PLACEHOLDER: Notify the client, retry the chain, or create a human # review task. Later steps may never run after this early stop. await mark_chain_failed(record, run.status, run.error) return if is_final_step: if run.status == "success": # PLACEHOLDER: Mark the full batch complete. await mark_chain_complete(record) elif run.status == "cancelled": # PLACEHOLDER: Mark finalization cancelled after earlier steps. await mark_chain_cancelled(record) else: # PLACEHOLDER: Handle finalization failure after prior mutations. await mark_chain_failed(record, run.status, run.error) if run.release_session_after: # PLACEHOLDER: Clear local locks/resources for this session. await cleanup_session(record["session_id"]) return # PLACEHOLDER: Persist intermediate step success and wait for next webhook. await mark_step_success(record, run.output_data) async def mark_chain_complete(record: dict[str, Any]) -> None: ... async def mark_chain_cancelled(record: dict[str, Any]) -> None: ... async def mark_chain_failed( record: dict[str, Any], status: str, error: Any ) -> None: ... async def mark_step_success(record: dict[str, Any], output_data: Any) -> None: ... async def cleanup_session(session_id: str) -> None: ... @app.post("/webhooks/cyberdesk") async def cyberdesk_webhook(request: Request, background_tasks: BackgroundTasks): raw_body = await request.body() message_id = request.headers.get("svix-id") headers = { "svix-id": message_id, "svix-timestamp": request.headers.get("svix-timestamp"), "svix-signature": request.headers.get("svix-signature"), } try: data = Webhook(os.environ["SVIX_WEBHOOK_SECRET"]).verify(raw_body, headers) except WebhookVerificationError: raise HTTPException(status_code=400, detail="Invalid signature") if message_id in processed_messages: return {"ok": True, "duplicate": True} if message_id: processed_messages.add(message_id) evt = RunCompletedEvent.from_dict(data) run: RunResponse = cast(RunResponse, evt.run) background_tasks.add_task(handle_run_terminal, run) return {"ok": True} ``` ```ts theme={null} import express from "express"; import { Webhook, WebhookVerificationError } from "svix"; import { createCyberdeskClient } from "cyberdesk"; import type { RunCompletedEvent, RunResponse, WorkflowChainCreate } from "cyberdesk"; const app = express(); const rawJson = express.raw({ type: "application/json" }); const jsonBody = express.json(); const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!); // Replace these with durable database tables in production. const chainsByRunId = new Map(); const processedMessages = new Set(); function assertRunCompletedEvent(x: unknown): asserts x is RunCompletedEvent { if (!x || (x as any).event_type !== "run_complete" || !(x as any).run) { throw new Error("Invalid run_complete payload"); } } function buildSteps(batch: any): WorkflowChainCreate["steps"] { const steps: WorkflowChainCreate["steps"] = [ { workflow_id: process.env.WORKFLOW_1_ID!, session_alias: "encounter", inputs: { FIN: batch.FIN, icd_codes: batch.icd_codes, notes: batch.notes, }, }, ]; if (batch.charges_to_remove?.length) { steps.push({ workflow_id: process.env.WORKFLOW_2_ID!, session_alias: "remove_charges", inputs: { charges_to_remove: batch.charges_to_remove }, }); } if (batch.charges_to_update?.length) { steps.push({ workflow_id: process.env.WORKFLOW_3_ID!, session_alias: "update_charges", inputs: { charges_to_update: batch.charges_to_update }, }); } if (batch.charges_to_add?.length) { steps.push({ workflow_id: process.env.WORKFLOW_4_ID!, session_alias: "add_charges", inputs: { charges_to_add: batch.charges_to_add }, }); } steps.push({ workflow_id: process.env.WORKFLOW_5_ID!, session_alias: "finalize", inputs: {}, }); return steps; } app.post("/charge-batches", jsonBody, async (req, res, next) => { try { const steps = buildSteps(req.body); const { data, error } = await client.runs.chain({ steps, keep_session_after_completion: false, }); if (error || !data) throw new Error(String(error)); data.run_ids.forEach((runId, index) => { chainsByRunId.set(runId, { sessionId: data.session_id, stepNumber: index + 1, totalSteps: data.run_ids.length, workflowId: steps[index].workflow_id, }); }); res.json({ session_id: data.session_id, run_ids: data.run_ids, included_steps: steps.map((step) => step.session_alias), }); } catch (err) { next(err); } }); async function handleRunTerminal(run: RunResponse) { const record = chainsByRunId.get(run.id); if (!record) return; const isFinalStep = record.stepNumber === record.totalSteps; if (!isFinalStep && run.status !== "success") { // PLACEHOLDER: Notify the client, retry the chain, or create a human // review task. Later steps may never run after this early stop. await markChainFailed(record, run.status, run.error); return; } if (isFinalStep) { if (run.status === "success") { // PLACEHOLDER: Mark the full batch complete. await markChainComplete(record); } else if (run.status === "cancelled") { // PLACEHOLDER: Mark finalization cancelled after earlier steps. await markChainCancelled(record); } else { // PLACEHOLDER: Handle finalization failure after prior mutations. await markChainFailed(record, run.status, run.error); } if (run.release_session_after) { // PLACEHOLDER: Clear local locks/resources for this session. await cleanupSession(record.sessionId); } return; } // PLACEHOLDER: Persist intermediate step success and wait for next webhook. await markStepSuccess(record, run.output_data); } async function markChainComplete(record: unknown) {} async function markChainCancelled(record: unknown) {} async function markChainFailed(record: unknown, status: string, error: unknown) {} async function markStepSuccess(record: unknown, outputData: unknown) {} async function cleanupSession(sessionId: string) {} app.post("/webhooks/cyberdesk", rawJson, async (req, res) => { const messageId = req.header("svix-id"); const headers = { "svix-id": messageId!, "svix-timestamp": req.header("svix-timestamp")!, "svix-signature": req.header("svix-signature")!, }; try { const payload = new Webhook(process.env.SVIX_WEBHOOK_SECRET!).verify(req.body, headers); assertRunCompletedEvent(payload); if (messageId && processedMessages.has(messageId)) { return res.json({ ok: true, duplicate: true }); } if (messageId) processedMessages.add(messageId); await handleRunTerminal(payload.run); res.json({ ok: true }); } catch (err) { if (err instanceof WebhookVerificationError) return res.status(400).end(); res.status(500).end(); } }); ``` ## Idempotency Deduplicate using `svix-id` (header) or `event_id` (payload). Store processed ids and ignore duplicates. ## Finding webhooks by run ID In the Svix dashboard, you can search for webhooks by their message ID. For most runs, the message ID equals the run ID, making it easy to find the webhook for a specific run. **Retried runs:** If you [retry a run](/sdk-guides/python#retrying-a-run-same-run_id) (same run\_id), the second webhook will have a timestamped message ID (e.g., `abc123..._1702310400000`) to ensure uniqueness. The first webhook for any run will always use just the run ID. ## Security Checklist * Verify signatures and timestamp * Use HTTPS only * Rotate secrets when needed * Return 2xx promptly; process work asynchronously ## Payload Transformations If your webhook payloads are too large or you need to reshape the data before it reaches your endpoint, you can use **transformations** to modify the payload server-side. This is particularly useful for: * Extracting only the fields your system needs * Reformatting data to match your expected schema See [Webhook Transformations](/webhooks/transformations) for setup instructions and examples. ## See Also * Webhooks Quickstart * API Reference → Event types and Run schema * [Webhook Transformations](/webhooks/transformations) — Modify payloads before delivery # Downed Machines Webhook Source: https://docs.cyberdesk.io/webhooks/downed-machines Alert your systems when important Cyberdesk machines appear unreachable ## Overview The `downed_machines` webhook tells your system when one or more important machines appear to need attention. Most teams route this event directly into Slack so operators see machine issues where they already work. Cyberdesk only monitors machines that you explicitly opt in. On a desktop's detail page, enable **Downed machine alerts** to include that machine in the monitor. Machines are not monitored by default. This avoids alerts for desktops that are intentionally offline or only used occasionally. ## Send Machine Alerts To Slack The fastest way to use machine alerts is with Cyberdesk's existing [Slack integration](/webhooks/slack-integration). You do not need to host your own webhook receiver unless you want custom routing logic outside Slack. Go to Dashboard -> Webhooks -> Endpoints, click **+ Add Endpoint**, then open the destination dropdown in the top right. Choose **Send downed machine alerts to Slack**. Click **Connect to Slack**. Slack will ask you to choose the workspace and channel that should receive machine alerts. Approve the connection, and you'll return to Cyberdesk. The connector subscribes to the `downed_machines` event and includes a default Slack transformation that formats machine alerts into a structured Slack message. Click **Create**. That's it: newly downed opted-in machines will now post to your selected Slack channel. You can use the default Slack message as-is. If your team wants different wording, routing context, or Slack Block Kit layout, edit the endpoint's transformation code before creating it or any time later. The machine-alert Slack connector includes a default Svix transformation similar to this. You only need to edit it if you want to customize the final Slack message: ```javascript theme={null} function handler(webhook) { if (webhook.eventType !== "downed_machines") { return webhook; } const payload = webhook.payload || {}; const machines = Array.isArray(payload.machines) ? payload.machines : []; if (machines.length === 0) { webhook.cancel = true; return webhook; } const maxMachinesToShow = 8; const visibleMachines = machines.slice(0, maxMachinesToShow); const hiddenCount = Math.max(0, machines.length - visibleMachines.length); const occurredAt = payload.occurred_at ? new Date(payload.occurred_at).toLocaleString() : "unknown time"; function escapeMrkdwn(value) { return String(value ?? "") .replace(/&/g, "&") .replace(//g, ">"); } function truncate(value, maxLength) { const text = escapeMrkdwn(value || "Machine appears unreachable"); return text.length > maxLength ? `${text.slice(0, maxLength - 1)}...` : text; } const machineBlocks = visibleMachines.map((machine, index) => { const name = escapeMrkdwn(machine.name || machine.machine_id || "Unnamed machine"); const id = escapeMrkdwn(machine.machine_id || "unknown"); const status = escapeMrkdwn(machine.status || "unknown"); const reason = truncate(machine.reason, 180); return { type: "section", text: { type: "mrkdwn", text: `*:desktop_computer: ${index + 1}. ${name}*\n` + `*Status:* \`${status}\`\n` + `*Machine ID:* \`${id}\`\n` + `*Reason:* ${reason}` } }; }); const overflowBlocks = hiddenCount > 0 ? [ { type: "context", elements: [ { type: "mrkdwn", text: `+ ${hiddenCount} additional downed machine${hiddenCount === 1 ? "" : "s"} not shown. Open Cyberdesk for the full list.` } ] } ] : []; webhook.payload = { blocks: [ { type: "header", text: { type: "plain_text", text: "Cyberdesk Machine Alert", emoji: true } }, { type: "section", text: { type: "mrkdwn", text: `*:rotating_light: ${machines.length} monitored machine${machines.length === 1 ? "" : "s"} currently need attention.*\n` + "This alert fired because at least one opted-in machine newly entered the downed state." } }, { type: "context", elements: [ { type: "mrkdwn", text: `Event: \`${payload.event_type || webhook.eventType}\` | Org: \`${payload.organization_id || "unknown"}\` | Detected: ${occurredAt}` } ] }, { type: "divider" }, ...machineBlocks, ...overflowBlocks, { type: "divider" }, { type: "actions", elements: [ { type: "button", text: { type: "plain_text", text: "Open Desktops Dashboard" }, url: "https://cyberdesk.io/dashboard/desktops" } ] } ] }; return webhook; } ``` For screenshots and more detail on the Slack setup flow, see the full [Slack integration guide](/webhooks/slack-integration). For more on editing and testing transform code, see [Webhook Transformations](/webhooks/transformations). ## When It Fires Cyberdesk can detect an opted-in downed machine in two ways: * **Immediately during a run:** If Cyberdriver becomes inaccessible at startup or during execution, Cyberdesk confirms the failure and publishes the event without waiting for the recurring monitor. Run-time detection currently leaves the machine's connection state, availability, pool memberships, and routing target unchanged. * **Recurring monitoring:** Cyberdesk checks opted-in machines on a schedule even when they are not running automation. For a connected machine, Cyberdesk requests display dimensions once. A successful response immediately qualifies the machine as up. After a failed request, Cyberdesk retries for a maximum of three total attempts. Each attempt uses a 10-second request timeout, and the pauses between failed attempts escalate from 2 seconds to 4 seconds (with a trailing 6-second delay reserved if the attempt count grows). Any unsuccessful response, including HTTP 4xx, counts as a failed attempt. When confirmation happens immediately after a run-owned Cyberdriver failure, Cyberdesk also waits 2 seconds before the first dimensions probe so Cyberdriver can finish abandoned work from the timed-out request. A monitored machine appears down when either: * Its Cyberdesk status is not `connected`. * Its status is `connected`, but Cyberdesk cannot complete a lightweight display dimensions probe after retries. The webhook fires only when a **new** opted-in machine enters the down set. The payload still includes the full array of machines that currently appear down for the organization, so your alert can include context about any machines that were already down. Immediate run detection and recurring monitoring use the same atomic alert-state transition. If the recurring monitor runs immediately after a run-time alert, it sees the machine as already down and does not send a duplicate. If immediate delivery fails, the monitor can retry the alert on its next run. Cyberdesk does not send a separate recovery webhook when a machine comes back online. For run-time confirmation details and operator recovery steps, see [Cyberdriver Reachability Checks](/cyberdriver/automatic-machine-quarantine). ## Enable Monitoring For A Machine 1. Go to Dashboard -> Desktops. 2. Enable alerts from the **Alerts** checkbox in the desktops table, or open a desktop and enable **Downed machine alerts** in the Machine Status card. 3. In Dashboard -> Webhooks, make sure your Slack endpoint or webhook endpoint subscribes to `downed_machines`. ## Payload ```json theme={null} { "event_id": "550e8400-e29b-41d4-a716-446655440000", "event_type": "downed_machines", "occurred_at": "2026-07-08T18:30:00Z", "organization_id": "org_123", "machines": [ { "machine_id": "9f2c0c07-f694-4c8f-b4d5-5a6f936a9a75", "name": "checkout-worker-1", "organization_id": "org_123", "status": "connected", "last_seen": "2026-07-08T18:21:00Z", "reason": "Machine dimensions probe timeout" } ] } ``` ## Verify And Handle The Event ```ts theme={null} import express from "express"; import { Webhook } from "svix"; import type { DownedMachinesEvent } from "cyberdesk"; const app = express(); app.use(express.raw({ type: "application/json" })); function assertDownedMachinesEvent(x: unknown): asserts x is DownedMachinesEvent { if (!x || (x as any).event_type !== "downed_machines" || !Array.isArray((x as any).machines)) { throw new Error("Invalid downed_machines payload"); } } app.post("/webhooks/cyberdesk", (req, res) => { const wh = new Webhook(process.env.SVIX_WEBHOOK_SECRET!); const payload = wh.verify(req.body, { "svix-id": req.header("svix-id")!, "svix-timestamp": req.header("svix-timestamp")!, "svix-signature": req.header("svix-signature")!, }); assertDownedMachinesEvent(payload); for (const machine of payload.machines) { console.log(`${machine.name ?? machine.machine_id}: ${machine.reason}`); } res.status(204).send(); }); ``` ```python theme={null} import os from flask import Flask, request from svix.webhooks import Webhook app = Flask(__name__) @app.post("/webhooks/cyberdesk") def cyberdesk_webhook(): wh = Webhook(os.environ["SVIX_WEBHOOK_SECRET"]) payload = wh.verify( request.get_data(), { "svix-id": request.headers["svix-id"], "svix-timestamp": request.headers["svix-timestamp"], "svix-signature": request.headers["svix-signature"], }, ) if payload.get("event_type") != "downed_machines": return "", 204 for machine in payload["machines"]: print(f"{machine.get('name') or machine['machine_id']}: {machine['reason']}") return "", 204 ``` ## Recommended Alerting Pattern Use `event_id` for idempotency, then fan out to your alerting system. Because Cyberdesk only sends this event when a newly downed machine is found, your receiver usually does not need to dedupe repeated deliveries beyond Svix retry handling. Svix may retry deliveries when your endpoint fails or times out. Always verify signatures and treat `event_id` or the `svix-id` header as idempotency keys. # Webhooks Quickstart Source: https://docs.cyberdesk.io/webhooks/quickstart Receive run_complete events from Cyberdesk without polling ## Overview Webhooks notify your app when something happens in Cyberdesk. Instead of polling, subscribe to the event types you care about (see the Event Catalog in the Webhooks portal). The most common one is `run_complete`, which we’ll use below as an example. Typical flow: create a run with the SDK, return immediately to your user, and update your system when you receive the webhook. ## 1) Enable Webhooks in the Dashboard 1. Go to the Cyberdesk Dashboard → Webhooks. 2. Click "Activate Webhooks" (creates a Svix Application for your organization). 3. Click "Add Endpoint" and enter your HTTPS URL. 4. Subscribe to the event types you need (for getting started, `run_complete` is typical). 5. Copy the endpoint's signing secret (`whsec_...`). Each endpoint has its own secret. Keep separate endpoints/secrets for dev and prod. ## 2) Implement the Endpoint (verify signatures) ### Install dependencies ```bash theme={null} npm install cyberdesk svix # or yarn add cyberdesk svix # or pnpm add cyberdesk svix ``` ```bash theme={null} pip install cyberdesk svix # or poetry add cyberdesk svix # or pipenv install cyberdesk svix ``` We’re working on a managed Webhooks SDK with built‑in verification and typing. If you’d like this prioritized, please let the team know. ### Endpoint handlers ```ts theme={null} import express from "express"; import { Webhook, WebhookVerificationError } from "svix"; import type { RunCompletedEvent, RunResponse } from "cyberdesk"; const app = express(); app.use(express.raw({ type: "application/json" })); // verify requires raw body function assertRunCompletedEvent(x: unknown): asserts x is RunCompletedEvent { if (!x || (x as any).event_type !== "run_complete" || !(x as any).run) { throw new Error("Invalid run_complete payload"); } } app.post("/webhooks/cyberdesk", (req, res) => { const secret = process.env.SVIX_WEBHOOK_SECRET!; const wh = new Webhook(secret); const headers = { "svix-id": req.header("svix-id")!, "svix-timestamp": req.header("svix-timestamp")!, "svix-signature": req.header("svix-signature")!, }; try { const payload = wh.verify(req.body, headers); assertRunCompletedEvent(payload); // Narrowed type: payload is RunCompletedEvent; run is fully typed const run: RunResponse = payload.run; // idempotency: upsert on headers["svix-id"] or payload.event_id // handle success/error/cancelled res.status(200).end(); } catch (err) { if (err instanceof WebhookVerificationError) return res.status(400).end(); res.status(500).end(); } }); ``` ```ts theme={null} import { createCyberdeskClient } from "cyberdesk"; import type { RunResponse } from "cyberdesk"; async function onRunComplete(run: RunResponse) { const client = createCyberdeskClient(process.env.CYBERDESK_API_KEY!); // Example A: Start a follow-up workflow await client.runs.create({ workflow_id: process.env.NEXT_WORKFLOW_ID!, input_values: { summary: run.output_data?.summary } }); // Example B: Persist and enqueue for async processing await db.runs.upsert({ id: run.id, status: run.status, output: run.output_data }); await queue.enqueue("postprocess-run", { runId: run.id }); } ``` ```python theme={null} from fastapi import FastAPI, Request, HTTPException from svix.webhooks import Webhook, WebhookVerificationError from openapi_client.cyberdesk_cloud_client.models.run_completed_event import RunCompletedEvent from openapi_client.cyberdesk_cloud_client.models.run_response import RunResponse from typing import cast import os app = FastAPI() @app.post("/webhooks/cyberdesk") async def cyberdesk_webhook(request: Request): secret = os.environ["SVIX_WEBHOOK_SECRET"] wh = Webhook(secret) payload = await request.body() headers = { "svix-id": request.headers.get("svix-id"), "svix-timestamp": request.headers.get("svix-timestamp"), "svix-signature": request.headers.get("svix-signature"), } try: data = wh.verify(payload, headers) evt = RunCompletedEvent.from_dict(data) # Help some IDEs narrow the type for evt.run run: RunResponse = cast(RunResponse, evt.run) # idempotency: upsert on headers["svix-id"] or evt.event_id return {"ok": True} except WebhookVerificationError: raise HTTPException(status_code=400, detail="Invalid signature") ``` ```python theme={null} from cyberdesk import CyberdeskClient, RunCreate from openapi_client.cyberdesk_cloud_client.models.run_response import RunResponse async def on_run_complete(run: RunResponse): client = CyberdeskClient(os.environ["CYBERDESK_API_KEY"]) # Handle different statuses if run.status == "success": # Example A: Start a follow-up workflow follow_up = RunCreate( workflow_id=os.environ["NEXT_WORKFLOW_ID"], input_values={"summary": (run.output_data or {}).get("summary")} ) await client.runs.create(follow_up) # Example B: Persist and enqueue await db.upsert_run(run.id, run.status, run.output_data) await queue.enqueue("postprocess-run", {"run_id": run.id}) elif run.status == "error": # Handle errors await notify_error(run.id, run.error) ``` Type safety: * TypeScript: use a type guard (like `assertRunCompletedEvent`) to narrow the Svix‑verified payload to `RunCompletedEvent`, then type `run` as `RunResponse`. * Python: `RunCompletedEvent.from_dict(data)` returns a typed attrs object; some IDEs may still show `Any | RunResponse` for `evt.run`, so `cast(RunResponse, evt.run)` helps IDEs while remaining safe after validation. Note: The SDK uses attrs classes, not Pydantic, so use `from_dict()` not `model_validate()`. ## 3) Event payload Payloads vary by event type. Refer to the Event Catalog in the portal for up‑to‑date schemas. Example for `run_complete`: ```json theme={null} { "event_id": "uuid", "event_type": "run_complete", "occurred_at": "2025-08-16T19:19:44Z", "run": { /* Run data: id, workflow_id, status, error, output_data, input_values, attachment ids, created_at, ... */ } } ``` Use `run.status` to branch your logic and `output_data` or attachments to continue your process. `run_message_history` is omitted from webhook payloads to keep deliveries small and reliable. If you need message history, fetch the run later through the API or SDK. Sensitive inputs: plaintext secrets are never included in webhook payloads. If a run used sensitive variables, you'll see `run.sensitive_input_aliases`, which maps each sensitive input key (for example, `password`) to the secure secret identifier used during execution. Actual values stay in the secure vault and are deleted after completion. ## 4) Test * In the Webhooks tab, send a test event to your endpoint. * Use the Logs/Activity views to inspect payloads and delivery attempts, replay failures, and recover from downtime. ## Learn more Signature details, retries, troubleshooting, and recovery patterns. Reshape or reduce webhook payloads before they reach your endpoint. Handle Post-run Check results from `run_complete` events. Run multi-step workflows on one reserved session. # Slack Integration Source: https://docs.cyberdesk.io/webhooks/slack-integration Get notified in Slack when your runs complete ## Overview Cyberdesk has a native integration with Slack. When enabled, you'll receive Slack messages whenever a run completes, showing: * Run status (success, error, or cancelled) * Date and time information * A direct link to the run in the Cyberdesk dashboard * Key run details This is a great way to stay informed about your automation runs without constantly checking the dashboard. ## Setup **Prerequisite:** You must first enable webhooks for your organization. See the [Webhooks Quickstart](/webhooks/quickstart) to get started. Go to the **Webhooks** tab in the Cyberdesk dashboard. Click **"+ Add Endpoint"** to create a new webhook endpoint. In the "New Endpoint" page, click the **"Webhook"** dropdown (with a down chevron) and select **"Slack"**. Click **"Connect to Slack"**. This will redirect you to Slack where you can select which Slack workspace and channel you'd like to receive messages. Once you're done, click **"Allow"**, and you'll be brought back to Cyberdesk. You can edit the **transformation code** to customize how messages appear in Slack. The transformation has access to all fields in the run (`webhook.payload.run`), so you can: * Filter to only send messages for certain statuses (e.g., skip `cancelled` runs) * Customize the message format * Include specific fields from `input_values` or `output_data` The transformation supports [Slack Block Kit](https://api.slack.com/block-kit) syntax, giving you full control over message formatting with headers, sections, buttons, and more. Copy the example transformation code below and paste it into ChatGPT, Claude, or another LLM along with your request. For example: > "Modify this code to only send Slack messages for errored and cancelled runs, and include the workflow\_id in the title." **Give the LLM this context:** * The run object is accessed via `webhook.payload.run` * Available fields: `id`, `workflow_id`, `session_id`, `status` (`success`, `error`, `cancelled`), `error` (array of strings), `input_values`, `output_data`, `created_at`, `started_at`, `ended_at`, `release_session_after` * Set `webhook.cancel = true` to skip sending the message * The payload supports [Slack Block Kit](https://api.slack.com/block-kit) syntax for rich formatting This example uses Slack Block Kit to create rich messages with status, timestamps, duration, errors, and a button linking to the run: ```javascript theme={null} function handler(webhook) { if (webhook.eventType === 'run_complete') { const run = webhook.payload.run; // Filter out cancelled runs if (run.status === 'cancelled') { webhook.cancel = true; return webhook; } // Normalize "session closed" flag const isSessionClosed = run.status === 'success' && (run.release_session_after === true || run.release_session_after === 'true'); // Base title strings let baseTitle; if (run.status === 'success') { if (isSessionClosed) { baseTitle = "Cyberdesk - Run Successful + Session Closed!"; } else { baseTitle = "Cyberdesk - Run Successful!"; } } else if (run.status === 'error') { baseTitle = "Cyberdesk - Run Error!"; } else { baseTitle = "Cyberdesk - Run Notification"; } // Add emoji prefix based on status let titleText; if (run.status === 'success') { titleText = `🚀 ${baseTitle}`; } else if (run.status === 'error') { titleText = `🔥 ${baseTitle}`; } else { titleText = `🖥️ ${baseTitle}`; } const statusEmoji = { scheduling: ':hourglass:', running: ':runner:', success: ':white_check_mark:', error: ':bangbang:' }; const createdAt = run.created_at ? new Date(run.created_at) : null; const startedAt = run.started_at ? new Date(run.started_at) : null; const endedAt = run.ended_at ? new Date(run.ended_at) : null; function formatDuration(ms) { let seconds = Math.floor(ms / 1000); const hrs = Math.floor(seconds / 3600); seconds -= hrs * 3600; const mins = Math.floor(seconds / 60); seconds -= mins * 60; const parts = []; if (hrs > 0) parts.push(`${hrs} ${hrs === 1 ? 'hr' : 'hrs'}`); if (mins > 0) parts.push(`${mins} ${mins === 1 ? 'min' : 'mins'}`); parts.push(`${seconds} ${seconds === 1 ? 'sec' : 'secs'}`); return parts.join(' '); } let duration = ""; if (startedAt && endedAt) { duration = formatDuration(endedAt - startedAt); } // Build error blocks if any let errorBlock = []; if (Array.isArray(run.error) && run.error.length > 0) { errorBlock = [ { type: "section", text: { type: "mrkdwn", text: "*:warning: Errors:*" } }, { type: "section", text: { type: "mrkdwn", text: run.error.map(e => `• :x: \`${e}\``).join("\n") } }, { type: "divider" } ]; } // Main metadata fields const metaFields = [ { type: "mrkdwn", text: `*Run ID:*\n\`${run.id}\`` }, { type: "mrkdwn", text: `*Workflow ID:*\n\`${run.workflow_id}\`` }, { type: "mrkdwn", text: `*Status:*\n${statusEmoji[run.status] || ':grey_question:'} \`${run.status}\`` }, { type: "mrkdwn", text: `*Session Closed:*\n\`${isSessionClosed ? 'yes' : 'no'}\`` } ]; if (run.session_id) { metaFields.push({ type: "mrkdwn", text: `*Session ID:*\n\`${run.session_id}\`` }); } webhook.payload = { blocks: [ // Title { type: "header", text: { type: "plain_text", text: titleText, emoji: true } }, // Status + IDs + session info { type: "section", fields: metaFields }, { type: "divider" }, // Timing { type: "section", fields: [ { type: "mrkdwn", text: `*Created At:*\n${createdAt ? createdAt.toLocaleString() : "N/A"}` }, { type: "mrkdwn", text: `*Started At:*\n${startedAt ? startedAt.toLocaleString() : "N/A"}` }, { type: "mrkdwn", text: `*Ended At:*\n${endedAt ? endedAt.toLocaleString() : "N/A"}` }, { type: "mrkdwn", text: `*Duration:*\n\`${duration || "N/A"}\`` } ] }, { type: "divider" }, // Errors (if any) ...errorBlock, // Link button { type: "actions", elements: [ { type: "button", text: { type: "plain_text", text: "View Run in Cyberdesk" }, url: `https://cyberdesk.io/dashboard/runs/${run.id}` } ] } ] }; } return webhook; } ``` To only receive Slack messages when runs fail, use this transformation. It creates a rich error alert with all error details, timing information, and a direct link to investigate: ```javascript theme={null} function handler(webhook) { if (webhook.eventType === 'run_complete') { const run = webhook.payload.run; // Only notify on errors - skip success and cancelled runs if (run.status !== 'error') { webhook.cancel = true; return webhook; } // Parse timestamps const createdAt = run.created_at ? new Date(run.created_at) : null; const startedAt = run.started_at ? new Date(run.started_at) : null; const endedAt = run.ended_at ? new Date(run.ended_at) : null; function formatDuration(ms) { let seconds = Math.floor(ms / 1000); const hrs = Math.floor(seconds / 3600); seconds -= hrs * 3600; const mins = Math.floor(seconds / 60); seconds -= mins * 60; const parts = []; if (hrs > 0) parts.push(`${hrs} ${hrs === 1 ? 'hr' : 'hrs'}`); if (mins > 0) parts.push(`${mins} ${mins === 1 ? 'min' : 'mins'}`); parts.push(`${seconds} ${seconds === 1 ? 'sec' : 'secs'}`); return parts.join(' '); } let duration = "N/A"; if (startedAt && endedAt) { duration = formatDuration(endedAt - startedAt); } // Build error list let errorBlocks = []; if (Array.isArray(run.error) && run.error.length > 0) { errorBlocks = [ { type: "section", text: { type: "mrkdwn", text: "*:rotating_light: Error Details:*" } }, { type: "section", text: { type: "mrkdwn", text: run.error.map((e, i) => `${i + 1}. \`${e}\``).join("\n") } } ]; } else { errorBlocks = [ { type: "section", text: { type: "mrkdwn", text: "*:rotating_light: Error Details:*\n_No error message available_" } } ]; } // Build metadata fields const metaFields = [ { type: "mrkdwn", text: `*Run ID:*\n\`${run.id}\`` }, { type: "mrkdwn", text: `*Workflow ID:*\n\`${run.workflow_id}\`` } ]; if (run.session_id) { metaFields.push({ type: "mrkdwn", text: `*Session ID:*\n\`${run.session_id}\`` }); } metaFields.push({ type: "mrkdwn", text: `*Duration:*\n\`${duration}\`` }); webhook.payload = { blocks: [ // Alert header { type: "header", text: { type: "plain_text", text: "🔥 Cyberdesk - Run Failed!", emoji: true } }, // Context line with timestamp { type: "context", elements: [ { type: "mrkdwn", text: `Failed at ${endedAt ? endedAt.toLocaleString() : 'unknown time'}` } ] }, { type: "divider" }, // Error details ...errorBlocks, { type: "divider" }, // Metadata { type: "section", fields: metaFields }, { type: "divider" }, // Action buttons { type: "actions", elements: [ { type: "button", text: { type: "plain_text", text: "🔍 Investigate Run" }, style: "danger", url: `https://cyberdesk.io/dashboard/runs/${run.id}` } ] } ] }; } return webhook; } ``` Click **"Create"** to complete the configuration. ## You're done! You will now receive Slack messages whenever a run completes. The messages include a direct link to view the full run details in the Cyberdesk dashboard. You can create multiple Slack integrations pointing to different channels — for example, one channel for production runs and another for development. # Webhook Transformations Source: https://docs.cyberdesk.io/webhooks/transformations Transform webhook payloads before delivery to your endpoint Webhook transformations let you modify the payload before it's delivered to your endpoint. This is useful for: * **Reducing payload size** — Remove fields your endpoint does not need * **Reshaping the schema** — Transform the data structure to match what your system expects * **Filtering data** — Remove fields you don't need Transformations run server-side before the webhook is sent, so your endpoint receives exactly what you want. ## Enabling Transformations Navigate to the **Webhooks** tab in your Cyberdesk dashboard. Click on the **Endpoints** tab and select the endpoint you want to transform. Click on the **Advanced** sub-tab within the endpoint details. Toggle **Enable Transformation** to on. Click **Edit Transformation** next to the toggle to open the transformation editor. Write your JavaScript transformation function, then use the **Test** feature to verify it works with sample payloads. ## Writing a Transformation Transformations are JavaScript functions that receive the webhook data and return a modified version. The function must be named `handler` and receives an object with these properties: | Property | Description | | ----------------------- | ---------------------------------------------------------- | | `payload` | The webhook payload (JSON object) — modify as needed | | `method` | HTTP method (`"POST"` or `"PUT"`) — can be changed | | `url` | Endpoint URL — can be changed to redirect | | `eventType` | Event type string (changes ignored) | | `transformationsParams` | Object passed via the create-message API (changes ignored) | The function must return the same object, but may modify its properties as described above. You can also set these additional properties on the returned object: | Property | Description | | --------- | ---------------------------------------------------------------------- | | `cancel` | Boolean to cancel dispatch (defaults to `false`) | | `headers` | Object of HTTP headers to add (takes precedence over endpoint headers) | ### Example: Remove Output Data If your endpoint only needs run status and metadata, remove `output_data` to reduce payload size: ```javascript theme={null} function handler(webhook) { if (webhook.payload.run && webhook.payload.run.output_data) { delete webhook.payload.run.output_data; } return webhook; } ``` ### Example: Extract Only What You Need Keep just the essential fields for your downstream system: ```javascript theme={null} function handler(webhook) { const run = webhook.payload.run; // Replace payload with only the fields you need webhook.payload = { event_type: webhook.payload.event_type, run_id: run.id, status: run.status, output_data: run.output_data, workflow_id: run.workflow_id, completed_at: run.ended_at }; return webhook; } ``` ### Example: Cancel Webhooks Conditionally You can cancel delivery for certain events: ```javascript theme={null} function handler(webhook) { // Don't send webhooks for cancelled runs if (webhook.payload.run?.status === "cancelled") { webhook.cancel = true; } return webhook; } ``` Cancelled messages appear as successful dispatches in the logs but are not actually sent to your endpoint. ### Example: Add Custom Headers Add headers for routing or authentication on your end: ```javascript theme={null} function handler(webhook) { webhook.headers = { "X-Workflow-Id": webhook.payload.run?.workflow_id || "", "X-Run-Status": webhook.payload.run?.status || "" }; return webhook; } ``` ## Testing Your Transformation The transformation editor includes a test panel where you can: 1. Select an event type to get a sample payload 2. Or paste a custom payload 3. Run your transformation 4. See the resulting webhook that would be sent Always test your transformation before saving to ensure it produces the expected output. ## Common Issues Make sure the **Enable Transformation** toggle is on. The toggle and edit button are on the same line in the Advanced tab. Verify you're returning the modified `webhook` object from your `handler` function. If you forget to return, the original payload is sent. ```javascript theme={null} // ❌ Wrong - no return function handler(webhook) { delete webhook.payload.run.output_data; } // ✅ Correct - returns modified webhook function handler(webhook) { delete webhook.payload.run.output_data; return webhook; } ``` Check the browser console for syntax errors. Common issues: * Missing semicolons or brackets * Accessing properties on undefined (use optional chaining: `webhook.payload.run?.status`) * Invalid JSON in test payloads Your transformation might be throwing an error. Wrap risky operations in try/catch: ```javascript theme={null} function handler(webhook) { try { if (webhook.payload.run) { delete webhook.payload.run.output_data; } } catch (e) { // Log error but don't break delivery console.error("Transformation error:", e); } return webhook; } ``` The default `run_complete` payload omits `run_message_history`. If your payloads are still too large, remove other large fields such as `output_data` or create a minimal payload with only the fields you need. ## Learn More For detailed documentation on writing transformations, including advanced patterns and the complete API reference, see the [Svix Transformations documentation](https://docs.svix.com/transformations#how-to-write-a-transformation). # Zapier Integration Source: https://docs.cyberdesk.io/webhooks/zapier-integration Connect Cyberdesk to 6,000+ apps with Zapier ## Overview Cyberdesk has a native Zapier connector. When enabled, you can trigger Zapier automations whenever a run completes — connecting your desktop automation to thousands of apps like Google Sheets, Notion, Airtable, email, and more. Since Zapier Webhooks support arbitrary payloads, **no transformation code is required**. The full run data is sent directly to Zapier, where you can map fields to any action. ## Setup **Prerequisite:** You must first enable webhooks for your organization. See the [Webhooks Quickstart](/webhooks/quickstart) to get started. Go to the **Webhooks** tab in the Cyberdesk dashboard. Click **"+ Add Endpoint"** to create a new webhook endpoint. In the "New Endpoint" page, click the **"Webhook"** dropdown (with a down chevron) and select **"Zapier"**. You'll see instructions for getting a Zapier webhook URL: 1. Go to [Zapier](https://zapier.com) and create a new Zap 2. Choose **"Webhooks by Zapier"** as your trigger 3. Select **"Catch Hook"** as the trigger event 4. Copy the webhook URL Zapier provides 5. Paste it into the Cyberdesk endpoint configuration After pasting the URL and creating the endpoint, trigger a test run in Cyberdesk. Then click "Test trigger" in Zapier to see the payload structure and set up your action. Click **"Create"** to complete the configuration. ## You're done! Your Zapier Zap will now trigger whenever a run completes. The webhook payload includes run data — status, timestamps, `input_values`, `output_data`, errors, and more — which you can map to any Zapier action. `run_message_history` is omitted to keep deliveries small and reliable. ## Example use cases * **Log runs to Google Sheets** — Append a row for each completed run with status, duration, and outputs * **Send email notifications** — Email yourself or your team when runs fail * **Update Notion databases** — Track automation results in your project management system * **Post to Discord/Teams** — Notify a channel when important workflows complete * **Trigger follow-up workflows** — Chain Cyberdesk runs with other automation tools Use Zapier's **Filter** step to only continue the Zap for specific statuses (e.g., only on `error`) or when certain output values are present. # Copy to Clipboard Source: https://docs.cyberdesk.io/workflow-prompting/copy-to-clipboard Deterministic data extraction via clipboard as an alternative to vision-based extraction ## What is Copy to Clipboard? Copy to Clipboard is a deterministic extraction tool that executes `Ctrl+C` on the remote machine, captures the clipboard contents, and automatically saves the result as a runtime variable. It provides a fast, reliable alternative to vision-based extraction methods. In your prompts, always refer to this tool as `copy_to_clipboard` (lowercase, with underscores). ## Why This Tool Exists Cyberdesk offers two primary methods for extracting data from screens: **Vision-Based Extraction** ([`focused_action`](/workflow-prompting/focused-action), [`extract_prompt`](/workflow-prompting/extract-prompt)) * Uses AI to read and interpret screenshots * Best for: Complex layouts, tables, ambiguous text, visual verification * Trade-offs: Requires AI inference, slightly slower, interpretation-based **Clipboard-Based Extraction** (`copy_to_clipboard`) * Deterministic copy via `Ctrl+C`, captures exact clipboard text * Best for: Selectable/copyable fields—IDs, numbers, dates, text inputs * Trade-offs: **Only works if text is copyable**, requires explicit selection The `copy_to_clipboard` tool is ideal when you want fast, deterministic extraction without vision model inference—but **only works when text is selectable**. Use vision-based extraction when text is embedded in images, PDFs, charts, or other non-copyable formats. ## How It Works 1. Agent selects the text/data on screen (e.g., triple-click, drag selection, or keyboard shortcuts) 2. Agent calls `copy_to_clipboard` with a runtime variable key name 3. System executes `Ctrl+C` on the remote machine 4. Clipboard contents are captured and returned to the agent 5. Data is **automatically saved as a runtime variable**: `{{key_name}}` 6. The captured value can be used in subsequent workflow steps 7. At run completion, runtime values are included when generating final `output_data` JSON The clipboard data is automatically saved as a **runtime variable**, making it immediately available for: * Use in later workflow steps via `{{variable_name}}` syntax * **Automatic inclusion in the final workflow output** when you have an output schema defined This means you don't need to manually include clipboard values in your output—they're captured automatically alongside focused action observations when transforming to the final output JSON. ## Choosing Between Vision and Clipboard Extraction Both extraction methods are valid—choose based on your use case: ### Use Copy to Clipboard When: * **The text is selectable and copyable** (works with Ctrl+C) * You know exactly which field to select * You want deterministic, byte-for-byte accuracy * The field has consistent location/structure * Speed is critical (no AI inference needed) * You're extracting simple IDs, numbers, or short text ### Use Vision-Based Extraction When: * **The text cannot be copied** (rendered in images, PDFs, scanned documents, charts) * **The text is not selectable** (displayed as graphics, embedded in screenshots) * Layout is complex or changes between runs * Data is in tables or complex formats * You need to verify visual states or conditions * Text location is unpredictable * You need decision-making alongside extraction **Pro Tip**: For workflows with output schemas, `copy_to_clipboard` is often faster and more reliable for extracting known fields like IDs, dates, or account numbers. The captured values automatically flow into your final output data. ## When to Use Copy to Clipboard ### 1. Extracting Non-Selectable IDs ```text theme={null} "Triple-click on the account number field to select it, then use copy_to_clipboard with key name 'account_number' to capture and save the value as {{account_number}}" ``` ### 2. Capturing Protected Text ```text theme={null} "Right-click on the customer ID, select 'Copy' from the context menu, then use copy_to_clipboard with key name 'customer_id' to save the value" ``` ### 3. Getting Values from Legacy Systems ```text theme={null} "In the patient record screen, click into the MRN field and press Ctrl+A to select all. Then use copy_to_clipboard with key name 'patient_mrn' to capture the medical record number as {{patient_mrn}}" ``` ### 4. Extracting Data for Later Use ```text theme={null} "Navigate to the invoice details page and copy the invoice number. Use copy_to_clipboard with key name 'invoice_id' to save it. Later, type this value into the payment field using {{invoice_id}}" ``` ### 5. When Vision is Required (Non-Copyable Text) ```text theme={null} "On the dashboard, you'll see a revenue chart with the total displayed inside the graph. Since this is a chart image and the number isn't copyable text, use focused_action to read the revenue total from the chart and save it as {{revenue_total}}" ``` Vision-based extraction is **essential** when working with scanned documents, image-based UIs, charts, graphs, or any content where the text isn't selectable. If you try `copy_to_clipboard` on non-copyable content, it will capture whatever was previously on the clipboard (likely empty or wrong). ## How to Prompt for Copy to Clipboard ### Best Practices 1. **First Select the Text**: Describe how to select the data (triple-click, drag, Ctrl+A, etc.) 2. **Specify the Key Name**: Choose a clear, descriptive name for the runtime variable 3. **Use the Value Later**: Reference the variable as `{{key_name}}` in subsequent steps 4. **Be Specific**: Indicate exactly what text should be selected and copied ### Prompt Template ```text theme={null} "[Describe how to select the data], then use copy_to_clipboard with key name '[variable_name]' to save it as {{[variable_name]}}" ``` ## Real-World Examples ### Healthcare: Extracting Patient MRN ```text theme={null} "On the patient demographics screen, locate the Medical Record Number field. Triple-click on the MRN to select it, then use copy_to_clipboard with key name 'patient_mrn' to capture the value. Use this {{patient_mrn}} later when filling out the lab requisition form." ``` ### Finance: Capturing Account Numbers ```text theme={null} "Navigate to the account summary page. Click into the account number field and press Ctrl+A to select all. Use copy_to_clipboard with key name 'account_num' to save it as {{account_num}}. Later, you'll need to paste this into the transfer form using {{account_num}}" ``` ### Insurance: Extracting Claim IDs ```text theme={null} "After submitting the claim, the system displays a claim ID. Select this ID by triple-clicking on it, then use copy_to_clipboard with key name 'claim_id' to capture it. Include {{claim_id}} in the workflow output data." ``` ### Legal: Copying Case References ```text theme={null} "In the case management system, locate the case reference number in the header. Right-click and select 'Copy Reference', then use copy_to_clipboard with key name 'case_ref' to save it as {{case_ref}} for use in document filing." ``` ## Working with Runtime Variables ### Setting the Variable ```text theme={null} "Copy the generated report ID using copy_to_clipboard with key name 'report_id' to save it as {{report_id}}" ``` ### Using the Variable ```text theme={null} "Later in the workflow, navigate to the report lookup page and type {{report_id}} into the search field to retrieve the report" ``` ### Multiple Clipboard Operations ```text theme={null} "Extract the customer name by selecting it and using copy_to_clipboard with key name 'customer_name'. Then navigate to the order ID field, select it, and use copy_to_clipboard with key name 'order_id'. Finally, submit a form with both {{customer_name}} and {{order_id}}" ``` ## Common Selection Patterns ### Triple-Click Selection ```text theme={null} "Triple-click on the text field to select all contents, then use copy_to_clipboard with key name 'field_value'" ``` ### Ctrl+A Selection ```text theme={null} "Click into the input field, press Ctrl+A to select all, then use copy_to_clipboard with key name 'full_text'" ``` ### Drag Selection ```text theme={null} "Click and drag to select the account number from the table cell, then use copy_to_clipboard with key name 'account_number'" ``` ### Context Menu Copy ```text theme={null} "Right-click on the value and select 'Copy' from the menu, then immediately use copy_to_clipboard with key name 'copied_value' to capture what was copied" ``` ## Integration with Other Tools ### With Focused Action ```text theme={null} "Use focused_action to locate the row matching {customer_name} in the table. Once found, copy the order ID from that row using copy_to_clipboard with key name 'order_id', then use {{order_id}} to look up the order details." ``` ### With Terminal Commands ```text theme={null} "After copying the file path with copy_to_clipboard and saving it as {{file_path}}, use execute_terminal_command to run 'Get-Item {{file_path}} | ConvertTo-Json' to get file metadata" ``` ### With Workflow Output Schema ```text theme={null} "Define your workflow output schema as: { 'customer_id': 'string', 'order_number': 'string', 'order_date': 'string', 'total_amount': 'string' } During the workflow: - Triple-click the Customer ID field and use copy_to_clipboard with key name 'customer_id' - Triple-click the Order Number and use copy_to_clipboard with key name 'order_number' - Use focused_action to extract the order date and save as {{order_date}} - Triple-click the Total Amount and use copy_to_clipboard with key name 'total_amount' The runtime variables will automatically be included when generating the final output_data JSON." ``` **Automatic Output Generation**: When you define an output schema, Cyberdesk automatically transforms runtime values (from `copy_to_clipboard`) and focused action observations into the final structured output. You don't need to manually construct the output JSON—just extract the values and they'll be included automatically. ## Error Prevention **Common mistakes to avoid:** 1. Calling `copy_to_clipboard` without first selecting the text 2. Using the same key name multiple times (will overwrite previous values) 3. Forgetting to select text before copying 4. Not accounting for empty clipboard (if Ctrl+C fails) ### ❌ Incorrect Usage ```text theme={null} "Use copy_to_clipboard to get the account number" // Didn't select first "Copy the value as 'value'" // Not clear which value "Use copy to save it" // Wrong tool name ``` ### ✅ Correct Usage ```text theme={null} "Triple-click the account number field, then use copy_to_clipboard with key name 'account_number'" "Select the text by clicking and dragging, then use copy_to_clipboard with key name 'selected_text'" "Press Ctrl+A to select all, then use copy_to_clipboard with key name 'full_content'" ``` ## Advanced Patterns ### Extracting from Read-Only Fields ```text theme={null} "Many legacy systems have read-only fields that can't be typed into but can be copied. For the patient ID field (which is disabled), click on it three times to select the value, then use copy_to_clipboard with key name 'patient_id' to extract it." ``` ### Copying from Non-Input Elements ```text theme={null} "The order confirmation number appears as plain text (not in an input field). Click and drag to select it, then use copy_to_clipboard with key name 'confirmation_number' to save it as {{confirmation_number}}" ``` ### Extracting Multiple Values in Sequence ```text theme={null} "On the summary page, extract the following values by copying each: 1. Select the customer ID and use copy_to_clipboard with key name 'customer_id' 2. Select the order date and use copy_to_clipboard with key name 'order_date' 3. Select the total amount and use copy_to_clipboard with key name 'total_amount' Include all three values in the workflow output: {{customer_id}}, {{order_date}}, {{total_amount}}" ``` ### Combining with Data Validation ```text theme={null} "Copy the account number using copy_to_clipboard with key name 'account_num'. Then use execute_terminal_command to validate it: 'if ('{{account_num}}' -match '^\d{10}$') { Write-Output 'Valid' } else { Write-Output 'Invalid' }' If invalid, use declare_task_failed to stop the workflow." ``` ## Tips for Reliable Copying **Selection is Key**: The quality of your clipboard capture depends entirely on proper text selection. Be explicit about the selection method (triple-click, Ctrl+A, drag, etc.) **Wait After Selection**: If the system is slow, consider adding a brief wait between selection and copy: "wait 0.5 seconds, then use copy\_to\_clipboard" **Verify the Copy**: For critical data, you can verify: "use copy\_to\_clipboard with key name 'value', then type `{{value}}` into the verification field to confirm it was captured correctly" ## Clipboard vs Vision Extraction: Detailed Comparison | Aspect | Copy to Clipboard | Vision-Based ([focused\_action](/workflow-prompting/focused-action) / [extract\_prompt](/workflow-prompting/extract-prompt)) | | -------------------- | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- | | **Text Requirement** | ✅ Text must be selectable & copyable | ✅ Works with ANY visible text (even non-copyable) | | **Speed** | ⚡ Instant (no AI inference) | 🐌 2-5 seconds per extraction | | **Accuracy** | 📍 Deterministic, byte-exact | 🎯 Interpretation-based, may vary | | **Selection** | ✋ Requires explicit text selection | 👁️ Reads from screenshot automatically | | **Use Case** | Copyable fields, IDs, numbers, dates | Non-copyable text, images, charts, PDFs | | **Cost** | 💰 No AI tokens | 💳 Uses vision model tokens | | **Reliability** | ✅ If copy works, always succeeds | ⚠️ Depends on OCR/vision quality | | **Format** | 📝 Literal clipboard text | 🧠 AI interprets and formats | ### Real-World Scenario **Extracting an Account Number:** **With `copy_to_clipboard` (Deterministic)** ```text theme={null} "Triple-click the account number field and use copy_to_clipboard with key name 'account_number' to save it" ``` Result: `"1234567890"` (exact text, instant) **With `focused_action` (Vision-Based)** ```text theme={null} "Use focused_action to locate and extract the account number from the screen" ``` Result: `"Account number: 1234567890"` or `"1234567890"` (interpreted, may include extra text) **Choose clipboard for**: Selectable text fields (IDs, account numbers, dates, amounts) where you want exact text without interpretation. **Choose vision for**: Non-copyable content (images, PDFs, scanned docs, charts), tables, or when text isn't selectable/copyable. ## Example Workflow with Output Schema Here's a complete workflow that demonstrates clipboard extraction with automatic output generation: ### Define Output Schema ```json theme={null} { "patient_mrn": "string", "date_of_birth": "string", "latest_lab_result": "string", "lab_date": "string" } ``` ### Workflow Instructions ```text theme={null} "Log into the medical records system with username {username} and password {$password}. Navigate to the patient search and search for patient name {patient_name}. Click on the matching patient record to open their file. In the patient demographics section: 1. Triple-click on the Medical Record Number field and use copy_to_clipboard with key name 'patient_mrn' to capture it as {{patient_mrn}} 2. Triple-click on the Date of Birth field and use copy_to_clipboard with key name 'date_of_birth' to capture it as {{date_of_birth}} Navigate to the 'Recent Labs' tab. Use focused_action to find the most recent lab result row and extract: - The result value, save as {{latest_lab_result}} - The lab date, save as {{lab_date}} That's it! The runtime values (patient_mrn, date_of_birth, latest_lab_result, lab_date) will automatically be transformed into the final output_data JSON matching your schema." ``` ### What Happens Behind the Scenes 1. `copy_to_clipboard` captures exact text from fields → saved as runtime variables 2. `focused_action` extracts additional data → saved as runtime variables 3. At run completion, Cyberdesk transforms all runtime values into structured `output_data`: ```json theme={null} { "patient_mrn": "MRN12345", "date_of_birth": "1985-03-15", "latest_lab_result": "Negative", "lab_date": "2024-10-20" } ``` No manual output construction needed! Just extract the values with `copy_to_clipboard` and `focused_action`, and Cyberdesk handles the rest. ## Best of Both Worlds: Hybrid Extraction For maximum reliability and efficiency, combine both methods in a single workflow: ```text theme={null} "Extract the following customer data: Use copy_to_clipboard for copyable text fields (fast, deterministic): - Customer ID (triple-click the ID field, use copy_to_clipboard with key name 'customer_id') - Account Number (triple-click, use copy_to_clipboard with key name 'account_number') - Phone Number (triple-click, use copy_to_clipboard with key name 'phone') Use focused_action for non-copyable or complex content: - Purchase history table (use focused_action to extract as JSON array, save as {{purchase_history}}) - Account status badge image (use focused_action to read the status, save as {{account_status}}) - Scanned signature image (use focused_action to verify if signature is present, save as {{signature_verified}}) All values will be included in the final output_data JSON automatically." ``` **Hybrid Strategy**: Use `copy_to_clipboard` for selectable text fields (faster, deterministic), and `focused_action` for non-copyable content (images, PDFs, charts) or complex data that requires interpretation. You get speed where possible, and vision where necessary! ## Troubleshooting ### Clipboard is Empty If the clipboard returns empty after `copy_to_clipboard`: * Ensure text was properly selected before copying * Some applications block clipboard access - try a different selection method * Consider adding a small wait after selection: "wait 0.3 seconds before copying" * **Fallback**: Use `focused_action` to extract via vision instead ### Wrong Value Captured If the wrong text is captured: * Be more specific about the selection method * Use visual cues: "the field with the label 'Account #'" * Take a screenshot before copying to verify the selection * **Alternative**: Switch to `focused_action` for that specific field ### Application Doesn't Support Ctrl+C Some legacy applications may not support standard copy: * Try using the application's menu: "right-click and select Copy" * Use keyboard alternatives: "press F2 to edit, then Ctrl+A and Ctrl+C" * **Best solution**: Use `focused_action` with vision-based extraction instead ### Text is Not Selectable (Use Vision Instead) Some scenarios where clipboard won't work and you **must use vision-based extraction**: **Text in Images** ```text theme={null} ❌ "Copy the patient name from the scanned ID card image" ✅ "Use focused_action to read the patient name from the scanned ID card image" ``` **PDF Documents** ```text theme={null} ❌ "Copy the total from the PDF invoice" ✅ "Use focused_action to extract the invoice total from the PDF" ``` **Chart/Graph Data** ```text theme={null} ❌ "Copy the revenue number from the sales chart" ✅ "Use focused_action to read the revenue value shown in the sales chart" ``` **Non-Selectable UI Labels** ```text theme={null} ❌ "Copy the status text from the colored badge" ✅ "Use focused_action to read the status shown in the badge (likely 'Active', 'Pending', or 'Disabled')" ``` If text is displayed as an image, graphic, or in a non-selectable format (common in legacy systems, PDFs, or scanned documents), `copy_to_clipboard` will fail. Always use vision-based extraction (`focused_action` or `extract_prompt`) for non-copyable content. # Declare Task Failed Source: https://docs.cyberdesk.io/workflow-prompting/declare-task-failed Define failure conditions to terminate workflows when errors occur ## What is Declare Task Failed? Declare Task Failed is a specialized tool that allows your agent to immediately terminate workflow execution when specific failure conditions are met. This prevents the agent from continuing to attempt a task that cannot be completed, saving time and resources. In your prompts, always refer to this tool as `declare_task_failed` (lowercase, with underscores). ## Why This Tool Exists Not all workflow failures are the same. Some errors are recoverable, while others indicate fundamental issues that make task completion impossible: * **Authentication Failures**: Locked accounts or invalid credentials * **Missing Prerequisites**: Required data or systems unavailable * **Business Rule Violations**: Conditions that invalidate the entire process * **Technical Limitations**: System errors that cannot be resolved * **User-Defined Conditions**: Specific scenarios you define as failures Without `declare_task_failed`, the agent might spend excessive time trying to work around insurmountable obstacles. ## How It Works 1. The agent encounters a condition you've defined as a failure 2. It calls `declare_task_failed` with a description of the failure 3. Workflow execution immediately terminates 4. The run is marked with status `task_failed` (shown in the dashboard as **Task Failed**) 5. No trajectory is saved (the run is not cached) 6. The failure reason is recorded for debugging **Important**: Once `declare_task_failed` is called, the workflow stops immediately. This tool is never cached because failed runs don't create trajectories. Think of `task_failed` as a terminal run outcome, not an infrastructure crash. Cyberdesk still keeps the run's message history and failure reason, and you can retry that run later from the Runs UI or API. The part that is discarded is the pending trajectory cache for that failed path. `task_failed` is reserved for explicit `declare_task_failed` calls. Other failure paths (for example infrastructure or unexpected execution errors) continue using status `error`.\ You can filter runs by this status using the Runs list `status=task_failed` filter. ## When to Use Declare Task Failed ### 1. Authentication Errors ```text theme={null} "If you see 'Account Locked' or 'Invalid Credentials' after attempting login, use declare_task_failed to terminate the workflow. Do not attempt password recovery or multiple login attempts." ``` ### 2. Missing Critical Data ```text theme={null} "Search for patient ID {patient_id}. If no results are found after searching, use declare_task_failed indicating 'Patient ID {patient_id} not found in system'. Do not proceed with empty data." ``` ### 3. System Unavailability ```text theme={null} "If the application shows 'System Maintenance in Progress' or fails to load after 30 seconds, use declare_task_failed stating 'Target system unavailable'." ``` ### 4. Business Rule Violations ```text theme={null} "Check the account balance before processing. If balance is less than {required_amount}, use declare_task_failed with message 'Insufficient funds: balance below required amount'." ``` ## How to Prompt for Task Failure ### Best Practices 1. **Be Specific**: Clearly define what constitutes a failure 2. **Provide Context**: Explain why this is a failure condition 3. **Include Messages**: Specify what message to include when failing 4. **Set Boundaries**: Define how many attempts before failing 5. **Consider Timing**: Specify timeouts for time-sensitive operations ### Prompt Template ```text theme={null} "If [specific condition occurs], use declare_task_failed with message '[descriptive failure reason]'. Do not [what not to do]." ``` ## Real-World Examples ### Healthcare: Patient Safety ```text theme={null} "When searching for the patient's medication list, if you see any alert about 'Drug Interaction Warning - Contraindicated', immediately use declare_task_failed with message 'Critical drug interaction detected - manual review required'. Do not proceed with the prescription." ``` ### Finance: Compliance Checks ```text theme={null} "Before initiating the transfer, verify the recipient is not on the sanctions list. If the screen shows 'Compliance Alert - Restricted Entity', use declare_task_failed stating 'Transfer blocked - recipient on sanctions list'. Do not attempt workarounds." ``` ### E-commerce: Inventory Management ```text theme={null} "Check inventory for all items in order {order_id}. If any item shows 'Out of Stock' or 'Discontinued', use declare_task_failed with message 'Order cannot be fulfilled - {item_name} unavailable'. Do not process partial orders." ``` ### IT Operations: Deployment Safety ```text theme={null} "Before deploying, check the pre-deployment tests. If status shows 'FAILED' for any critical test, use declare_task_failed with 'Deployment aborted - critical test failures detected'. Do not override or skip failed tests." ``` ## Common Failure Patterns ### Multiple Attempt Failures ```text theme={null} "Attempt to connect to the database. If connection fails, retry up to 3 times with 10-second intervals. If still failing after 3 attempts, use declare_task_failed with 'Database connection failed after 3 attempts'." ``` ### Timeout Conditions ```text theme={null} "Click 'Generate Report' and wait for completion. If the progress bar doesn't finish within 5 minutes, use declare_task_failed stating 'Report generation timeout - exceeded 5 minute limit'." ``` ### Validation Failures ```text theme={null} "After filling the form, click Submit. If validation errors appear for required fields that cannot be populated from the provided data {input_data}, use declare_task_failed listing the missing required fields." ``` ### Permission Denials ```text theme={null} "Navigate to the admin panel. If you see 'Access Denied' or 'Insufficient Privileges', use declare_task_failed with 'User lacks required admin permissions'. Do not attempt to access via other routes." ``` ## Failure vs. Recovery Strategies ### When to Fail vs. Retry ```text theme={null} "If login fails with 'Account Locked', use declare_task_failed" "If the file is corrupted (shows 'Cannot read file'), use declare_task_failed" "If payment is declined with 'Card Reported Stolen', use declare_task_failed" ``` ```text theme={null} "If page doesn't load, refresh and try again" "If button click doesn't respond, wait 2 seconds and retry" "If search returns no results, try alternative search terms" ``` ### Cascading Failures ```text theme={null} "The workflow requires sequential approvals from 3 departments. If any department shows 'Request Denied' or 'Not Authorized', use declare_task_failed immediately with 'Approval chain broken at {department_name}'. Don't continue to other departments." ``` ## Integration with Other Tools ### With Focused Action ```text theme={null} "Use focused_action to check if an error message appears after submission. If the error contains 'Fatal', 'Critical', or 'Unrecoverable', use declare_task_failed with the full error message." ``` ### With Terminal Commands ```text theme={null} "Use execute_terminal_command to run 'Test-Path C:\RequiredFiles\config.xml'. If it returns False, use declare_task_failed with 'Required configuration file missing'." ``` ### Conditional Workflow Paths ```text theme={null} "Check the order status. If status is 'Cancelled' or 'Refunded', use declare_task_failed with 'Order {order_id} already {status} - no action needed'. Only proceed if status is 'Pending' or 'Processing'." ``` ## Advanced Usage ### Complex Failure Conditions ```text theme={null} "Monitor the batch processing progress. Use declare_task_failed if: 1. More than 10% of items fail processing 2. Any critical error (severity = 'CRITICAL') appears in logs 3. Processing time exceeds 30 minutes 4. System memory usage exceeds 90% Include specific failure reason in the message." ``` ### Partial Success Handling ```text theme={null} "Process all invoices in the list. Keep track of successes and failures. If more than 50% fail, use declare_task_failed with 'Batch processing failed - {failed_count} of {total_count} invoices failed'. Otherwise, complete successfully even with some failures." ``` ### Pre-flight Checks ```text theme={null} "Before starting the main workflow: 1. Verify all required applications are running 2. Check available disk space > 10GB 3. Confirm network connectivity to required services If any check fails, use declare_task_failed with specific check that failed. This prevents wasting time on doomed workflows." ``` ## Best Practices Summary 1. **Define clear, specific failure conditions** 2. **Fail fast when recovery is impossible** 3. **Include descriptive failure messages** 4. **Don't use for recoverable errors** 5. **Consider the business impact of failing vs. continuing** 6. **Set reasonable retry limits before failing** 7. **Document why each failure condition exists** 8. **Test failure scenarios to ensure proper handling** ## Common Mistakes to Avoid **Don't use declare\_task\_failed for:** * Temporary UI delays (use wait/retry instead) * Minor data variations (use focused\_action to adapt) * Recoverable errors (implement retry logic) * Success scenarios (even if no action needed) ### ❌ Incorrect Usage ```text theme={null} "If the page takes more than 2 seconds to load, use declare_task_failed" // Too aggressive "If anything goes wrong, use declare_task_failed" // Too vague "After completing successfully, use declare_task_failed" // Wrong tool ``` ### ✅ Correct Usage ```text theme={null} "If login fails with 'Account Suspended', use declare_task_failed" "If required file is missing after checking 3 locations, use declare_task_failed" "If the API returns 403 Forbidden, use declare_task_failed with 'API access denied'" ``` # Declare Task Succeeded Source: https://docs.cyberdesk.io/workflow-prompting/declare-task-succeeded Allow focused actions to signal workflow completion when success conditions are met ## What is Declare Task Succeeded? Declare Task Succeeded is a specialized tool that allows a **focused action agent** to immediately terminate workflow execution with a success status when specific success conditions are met. In production, this is an explicit opt-in tool: wire it into a `focused_action` prompt only when you want that focused action to be allowed to end the entire workflow early. In your prompts, always refer to this tool as `declare_task_succeeded` (lowercase, with underscores). **Focused Action Only**: This tool is only available within `focused_action`. The main agent and recovery agent can already signal success by completing their task normally (sending a final message without a tool call). Use `declare_task_succeeded` only when your workflow instructions explicitly tell the focused action to call `declare_task_succeeded` if a named success condition is met. Think of `declare_task_succeeded` as an early-exit switch that you intentionally wire into a focused action prompt, not as a general "the agent can decide to stop whenever it thinks the task is done" behavior. ## Why This Tool Exists During cached workflow replay, the main agent is not invoked—actions are replayed deterministically from the trajectory. However, `focused_action` always runs dynamically, even during cached runs. This creates situations where: * A focused action discovers the task is already complete (e.g., data was already processed) * A user-defined success condition is met earlier than expected * The focused action determines no further steps are needed Without `declare_task_succeeded`, the focused action would return to the trajectory replay, which would continue executing remaining steps unnecessarily. ## How It Works 1. A focused action evaluates the current screen state 2. Your prompt tells that focused action exactly when it should call `declare_task_succeeded` 3. If that success condition is met, it calls `declare_task_succeeded` (optionally with a description of why) 4. Workflow execution immediately terminates with success status 5. Trajectories are committed (the successful path is saved) 6. Post-run Checks run normally, unless the focused action called `declare_task_succeeded` with `skip_post_run_checks=True` 7. Normal success cleanup occurs (status update, webhooks, etc.) The `text` parameter is optional. If omitted, a default "Task completed successfully" message is used. `declare_task_succeeded` also accepts `skip_post_run_checks=True`. Use this only when the prompt explicitly says the early-success path should skip Post-run Checks, such as when the task was already complete before the workflow produced the attachments or output data that those checks normally verify. Unlike `declare_task_failed` which clears pending trajectories, `declare_task_succeeded` commits them because the workflow reached a valid success state. Explicit success still goes through the normal terminal output-processing path. If a workflow promises structured `output_data` and that final transformation fails, Cyberdesk preserves the pending trajectories but surfaces the run as `error` instead of silently returning `success` with missing output. ## When to Use Declare Task Succeeded ### 1. Early Success Detection When a focused action discovers the goal is already achieved and your prompt explicitly allows early success: ```text theme={null} "Use focused_action to check if the invoice has already been processed. If you see 'Status: Paid' or 'Status: Complete', use declare_task_succeeded with message 'Invoice already processed - no action needed'." ``` ### 2. Conditional Workflow Completion When success depends on dynamic content evaluation and you want the focused action to end the run immediately: ```text theme={null} "Use focused_action to verify the data migration status. If the progress shows '100% Complete' and no errors are listed, use declare_task_succeeded with 'Migration completed successfully'." ``` ### 3. Goal-Based Termination When the focused action can determine the workflow's goal is met and you have explicitly granted it permission to stop the run: ```text theme={null} "Use focused_action to check if the patient's appointment has been confirmed. If you see the confirmation number and 'Appointment Scheduled' message, use declare_task_succeeded with the confirmation details." ``` ## How to Prompt for Task Success ### Best Practices 1. **Be Specific**: Clearly define what constitutes success 2. **Explain the Condition**: Describe what the focused action should look for 3. **Write the Exact Tool Name**: Include the literal phrase `declare_task_succeeded` in the prompt when you want to enable this behavior 4. **Include Context**: Specify what message to include when succeeding 5. **Use in Focused Actions**: Remember this is only for dynamic success detection ### Prompt Template ```text theme={null} "Use focused_action to [check specific condition]. If [success condition is met], use declare_task_succeeded with message '[descriptive success reason]'. Otherwise, continue normally." ``` ### Skipping Post-run Checks If your workflow has Post-run Checks that only make sense after the normal path creates files, screenshots, or structured output, tell the focused action to use `skip_post_run_checks=True` only for the early-success case: ```text theme={null} "Use focused_action to check if {invoice_id} is already marked Paid. If it is already Paid, call declare_task_succeeded with skip_post_run_checks=True and message 'Invoice {invoice_id} was already paid'. Otherwise, continue normally so the workflow can produce the usual attachments and output data for Post-run Checks." ``` When this parameter is set, Cyberdesk marks each configured Post-run Check as `success` and records a message explaining that it was skipped because `declare_task_succeeded` was called with `skip_post_run_checks=True`. Cyberdesk also does not generate a reusable trajectory for that early-success path, since it represents an idempotent shortcut rather than the workflow's normal repeatable path. ## Real-World Examples ### Healthcare: Record Already Updated ```text theme={null} "Use focused_action to navigate to the patient's medication list and check if {medication_name} with dosage {dosage} is already present. If the medication is already listed with the correct dosage, use declare_task_succeeded with 'Medication {medication_name} already exists in patient record'." ``` ### Finance: Transaction Already Processed ```text theme={null} "Use focused_action to search for transaction ID {transaction_id}. If the transaction shows 'Status: Completed' with the correct amount {amount}, use declare_task_succeeded with 'Transaction {transaction_id} already processed'." ``` ### E-commerce: Order Already Fulfilled ```text theme={null} "Use focused_action to check the order status for {order_id}. If the order shows 'Shipped' or 'Delivered' with a tracking number, use declare_task_succeeded with 'Order {order_id} already fulfilled - tracking: [tracking number]'." ``` ### IT Operations: System Already Configured ```text theme={null} "Use focused_action to verify the server configuration. If all required settings match the expected values and the health check shows 'Healthy', use declare_task_succeeded with 'Server configuration verified - all checks passed'." ``` ## Comparison: declare\_task\_succeeded vs declare\_task\_failed | Aspect | declare\_task\_succeeded | declare\_task\_failed | | ---------------- | ---------------------------------------------------------------------- | --------------------------------------------------------- | | **Purpose** | Signal successful completion | Signal unrecoverable failure | | **Trajectory** | Committed (saved) | Cleared (not saved) | | **Run Status** | Usually `success` | Usually `task_failed` for explicit agent-declared failure | | **Use Case** | Goal achieved early | Insurmountable obstacle | | **Availability** | Focused action only, and only when explicitly instructed in the prompt | All agents | ## Integration with Cached Workflows This tool is particularly valuable in cached workflows where you want a dynamically evaluated `focused_action` check to have permission to end the run: ```text theme={null} "The workflow normally processes an order through multiple steps. However, if the order was already processed in a previous run or by another system: 1. Navigate to order {order_id} 2. Use focused_action to check the current status - If 'Pending': Continue with normal processing - If 'Complete': Use declare_task_succeeded with 'Order already complete' - If 'Cancelled': Use declare_task_failed with 'Order was cancelled' 3. [Remaining processing steps...]" ``` During cached replay, the focused action will dynamically evaluate step 2 and can terminate the workflow early if the order is already complete, preventing unnecessary cached actions from executing. ## Common Patterns ### Pre-flight Success Check ```text theme={null} "Before starting the main workflow, use focused_action to check if the target state already exists. If the report for {report_date} already exists in the Reports folder, use declare_task_succeeded with 'Report already generated for {report_date}'." ``` ### Polling with Success Detection ```text theme={null} "Use focused_action to check the job status. - If 'Running' or 'Pending': Wait 10 seconds and check again - If 'Completed': Use declare_task_succeeded with the job results - If 'Failed': Use declare_task_failed with the error message" ``` ### Idempotent Operations ```text theme={null} "Use focused_action to verify the user account status: - If account already exists with correct permissions: Use declare_task_succeeded - If account doesn't exist: Proceed to create account - If account exists with wrong permissions: Update permissions" ``` ## Common Mistakes to Avoid **Don't use declare\_task\_succeeded for:** * Normal workflow completion (just let the workflow end naturally) * Main agent success detection (the main agent completes by sending a final message) * Cases where the prompt never explicitly asked for `declare_task_succeeded` * Partial success (if more steps are needed, don't terminate early) ### Incorrect Usage ```text theme={null} // Wrong: Using in main agent context (main agent doesn't need this) "After completing all steps, use declare_task_succeeded" // Wrong: Using outside focused_action "Click submit, then use declare_task_succeeded if it works" // Wrong: Expecting the focused action to infer early-success behavior on its own "Use focused_action to check if the invoice is already paid" // Wrong: Terminating when more work remains "Use focused_action to check the first item. If it's correct, use declare_task_succeeded" ``` ### Correct Usage ```text theme={null} // Correct: Dynamic success detection in focused_action with explicit opt-in "Use focused_action to verify all items have been processed. If the status shows 'All items complete', use declare_task_succeeded" // Correct: Early termination when goal is already met "Use focused_action to check if the file already exists at the destination. If it exists with matching checksum, use declare_task_succeeded with 'File already present at destination'" ``` ## Best Practices Summary 1. **Use only within focused\_action** - This is the key constraint 2. **Explicitly include `declare_task_succeeded` in the instructions** - The focused action should not infer this behavior on its own 3. **Define clear success conditions** - Be specific about what "success" means 4. **Consider adding descriptive messages** - Optionally explain why the workflow succeeded (helps debugging) 5. **Consider idempotency** - Great for workflows that should be safe to re-run 6. **Pair with declare\_task\_failed** - Handle both success and failure conditions 7. **Think about cached replay** - Most valuable when focused actions run during trajectory replay # Execute Batch Tools Source: https://docs.cyberdesk.io/workflow-prompting/execute-batch-tools Bundle multiple computer actions into a single, faster trajectory step `execute_batch_tools` is an explicit `computer` action that lets the agent run several computer actions back-to-back as a **single trajectory step**, with a single screenshot at the end instead of one screenshot after every action. It is the explicit counterpart to the implicit "natively batched tool calls" behavior the v1 agent harnesses already support — both forms produce the **exact same trajectory step** (`execute_batch_tools` with the same `tool_calls` payload), so they are interchangeable from caching, replay, and billing perspectives. Use the `computer` tool with `action='execute_batch_tools'` and pass the actions in the `tool_calls` parameter. ## Why this tool exists When the agent generates **multiple `computer` tool calls in a single response**, Cyberdesk already collapses them into one `execute_batch_tools` trajectory step automatically. That implicit form continues to work as before. This explicit tool gives you (and the agent) a way to express the same intent **inside a single tool call** rather than relying on the model emitting parallel tool calls. It is useful when: * The model you are running does not parallelize tool calls reliably (some models prefer to emit one tool call per turn). * You want the prompt to make batching unambiguous: "use `execute_batch_tools` to do X, Y, Z" reads more clearly than "batch these in one response". * You want a clear, copy-pasteable shape for grouping actions inside `` blocks in your workflow instructions. Both implicit (native) and explicit batching map to the same `env.execute_batch_tools(tool_calls)` call internally. Existing trajectories that used implicit batching continue to replay unchanged. New trajectories that use the explicit tool save and replay in the **identical** format. ## Why batching saves time Every non-batched computer action automatically attaches a fresh full-screen screenshot to the next model turn. That round-trip — screenshot, base64-encode, send to the model, wait for the next decision — is the dominant cost on tasks where the agent already knows the next several actions from the **current** screen. Batching a sequence of actions into one trajectory step: * **Skips intermediate screenshots.** Only the **last** computer action in the batch produces a screenshot (with one exception — see below). * **Skips intermediate model calls.** The model decides on the whole batch once, instead of being re-asked after each action. * **Stays cache-replayable.** A batched trajectory step replays the actions in order on cached runs, with the same speed-up. Loops also expand and re-execute batched bodies correctly. For form filling, multi-key navigation, hover-then-click sequences, and similar deterministic flows, this can be many seconds faster per turn. ## When to use it (and when NOT to) Use `execute_batch_tools` when **every action in the batch is safe to run without re-checking the screen between them**. Good candidates: * Filling a known form: `left_click` field → `type` value → `key` to advance → `type` next value → ... * Hover/move-then-click sequences where the click target is already determined by the current screenshot. * Pressing a known sequence of keys (`tab tab tab enter`). * Closing a known modal then clicking the next button. **Do NOT batch when any later action depends on:** * Whether a click opened or changed something on screen. * Whether validation, navigation, a popup, a dialog, a dropdown, or a layout change occurred. * Discovering a coordinate or target that is not already clearly known on the most recent screenshot. If you are unsure, keep the actions separate — a single missed condition can cascade into multiple wrong actions before the agent gets the next screenshot. ### Things to avoid * Do **not** put an `execute_batch_tools` call inside another `execute_batch_tools.tool_calls` array. Nesting is rejected. * Do **not** put `focused_action`, `start_loop`, or `end_loop_iteration` inside a batch unless the rest of the workflow can tolerate the loop / focused-action firing without intermediate observation. These actions still work inside a batch, but you usually want a fresh screenshot afterwards. * Do **not** rely on `execute_batch_tools` to "force" the model to plan farther ahead than it should. Batching is for actions that you genuinely know are safe right now. ## Parameters | Parameter | Type | Description | | ------------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `action` | `string` | Must be `"execute_batch_tools"`. | | `tool_calls` | `array` | Ordered list of computer actions to run. Each entry is shaped exactly like a normal `computer` tool invocation: an object with its own `action` field and that action's parameters. | Each entry in `tool_calls` may use either the canonical wrapper form (which exactly matches what natively batched tool calls produce on the wire): ```json theme={null} { "name": "computer", "input": { "action": "left_click", "coordinate": [120, 240] } } ``` …or the equivalent shorthand the model can emit directly: ```json theme={null} { "action": "left_click", "coordinate": [120, 240] } ``` Both are accepted and normalize to the same trajectory shape. ## Examples ### Filling a login form ```text theme={null} Use execute_batch_tools with tool_calls=[ {"action": "left_click", "coordinate": [320, 410]}, {"action": "type", "text": "{username}"}, {"action": "key", "text": "tab"}, {"action": "type", "text": "{$password}"}, {"action": "left_click", "coordinate": [320, 540]} ] ``` This runs five actions in a single trajectory step, with a single screenshot at the end (after the final `left_click`). Equivalent to the agent emitting five `computer` tool calls in one response. ### Closing a popup, then opening a menu ```text theme={null} In a batched tool call, close the cookie banner and open the user menu: - left_click on the banner's "Accept" button at the known coordinate - left_click on the avatar in the top-right corner ``` The agent should produce one `execute_batch_tools` call (or one batch of native tool calls) covering both clicks, since the avatar coordinate is already known from the current screenshot. ### Pressing a sequence of keys ```text theme={null} Use execute_batch_tools with tool_calls=[ {"action": "key", "text": "ctrl+a"}, {"action": "key", "text": "delete"}, {"action": "type", "text": "{{new_value}}"} ] ``` ## How it behaves at runtime * **Single trajectory step.** The whole batch is recorded as one `execute_batch_tools` step in the trajectory, with `tool_calls` as its argument. This is the same shape that natively batched parallel tool calls produce, so caching and replay are identical for both. * **Single screenshot.** Only the **last** computer action attaches a screenshot, except when an inner action is `screenshot` with an `extract_prompt` (those always produce their image because the extraction needs it). * **Inter-action pacing.** A small delay is inserted between consecutive actions to keep dense bursts from overwhelming the target machine. This matches the pacing used by implicit batching and cached replay. * **Loops.** When the body of a loop iteration uses `execute_batch_tools`, the system expands the batch when replaying the loop, so each remaining iteration reproduces the same actions in the same order. * **Errors.** If one action inside the batch fails, the failure is included in the aggregated tool result alongside the successes. The model sees the full per-action outcome list and can decide how to recover. ## Backward compatibility * Existing trajectories that used the implicit native-batching path (multiple `computer` tool calls in one response) continue to load and replay unchanged. Their stored step is already named `execute_batch_tools`, so nothing about replay or billing changes. * New runs may freely mix both forms — the agent can natively batch in one turn and explicitly call `execute_batch_tools` in another, and both turns are persisted in the **same** trajectory format. * v0 harnesses do **not** expose this tool. It is available on the v1 main and v1 focused agents only. ## Related * [Async extraction patterns](/concepts/async-extraction-patterns) — how `extract_prompt` interacts with batched and run-scoped extractions. * [Trajectories](/concepts/trajectories) — how trajectory steps are recorded and replayed. * [Usage-based billing](/additional-details/usage-based-billing) — how batched steps are counted (`execute_batch_tools` is billed by the number of inner actions). # Execute Terminal Command Source: https://docs.cyberdesk.io/workflow-prompting/execute-terminal Run PowerShell commands on Windows machines during workflows ## What is Execute Terminal Command? Execute Terminal Command is a specialized tool that allows your agent to run PowerShell commands on the Windows machine during workflow execution. This enables system-level operations, file manipulations, and integrations that go beyond GUI interactions. In your prompts, always refer to this tool as `execute_terminal_command` (lowercase, with underscores). **PowerShell Only**: This tool executes PowerShell commands on Windows. Do NOT use Unix/Bash syntax. Always use proper PowerShell cmdlets and syntax. ## Why This Tool Exists Many enterprise workflows require system-level operations: * File system operations (copy, move, delete, rename) * System administration tasks * API calls and web requests * Data processing and transformation * Integration with command-line tools * Batch operations on multiple files The `execute_terminal_command` tool ensures these operations are performed consistently, even during cached workflow runs. ## Session Behavior **Persistent by default**: `execute_terminal_command` reuses the same PowerShell session by default within a workflow run. * Variables, the current directory, and other shell state can carry over between consecutive terminal commands ## Timeout Behavior By default, execute\_terminal\_command waits up to 30 seconds for a command to complete. For long-running operations, you can specify a custom timeout using the `duration` parameter. **Cyberdriver 1.0.2+**: If a one-shot PowerShell command reaches its timeout, Cyberdriver returns control to Cyberdesk but lets the command keep running in the background. To avoid runaway resource usage, Cyberdriver allows up to 10 timed-out background PowerShell commands at once; additional timed-out commands are terminated until a background slot is released. ### Using Duration Parameter ```text theme={null} "Use execute_terminal_command with duration=180 (3 minutes) to run the data processing script: 'python C:\Scripts\process_large_file.py' and allow it to complete. The workflow will wait up to 3 minutes before continuing." ``` ### Background Execution On Cyberdriver 1.0.2+, when a one-shot command exceeds its timeout and a background slot is available, it continues running in the background while the workflow proceeds. This is useful for: * Fire-and-forget operations * Long-running background tasks * Commands that write output to files **Example:** ```text theme={null} "Use execute_terminal_command with duration=5 to start the backup process: 'Start-Process powershell -ArgumentList "-File C:\Scripts\backup.ps1" -RedirectStandardOutput C:\Logs\backup.log'. The command will start the backup in background and continue immediately." ``` ### Best Practices for Long Commands 1. **Redirect output to files** - Use `> output.txt` or `-RedirectStandardOutput` 2. **Don't rely on command output** - If timeout is reached, stdout/stderr will be empty 3. **Verify completion separately** - Use subsequent commands or focused\_action to check results 4. **Use appropriate timeouts** - Match duration to expected command runtime **Example with file output:** ```text theme={null} "Use execute_terminal_command with duration=10 to start data export: 'python export_script.py > C:\Output\export_log.txt 2>&1' Then use execute_terminal_command to check progress: 'Get-Content C:\Output\export_log.txt -Tail 20 | ConvertTo-Json'" ``` ## PowerShell Syntax Guide ### ❌ Unix/Bash Commands (DON'T USE) ```bash theme={null} curl http://api.example.com cat file.txt ls -la grep "pattern" file.txt ``` ### ✅ PowerShell Equivalents (USE THESE) ```powershell theme={null} Invoke-RestMethod -Uri "http://api.example.com" Get-Content file.txt Get-ChildItem -Force Select-String -Pattern "pattern" -Path file.txt ``` ## How to Prompt for Terminal Commands ### Best Practices 1. **Use PowerShell Syntax**: Always use proper PowerShell cmdlets 2. **Include ConvertTo-Json**: For commands that return objects, pipe to `ConvertTo-Json` to avoid truncation 3. **Handle Dynamic Commands**: Use input variables for commands that change between runs 4. **Specify Full Paths**: Use absolute paths when working with files 5. **Error Handling**: Consider what should happen if a command fails 6. **Use a Workflow Allowlist For Sensitive Flows**: If a workflow only needs a small set of terminal commands, configure the workflow's terminal command allowlist so the agent can only run those approved command templates ### Prompt Template ```text theme={null} "Use execute_terminal_command to run '[PowerShell command]' to [purpose of command]" ``` ## Workflow Terminal Allowlist Cyberdesk workflows can optionally define a **terminal command allowlist** in the workflow editor. When present: * The allowlist is enforced server-side before a command is sent to Cyberdriver. * The agent may only run commands that match one of the approved command templates, or the resolved command produced from that template for the current run. * Allowlist templates may include input values like `{value}`, sensitive values like `{$value}`, runtime values like `{{value}}`, and loop-item runtime values like `{{loop_item}}`. * `working_directory` is not allowed with an active allowlist. Prefer explicit paths inside the approved command itself. * Dynamic input or runtime variables are still fine as long as the resulting command matches an approved template. **Recommended pattern:** list one approved PowerShell command per line in the workflow editor, then prompt the agent to use only those approved commands for terminal work. ### Using Persistent Shell State ```text theme={null} "Use execute_terminal_command to run '$env:REPORT_DIR = \"C:\Reports\"'. Later in the workflow, use execute_terminal_command again to run 'Get-ChildItem $env:REPORT_DIR | ConvertTo-Json'" ``` ## Real-World Examples ### File Operations ```text theme={null} "After downloading the reports, use execute_terminal_command to run 'Move-Item -Path C:\Downloads\*.pdf -Destination D:\Reports\{year}\{month}\ -Force' to organize them by date" ``` ### API Integration ```text theme={null} "Use execute_terminal_command to run this command: 'Invoke-RestMethod -Uri "https://api.company.com/webhook" -Method POST -Body (@{ order_id = "{order_id}" status = "completed" timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss" } | ConvertTo-Json) -ContentType "application/json" | ConvertTo-Json -Depth 10' to notify the system of order completion" ``` ### Data Processing ```text theme={null} "Extract customer emails from the CSV file using execute_terminal_command: 'Import-Csv C:\Data\customers.csv | Select-Object -ExpandProperty Email | Out-File C:\Data\email_list.txt' Then use mark_file_for_export on C:\Data\email_list.txt" ``` ### System Information ```text theme={null} "Before starting the process, use execute_terminal_command to run 'Get-Process | Where-Object {$_.ProcessName -eq "TargetApp"} | ConvertTo-Json' to check if the application is already running" ``` ## Common PowerShell Commands ### File and Directory Operations ```powershell theme={null} # List files Get-ChildItem -Path "C:\Data" -Filter "*.pdf" | ConvertTo-Json # Copy files Copy-Item -Path "C:\Source\*" -Destination "D:\Backup\" -Recurse # Create directory New-Item -ItemType Directory -Path "C:\Reports\{date}" -Force # Delete old files Get-ChildItem -Path "C:\Temp" -Filter "*.tmp" | Where-Object {$_.LastWriteTime -lt (Get-Date).AddDays(-7)} | Remove-Item -Force ``` ### Text Processing ```powershell theme={null} # Search in files Select-String -Path "C:\Logs\*.log" -Pattern "ERROR" | ConvertTo-Json # Replace text in file (Get-Content "C:\Config\settings.ini") -replace 'old_value', '{new_value}' | Set-Content "C:\Config\settings.ini" # Merge multiple files Get-Content C:\Data\file1.txt, C:\Data\file2.txt | Out-File C:\Data\merged.txt ``` ### Web Requests ```powershell theme={null} # GET request Invoke-RestMethod -Uri "https://api.example.com/data/{id}" | ConvertTo-Json -Depth 10 # POST with JSON body $body = @{ name = "{customer_name}" email = "{customer_email}" } | ConvertTo-Json Invoke-RestMethod -Uri "https://api.example.com/customers" -Method POST -Body $body -ContentType "application/json" | ConvertTo-Json ``` ## Dynamic Commands with Variables ### Using Workflow Input Variables ```text theme={null} "Use execute_terminal_command to run the entire command provided in the input variable {powershell_command}. This allows different commands for each run." ``` ### Using Runtime Variables ```text theme={null} "First, use focused_action to find the process ID of the hung application and save it as {{process_id}}. Then use execute_terminal_command to run 'Stop-Process -Id {{process_id}} -Force' to terminate that specific process." ``` ### Building Dynamic Commands ```text theme={null} "Construct and execute a PowerShell command to rename the file: execute_terminal_command: 'Rename-Item -Path "C:\Output\report.pdf" -NewName "{client_name}_{report_date}.pdf"' The {client_name} and {report_date} will be replaced with actual values." ``` ### Conditional Command Execution ```text theme={null} "First check if the backup folder exists using execute_terminal_command: 'Test-Path D:\Backups\{date}' If it returns False, create it using execute_terminal_command: 'New-Item -ItemType Directory -Path D:\Backups\{date} -Force'" ``` ## Sensitive Variables in Terminal Commands Use `{$variable}` for secrets required by terminal actions (e.g., tokens). Secrets are never logged or sent to LLMs and are resolved only at execution time. Avoid echoing them in command output or saving them to files. ### Best Practices * Do not print secrets (avoid `Write-Host` of secret values) * Prefer passing secrets to commands that do not echo them to stdout * Never persist secrets to disk or logs * If a command would reveal the secret in output, capture only status and verify success via UI with `focused_action` ### Example ```text theme={null} "Authenticate using a token {$api_token} via execute_terminal_command without printing it. Verify success on screen with focused_action instead of echoing the token." ``` ## Advanced Patterns ### Batch Processing ```text theme={null} "Process all CSV files in the input directory. Use execute_terminal_command: 'Get-ChildItem -Path C:\Input -Filter *.csv | ForEach-Object { $data = Import-Csv $_.FullName $data | Where-Object {$_.Status -eq "Active"} | Export-Csv -Path ("C:\Output\" + $_.BaseName + "_filtered.csv") -NoTypeInformation }'" ``` ### System Monitoring ```text theme={null} "Check system resources before starting intensive process using execute_terminal_command: '@{ CPU = (Get-Counter "\Processor(_Total)\% Processor Time" -SampleInterval 1 -MaxSamples 1).CounterSamples.CookedValue Memory = (Get-Counter "\Memory\Available MBytes").CounterSamples.CookedValue Disk = (Get-PSDrive C | Select-Object -ExpandProperty Free) / 1GB } | ConvertTo-Json'" ``` ### Log Analysis ```text theme={null} "Extract error counts from today's logs using execute_terminal_command: 'Get-Content C:\Logs\app_{date}.log | Select-String -Pattern "ERROR|CRITICAL" | Group-Object -Property Line | Select-Object Count, Name | ConvertTo-Json'" ``` ## Integration with Other Tools ### With File Export ```text theme={null} "Use execute_terminal_command to create a ZIP archive: 'Compress-Archive -Path C:\Reports\*.pdf -DestinationPath C:\Archive\reports_{date}.zip' Then use mark_file_for_export on C:\Archive\reports_{date}.zip" ``` ### With Focused Action ```text theme={null} "Use focused_action to read the order number from the screen. Then use execute_terminal_command to update the database: 'Invoke-SqlCmd -Query "UPDATE Orders SET Status = ''Processed'' WHERE OrderID = ''{order_number}''" -ServerInstance "localhost\SQLEXPRESS"'" ``` ### Creating Reports ```text theme={null} "Gather system information using multiple execute_terminal_command calls: 1. 'Get-ComputerInfo | Select-Object CsName, OsName, OsVersion | ConvertTo-Json' 2. 'Get-Service | Where-Object {$_.Status -eq "Running"} | Select-Object Name, DisplayName | ConvertTo-Json' 3. 'Get-EventLog -LogName Application -Newest 10 | Select-Object TimeGenerated, Message | ConvertTo-Json' Compile results into a system report." ``` ## Error Handling ### Checking Command Success ```text theme={null} "Try to stop the service using execute_terminal_command: 'Stop-Service -Name "ServiceName" -Force -PassThru | ConvertTo-Json' If the command fails or returns an error, use focused_action to check for error messages and try alternative methods." ``` ### Graceful Failures ```text theme={null} "Attempt to delete temporary files using execute_terminal_command: 'Remove-Item -Path C:\Temp\* -Force -ErrorAction SilentlyContinue' The -ErrorAction SilentlyContinue ensures the workflow continues even if some files cannot be deleted." ``` ## Best Practices Summary 1. **Always use PowerShell syntax, not Unix/Bash commands** 2. **Pipe object output to `ConvertTo-Json` to see full results** 3. **Use absolute paths for file operations** 4. **Consider using `-Force` parameters to avoid prompts** 5. **Handle errors gracefully with `-ErrorAction`** 6. **Test complex commands before including in workflows** 7. **Use input variables for dynamic command components** 8. **Reuse the default persistent session when follow-up commands depend on earlier shell state** # Extract Prompt Source: https://docs.cyberdesk.io/workflow-prompting/extract-prompt Vision-based data extraction with flexible async processing modes ## What is Extract Prompt? Extract Prompt is a powerful vision-based extraction tool you add on top of a screenshot, that uses advanced AI vision models to read, interpret, and extract data from screenshots. Unlike clipboard-based extraction which requires selectable text, Extract Prompt can extract data from any visible content—including images, PDFs, charts, tables, and complex layouts. In your prompts, use the `screenshot` action with the `extract_prompt` parameter to trigger this feature. **Trajectories & Extract Prompt**: Extract prompts work seamlessly with Cyberdesk's [trajectory caching system](/concepts/trajectories). When a trajectory is replayed, extract prompts re-execute to capture fresh data from the current screen, ensuring dynamic data extraction even in cached workflows. ## Why Extract Prompt Exists Cyberdesk offers three complementary methods for data extraction: **Clipboard Extraction** ([`copy_to_clipboard`](/workflow-prompting/copy-to-clipboard)) * Direct copy via Ctrl+C, deterministic and instant * Best for: Selectable text fields (IDs, numbers, dates) * Limitation: Only works with copyable text **Focused Action** ([`focused_action`](/workflow-prompting/focused-action)) * Dynamic decision-making with vision-based extraction * Best for: Runtime decisions, conditional logic, setting runtime variables * Use when: You need to make decisions during workflow execution **Extract Prompt** (This Tool) * Pure vision-based extraction with flexible async processing * Best for: Large-scale data extraction, non-copyable content, parallel processing * Use when: Extracting data for output, not needed for navigation decisions Extract Prompt excels at extracting data from: * Non-selectable or non-copyable text * Images, PDFs, scanned documents * Charts, graphs, and visualizations * Complex tables and multi-column layouts * Dynamic content that changes between runs ## How It Works ### Basic Flow 1. Agent takes a screenshot (optionally with zoom) 2. Screenshot is sent to a strong vision model with your extraction instruction 3. Vision model reads the screen and extracts the requested data 4. Extracted text is returned as the tool result ### With Async Processing When using `process_async`, extractions can run in the background: * **Batch scope**: Extractions run in parallel within current tool call batch * **Run scope**: Extractions run for entire workflow lifetime, only awaited at the end ## The process\_async Parameter The `process_async` parameter controls when and how extractions are processed: ### Synchronous (Default) ```text theme={null} screenshot with extract_prompt="Extract customer name and ID as JSON" ``` * **When**: `process_async` not set, `false`, or `None` * **Behavior**: Blocks until extraction completes * **Use for**: Single extractions where you need immediate results * **Runtime values**: If your prompt explicitly says to save or store something, the extraction agent can use `upsert_runtime_values` before returning its final text result ### Batch-Scoped Async ```text theme={null} screenshot with extract_prompt="Extract order data" and process_async="batch" ``` * **When**: `process_async=true` or `process_async="batch"` * **Behavior**: When the screenshot runs inside an actual batched tool phase, extraction runs in parallel with other batch extractions and all complete before the next agent step * **Use for**: Scrolling through lists, extracting from multiple views in sequence * **Fallback**: If Cyberdesk executes that screenshot as a standalone tool call instead of a batch, it falls back to synchronous extraction * **Special**: The extraction agent can still call `upsert_runtime_values` when your prompt explicitly asks it to save or store values (see below) ### Run-Scoped Async ```text theme={null} screenshot with extract_prompt="Extract all product catalog data" and process_async="run" ``` * **When**: `process_async="run"` * **Behavior**: Extraction runs completely in background for entire run, only awaited at final output * **Use for**: Large extractions not needed for navigation, maximum parallelism * **Requirement**: The workflow must have an `output_schema`; otherwise Cyberdesk returns an error and asks you to use synchronous or batch mode instead * **Special**: The extraction agent can still call `upsert_runtime_values` when your prompt explicitly asks it to save or store values (see below) ## When to Use Each Mode ### Use Synchronous When: * Extracting a single value you need immediately * The extraction result determines next steps * Speed is not critical (\< 5 extractions total) * You want simple, predictable behavior **Example:** ```text theme={null} "Navigate to the order details page. Take a screenshot with extract_prompt 'Extract the order status as a single word: Pending, Processing, or Shipped' and use this to decide the next action." ``` ### Use Batch-Scoped Async When: * Scrolling through lists or paginated content * Extracting from multiple sequential views * Extractions don't depend on each other * Results should be ready for next agent decision **Example:** ```text theme={null} "Scroll through the inventory list and extract data from each page: - Take screenshot with extract_prompt='Extract all visible product SKUs, names, and quantities as JSON array' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract all visible product SKUs, names, and quantities as JSON array' and process_async='batch' - Repeat until bottom of list All batch extractions will complete in parallel before the next agent step." ``` ### Use Run-Scoped Async When: * Extracting large amounts of data * Extraction is only needed for final output, not navigation * You want maximum performance (fully non-blocking) * You may want to store specific values as runtime variables mid-extraction **Example:** ```text theme={null} "Take a screenshot of the dashboard with extract_prompt='Extract all customer metrics, revenue charts, and KPIs as detailed JSON' and process_async='run'. Continue with other tasks—this extraction will complete in the background and be included automatically in the final output." ``` ## Runtime Values with Extract Prompt All `extract_prompt` modes use the extraction agent, and that agent can call `upsert_runtime_values` when your prompt explicitly tells it to save or store runtime values. Async modes add concurrency and background execution, but runtime-value storage itself is not limited to async. ### The upsert\_runtime\_values Tool The extraction agent can: 1. **Store specific values** as runtime variables via `upsert_runtime_values` 2. **Provide final observations** as text 3. **Do BOTH**: Store values AND provide observations For a dedicated reference on `upsert_runtime_values`, including merge/append operators and focused-action usage, see [Upsert Runtime Values](/workflow-prompting/upsert-runtime-values). ### System Prompt Behavior The extraction agent receives guidance like this: ``` You are an extraction assistant. You have two capabilities: 1. Call upsert_runtime_values to store specific extracted values that should be available throughout the workflow as {{key_name}} placeholders. 2. Provide final observations as text describing what you see on screen. You can do BOTH: store specific values AND provide observations, or just do one or the other. Your final text message will be recorded as the extraction result. ``` ### Use Cases for Async Extraction with Runtime Variables **Batch-Scoped: Store and Observe During List Processing:** ```text theme={null} "Scroll through product catalog: - Take screenshot with extract_prompt='Extract product_sku as runtime variable using upsert_runtime_values. Then describe the product details including name, price, and description.' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract product_sku as runtime variable. Then describe product details.' and process_async='batch' - Repeat All extractions run in parallel. The {{product_sku}} values become available before the next agent step, and all product descriptions are included in final output." ``` **Run-Scoped: Store Specific Fields, Observe the Rest:** ```text theme={null} "Take a screenshot with extract_prompt='Extract the invoice number as invoice_number and store it as a runtime variable. Also describe the payment status and due date in your observation.' and process_async='run' Continue with other workflow steps. The {{invoice_number}} will be available immediately once the extraction completes, and the full observation will be in the final output." ``` **Pure Observation (No Runtime Variables):** ```text theme={null} "Take a screenshot with extract_prompt='Describe all visible customer information including name, address, contact details, and account status. Format as JSON.' and process_async='run' This large extraction will run in background and be included in final output." ``` **Multiple Runtime Variables with Observation:** ```text theme={null} "Take a screenshot with extract_prompt='Extract customer_id and order_total as runtime variables. Then provide a detailed summary of the order including line items, shipping address, and special instructions.' and process_async='run' The {{customer_id}} and {{order_total}} will be available for later workflow steps once extraction completes." ``` **Extract Prompt can both store runtime values and return observations in any mode.** Choose sync when you need the answer immediately, batch when you want parallel work inside a batched tool phase, and run-scope when the result is only needed for final structured output. ## Real-World Examples ### Healthcare: Patient Data Extraction **Synchronous (Simple):** ```text theme={null} "Navigate to patient record for MRN {patient_mrn}. Take a screenshot with extract_prompt 'Extract the patient age as a number' to determine if pediatric workflow is needed." ``` **Batch-Scoped (List Processing):** ```text theme={null} "Go to the lab results page and extract all results: - Take screenshot with extract_prompt='Extract all visible lab tests as JSON array with fields: test_name, result, reference_range, status' and process_async='batch' - Scroll down to next page - Take screenshot with extract_prompt='Extract all visible lab tests as JSON array with fields: test_name, result, reference_range, status' and process_async='batch' - Continue until all pages extracted All extractions will complete in parallel at end of batch." ``` **Run-Scoped (Large Extraction):** ```text theme={null} "Take a screenshot of the entire patient chart summary with extract_prompt= 'Extract comprehensive patient data including demographics, vital signs, medications, allergies, and recent visit notes. Format as detailed JSON.' and process_async='run' Continue documenting the visit—the chart data will be extracted in background and included in final documentation output." ``` ### E-Commerce: Product Catalog Extraction **Batch-Scoped with Runtime Variable:** ```text theme={null} "Navigate to product catalog. For each page: - Take screenshot with extract_prompt='Extract all product data as JSON array: {sku, name, price, stock}. If you see a product with sku={target_sku}, store its price as target_product_price runtime variable.' and process_async='batch' - Scroll to next page - Repeat Once {{target_product_price}} is set, use it for price comparison calculations." ``` **Run-Scoped (Full Catalog):** ```text theme={null} "Take screenshot of product grid with extract_prompt='Extract all visible products with full details: SKU, name, description, price, images, ratings, reviews. Format as comprehensive JSON array.' and process_async='run' Continue with inventory reconciliation workflow. The full product data will be extracted in background and included in final export." ``` ### Finance: Transaction Data **Synchronous (Decision-Making):** ```text theme={null} "Open transaction {transaction_id}. Take screenshot with extract_prompt= 'Extract the transaction status: Pending, Completed, or Failed' to determine if manual review is needed." ``` **Run-Scoped with Multiple Runtime Variables:** ```text theme={null} "Open the monthly statement. Take screenshot with extract_prompt='Extract statement_date and total_balance as runtime variables, then provide detailed breakdown of all transactions, fees, and interest charges.' and process_async='run' Use {{statement_date}} and {{total_balance}} in the reconciliation report filename." ``` ### Insurance: Claims Processing **Batch-Scoped (Multiple Claims):** ```text theme={null} "Navigate to pending claims queue. For each claim: - Take screenshot with extract_prompt='Extract claim_id, patient_name, service_date, amount, status as JSON' and process_async='batch' - Click next claim - Repeat for first 20 claims All claim extractions process in parallel before next step." ``` **Run-Scoped (Detailed Claim):** ```text theme={null} "Open claim {claim_id}. Take screenshot with extract_prompt='Extract claim_id and policy_number as runtime variables. Then extract complete claim details: diagnosis codes, procedure codes, provider info, dates of service, all line items with amounts, adjustments, and approval status.' and process_async='run' Continue with approval workflow. The {{claim_id}} and {{policy_number}} are immediately available, and full claim data will be in final output." ``` ## Formatting Guidelines ### Always Request JSON for Structured Data **Best Practice**: Always request strict JSON with explicit keys for structured data extraction. **Good:** ```text theme={null} extract_prompt='Extract customer data as JSON: {customer_id: string, name: string, email: string, phone: string, status: string}' ``` **Also Good:** ```text theme={null} extract_prompt='Extract all visible products as JSON array where each item has: {sku: string, name: string, price: number, stock: number}' ``` **Avoid:** ```text theme={null} extract_prompt='Extract the customer information' // Too vague, may return prose ``` ### Multi-Field Extraction Templates **Simple Object:** ```json theme={null} { "field_name": "type", "another_field": "type" } ``` **Array of Objects:** ```json theme={null} [ { "field1": "value1", "field2": "value2" } ] ``` **Nested Structure:** ```json theme={null} { "customer": { "id": "string", "name": "string" }, "orders": [ { "order_id": "string", "total": "number" } ] } ``` ## Integration with Output Schemas When your workflow has an output schema defined, extracted data automatically flows into the final structured output. ### How It Works 1. **Define Output Schema** in workflow settings: ```json theme={null} { "customer_id": "string", "order_total": "number", "order_items": "array", "shipping_address": "string" } ``` 2. **Extract Data During Workflow:** ```text theme={null} "Take screenshot with extract_prompt='Extract customer_id, order_total, order_items array, and shipping_address as JSON' and process_async='run'" ``` 3. **Automatic Transformation:** * At run completion, extraction results (+ any runtime variables + focused action observations) are automatically transformed to match your output schema * No manual output construction needed! **Optimization Tip**: If all your data is already in runtime values and you want zero LLM transformation, set your output schema to `{"only_runtime_values": true}` to return runtime values directly. The transformation agent also automatically references existing runtime values instead of regenerating them, reducing lossiness. [Learn more about output optimization](/concepts/generating-output-data#output-data-optimization-features). ### Example: Healthcare Record Extraction **Output Schema:** ```json theme={null} { "patient_mrn": "string", "vital_signs": { "blood_pressure": "string", "heart_rate": "number", "temperature": "number" }, "medications": "array", "lab_results": "array" } ``` **Workflow:** ```text theme={null} "Navigate to patient {patient_name} record. Take screenshot with extract_prompt='Extract complete patient data as JSON matching the schema: patient_mrn, vital_signs (blood_pressure, heart_rate, temperature), medications array, and lab_results array. Store patient_mrn as a runtime variable for later use.' and process_async='run' Continue with documentation workflow. The patient_mrn is available as {{patient_mrn}}, and all extracted data will be automatically transformed to the output schema." ``` ## Comparison with Other Extraction Methods | Feature | Extract Prompt | [Focused Action](/workflow-prompting/focused-action) | [Copy to Clipboard](/workflow-prompting/copy-to-clipboard) | | --------------------- | --------------------------------------------------------- | ---------------------------------------------------- | ---------------------------------------------------------- | | **Text Type** | ✅ Any visible text | ✅ Any visible text | ⚠️ Only copyable text | | **Speed** | 🐌 2-5 sec (sync)
⚡ Non-blocking (async) | 🐌 5-10 sec | ⚡ Instant | | **Decision Making** | ❌ Read-only | ✅ Dynamic decisions | ❌ Read-only | | **Runtime Variables** | ✅ Yes, when the prompt explicitly says to save/store them | ✅ Yes | ✅ Yes | | **Async Options** | ✅ Batch & Run scopes | ❌ Always synchronous | ❌ Always synchronous | | **Use Case** | Large-scale extraction, non-copyable content | Dynamic navigation, decisions | Fast field extraction | | **Cost** | 💳 Vision tokens | 💳 Vision tokens | 💰 No AI cost | ### When to Use Each **Use Extract Prompt When:** * Extracting large amounts of data for output * Working with non-copyable content (images, PDFs, charts) * Need parallel/async processing for speed * Extraction doesn't affect navigation decisions * Want to store runtime values while also returning an extraction result **Use [Focused Action](/workflow-prompting/focused-action) When:** * Making dynamic decisions during workflow * Conditional logic based on screen content * Selecting from lists based on runtime criteria * Verification steps that determine next actions **Use [Copy to Clipboard](/workflow-prompting/copy-to-clipboard) When:** * Text is selectable and copyable * Need deterministic, byte-exact extraction * Speed is critical * Working with simple text fields (IDs, numbers, dates) ## Advanced Patterns ### Hybrid Extraction Strategy Combine all three methods for optimal performance: ```text theme={null} "Extract order data using the most efficient method for each field: Fast clipboard extraction (copyable fields): - Triple-click Order ID field, use copy_to_clipboard with key name 'order_id' - Triple-click Customer Name field, use copy_to_clipboard with key name 'customer_name' Vision-based extraction (non-copyable content): - Take screenshot of order items table with extract_prompt='Extract all line items as JSON array: {item_name, sku, quantity, unit_price, total}' and process_async='batch' - Take screenshot of shipping label image with extract_prompt='Extract tracking number and carrier name' and process_async='batch' Dynamic decision (navigation): - Use focused_action to check if order requires special handling based on total amount All values automatically included in final output." ``` ### Cascading Extractions Extract summary data first, then detailed data based on results: ```text theme={null} "Step 1 - Summary (Synchronous): Take screenshot with extract_prompt='Count how many pending orders are visible and return as: {pending_count: number}' Step 2 - Details (Batch-Scoped): If pending_count > 0, for each pending order: - Take screenshot with extract_prompt='Extract order details as JSON' and process_async='batch' - Click next order - Repeat Step 3 - Analytics (Run-Scoped): Take screenshot of analytics dashboard with extract_prompt='Extract complete analytics: sales trends, top products, customer segments' and process_async='run' Continue workflow while analytics extraction runs in background." ``` ### Progressive Detail Extraction Start with high-level extraction, add detail as needed: ```text theme={null} "Navigate to dashboard. Level 1 - Overview (Run-Scoped): Take screenshot with extract_prompt='Extract high-level metrics: total_revenue, total_orders, active_customers, growth_rate. Store total_revenue as runtime variable.' and process_async='run' Level 2 - Category Breakdown (Batch-Scoped): For each product category: - Take screenshot with extract_prompt='Extract category sales data as JSON' and process_async='batch' - Click next category Level 3 - Individual Products (Decision-Based): Use focused_action to identify top 5 products by revenue, then for each: - Take screenshot with extract_prompt='Extract detailed product metrics' and process_async='batch' All extractions complete before final output generation." ``` ### Runtime Variable Integration Use run-scoped extraction to set variables for later workflow steps: ```text theme={null} "Open invoice {invoice_number}. Take screenshot with extract_prompt='Extract invoice_date, due_date, and total_amount as runtime variables. Then extract complete line item details, tax breakdown, and payment terms.' and process_async='run' While that extraction runs in background: - Navigate to customer portal - Login with credentials - Go to payment page - Wait for {{total_amount}} to be available - Enter {{total_amount}} in payment field - Set payment date to {{due_date}} - Submit payment The full invoice details will be in the final output along with payment confirmation." ``` ## Model Override (Optional) You can optionally specify which model to use for extraction by adding the `model` parameter: ```text theme={null} Take screenshot with extract_prompt="Extract all invoice data as JSON" and model="Sonnet 4.5" ``` Unlike `focused_action`, any vision-capable model works for `extract_prompt`—it doesn't need computer use capabilities since it only reads the screen without performing actions. In the prompt editor, type `model=""` or use the `/` slash menu and select "Model Override" to access the model picker. See [Model Configuration](/concepts/model-configuration) for details on per-action model overrides and using different models for cost optimization. ## Error Handling and Best Practices ### Clear Instructions **Be Specific**: The clearer your extraction prompt, the better the results. **Good:** ```text theme={null} extract_prompt='Extract patient vital signs as JSON: {systolic: number, diastolic: number, heart_rate: number, temperature: number, oxygen_saturation: number}' ``` **Avoid:** ```text theme={null} extract_prompt='Get the vital signs' // Too vague ``` ### Field Type Specification Always specify expected data types: ```text theme={null} extract_prompt='Extract order data as JSON: { order_id: string, order_date: string (YYYY-MM-DD), total: number, items: array of {name: string, quantity: number, price: number}, status: string (one of: Pending, Shipped, Delivered) }' ``` ### Handling Missing Data Instruct the model how to handle missing fields: ```text theme={null} extract_prompt='Extract customer data as JSON. If any field is not visible or not available, use null for that field: {name: string|null, email: string|null, phone: string|null, address: string|null}' ``` ### Vision Model Limitations **Vision Model Considerations:** * Complex tables may require multiple extractions * Very small text may not be readable (use zoom if needed) * Similar-looking characters (0/O, 1/l) may be confused * For critical data, consider verification steps ### Zoom for Better Accuracy ```text theme={null} "Take screenshot with zoom_bounding_box=[x1, y1, x2, y2] and extract_prompt= 'Extract the small-print account number visible in the zoomed area' for higher accuracy on tiny text" ``` ## Performance Optimization ### Use Async When Possible **Performance Tip**: If extraction results aren't needed for navigation and your workflow already has an `output_schema`, use `process_async="run"` for maximum parallelism and speed. ### Batch Similar Extractions Instead of: ```text theme={null} Take screenshot, extract field1 Take screenshot, extract field2 Take screenshot, extract field3 ``` Do this: ```text theme={null} Take screenshot with extract_prompt='Extract all three fields as JSON: {field1, field2, field3}' ``` ### Minimize Synchronous Extractions **Before (Slow):** ```text theme={null} Extract A (synchronous, blocks 3s) Extract B (synchronous, blocks 3s) Extract C (synchronous, blocks 3s) Total: 9 seconds ``` **After (Fast):** ```text theme={null} Extract A with process_async='batch' Extract B with process_async='batch' Extract C with process_async='batch' All complete in parallel: ~3 seconds total ``` ## Common Pitfalls **Avoid These Mistakes:** 1. **Using synchronous for large extractions** that aren't needed for navigation 2. **Not specifying JSON format** for structured data 3. **Vague instructions** that lead to unpredictable results 4. **Extracting copyable text** instead of using `copy_to_clipboard` 5. **Not using async** when extracting from multiple pages/views ### ❌ Incorrect Usage ```text theme={null} "Extract the data from the screen" // Too vague "Get all the information" // No format specified "Read the table" // No structure guidance ``` ### ✅ Correct Usage ```text theme={null} "Take screenshot with extract_prompt='Extract invoice data as JSON: {invoice_number: string, date: string, total: number, line_items: array of {description: string, amount: number}}' and process_async='run'" ``` ## Complete Workflow Example ### Scenario: E-Commerce Order Processing **Output Schema:** ```json theme={null} { "order_id": "string", "customer": { "name": "string", "email": "string" }, "items": "array", "total": "number", "shipping": { "address": "string", "method": "string", "tracking": "string" } } ``` **Workflow Instructions:** ```text theme={null} "Log into admin portal with {admin_username} and {$admin_password}. Navigate to order {order_id} details page. Extract order data using optimal methods: 1. Fast clipboard extraction (copyable fields): - Triple-click Order ID, use copy_to_clipboard with key name 'order_id' - Triple-click Customer Email, use copy_to_clipboard with key name 'customer_email' 2. Vision extraction with runtime variables (run-scoped): Take screenshot with extract_prompt='Extract customer_name and order_total as runtime variables. Then extract complete order details: all line items with names/prices/quantities, shipping address, shipping method, and tracking number. Format as detailed JSON matching the output schema.' and process_async='run' 3. Continue workflow while extraction runs: - Generate shipping label - Update inventory system with {{order_total}} - Send notification to {{customer_name}} - Download packing slip and mark_file_for_export The extracted data, clipboard values, and runtime variables will automatically be transformed into the final output_data JSON." ``` **Result:** * Order ID and email extracted instantly via clipboard * Large extraction runs in background while workflow continues * Runtime variables available for inventory and notification steps * All data automatically formatted to output schema * Maximum performance with hybrid approach ## Summary Extract Prompt is your go-to tool for vision-based data extraction with flexible async processing: * **Synchronous**: Simple, immediate extractions for decision-making * **Batch-Scoped Async**: Parallel processing within tool batches (great for lists) * **Run-Scoped Async**: Fully non-blocking extraction for entire run lifetime Combine with [focused\_action](/workflow-prompting/focused-action) for dynamic decisions, [copy\_to\_clipboard](/workflow-prompting/copy-to-clipboard) for fast clipboard extraction, and [looping tools](/workflow-prompting/looping-tools) for efficient batch processing to create highly efficient workflows. **Remember**: Choose the right tool for each task—clipboard for copyable text (fast), extract\_prompt for large-scale/async extraction (flexible), focused\_action for decisions (dynamic), and loops for repetitive patterns (efficient). # Focused Action Source: https://docs.cyberdesk.io/workflow-prompting/focused-action Capture dynamic content and make context-aware decisions in cached workflows ## What is Focused Action? Focused Action is a specialized tool that enables your agent to make observations and decisions that need to be evaluated dynamically during every workflow run, even when the workflow is cached and running deterministically via [trajectories](/concepts/trajectories). In your prompts, always refer to this tool as `focused_action` (lowercase, with underscore). **Trajectories & Focused Actions**: When a workflow trajectory is being replayed, focused actions still execute dynamically to capture fresh data. This allows cached workflows to handle dynamic content while maintaining fast execution. Cyberdesk saves the outer `focused_action` call as the trajectory step, but the focused agent's internal clicks and keystrokes are not recorded as separate steps. During a pure full-cache-hit replay, Cyberdesk does not save a brand-new trajectory unless the workflow falls back to the agent or recovery path. Learn more about [how trajectories work](/concepts/trajectories). ## Why Focused Action Exists In Cyberdesk's caching system, workflows are recorded and can be replayed deterministically. However, certain actions require real-time evaluation: * **Dynamic Selection**: Choosing different items from a list based on runtime criteria * **Data Extraction**: Capturing values that change between runs (prices, dates, patient data) * **Conditional Logic**: Making decisions based on current screen content Without focused\_action, these dynamic elements would be frozen to their first recorded values during cached runs. ## How It Works When you instruct the agent to use `focused_action`, it: 1. Records a screen snapshot for cache detection 2. Launches a focused sub-agent that can take fresh screenshots and perform live computer actions as needed 3. Stores the resulting observation for the run and continues the workflow with up-to-date information This happens **every time** the workflow runs, ensuring fresh data capture even in cached executions. ## When to Use Focused Action ### 1. Dynamic List Selection ```text theme={null} "Navigate to the patient list, then use focused_action to find and click on the patient whose name is {patient.name.full} and date of birth is {patient.demographics.dob}" ``` ### 2. Data Extraction for Output Schema ```text theme={null} "Once you reach the invoice details page, use focused_action to extract the total amount, invoice number, and due date. These values will be different for each run and should be included in the output." ``` ### 3. Conditional Workflow Branching ```text theme={null} "After submitting the form, use focused_action to check if an error message appears. If it says 'Invalid credentials', try the alternate login method. If it says 'Account locked', stop the workflow." ``` ### 4. Verification Steps ```text theme={null} "After clicking submit, use focused_action to verify that the success message appears and contains the confirmation number starting with 'CNF-'" ``` ## How to Prompt for Focused Action ### Best Practices 1. **Be Explicit**: Include the exact token `focused_action` in your prompt, for example `use focused_action to ...` or `focused_action: ...` 2. **Provide Context**: Explain what the agent should look for and why 3. **Specify the Action**: Clearly state what decision or extraction should occur 4. **Think Repeatability**: Frame instructions for what should happen EVERY time 5. **Grant Broader Context Deliberately**: If you want the focused agent to reread the full workflow prompt during recovery, explicitly mention `get_main_instructions` ### Prompt Template ```text theme={null} "[Navigate to specific screen], then use focused_action to [specific observation/action]. [Explain what to look for and what to do with the information]" ``` If a focused action may need broader workflow context to recover from an unexpected state, explicitly mention `get_main_instructions` in that focused prompt. See [Get Main Instructions](/workflow-prompting/get-main-instructions). ## Real-World Examples ### Healthcare: Patient Record Selection ```text theme={null} "Go to the patient search page and search for the last name {patient_last_name}. Once the results load, use focused_action to find the patient whose full name is {patient_full_name} and medical record number ends with {mrn_last_4}, saving their full MRN as {{patient_mrn}}. Click on that patient's record to open their details. Later, when saving reports, use {{patient_mrn}} in the filename." ``` ### E-commerce: Dynamic Price Extraction ```text theme={null} "Search for product SKU {product_sku} and open the product page. Use focused_action to extract the current price, availability status, and any promotional discount percentage. These values should be captured in the output schema as 'current_price', 'in_stock', and 'discount_percent' respectively." ``` ### Finance: Transaction Verification ```text theme={null} "After initiating the transfer of ${amount} to account {recipient_account}, use focused_action to verify the confirmation screen shows the correct amount and recipient details. Extract the transaction ID for the output." ``` ### Document Processing: Dynamic Form Fields ```text theme={null} "Open the claim form for claim number {claim_id}. Use focused_action to identify which optional sections are present (Medical History, Prior Authorization, or Specialist Referral) and fill only those sections that appear." ``` ## Common Pitfalls to Avoid **Don't use focused\_action for static actions** If the action is always the same (e.g., "click the Submit button"), you don't need focused\_action. It's only for dynamic decisions and observations. ### ❌ Incorrect Usage ```text theme={null} "Use focused_action to click the Submit button" // Static action, always the same "Use focused_action to wait 3 seconds" // Static wait, not dynamic ``` ### ✅ Correct Usage ```text theme={null} "Use focused_action to click on the appointment slot that shows {preferred_time}" "Use focused_action to wait until the processing message changes to 'Complete'" ``` ## Working with Runtime Variables **Important**: Runtime variables can be set by: * `focused_action` instructions that save to `{{variable}}`, and the focused agent's explicit `upsert_runtime_values` calls * `copy_to_clipboard` with key names * `extract_prompt` extraction with its `upsert_runtime_values` tool (synchronous, `process_async="batch"`, or `process_async="run"`) No other tool or action can create or update runtime variables. For a dedicated guide to `upsert_runtime_values`, including array/object operators like `$append` and `$merge`, see [Upsert Runtime Values](/workflow-prompting/upsert-runtime-values). **Trajectory Editing**: If you edit a focused action in a trajectory (changing the observation or instruction), you should also update the underlying workflow prompt to match. This ensures consistent behavior when the system falls back to the agent during cache misses. If you add a brand-new focused action step directly in the trajectory editor, that step cannot use cache detection because there is no reference screenshot to compare against. See [Trajectories 101](/concepts/trajectories) for details. Focused actions are a primary way to set runtime variables during navigation, either by prompting them to save to `{{variable}}` or by having the focused agent call `upsert_runtime_values`. `extract_prompt` can also set `{{variables}}` when the prompt explicitly asks it to save or store them. Once set, these variables can be used by any subsequent steps in the workflow. ### Setting Runtime Variables When you need to capture a value for later use, instruct the focused action to save it: ```text theme={null} "Use focused_action to find the generated report file and note its full path, something like 'C:\Reports\2024\Report_12345.pdf'. Save this as {{report_path}}." ``` ### Using Runtime Variables Once set, runtime variables can be used in any subsequent action: ```text theme={null} "First, use focused_action to find the invoice number on the screen and save it as {{invoice_id}}. Then navigate to the search page and type {{invoice_id}} in the search box. Finally, use mark_file_for_export to export the file at C:\Invoices\{{invoice_id}}.pdf" ``` When a focused action receives an instruction with an already-set runtime variable, it will see the current value in parentheses: * `{{invoice_id}}` → `{{invoice_id (currently: 'INV-12345')}}` This makes it clear what the current value is without confusion about assignment. ### Real-World Example: Dynamic File Discovery ```text theme={null} "1. Use focused_action to look at the downloads folder and find the most recent file matching pattern 'Export_*.xlsx'. Save the full file path as {{export_file}}. 2. Use execute_terminal_command to run 'Copy-Item {{export_file}} D:\ProcessedReports\' 3. Use mark_file_for_export to export {{export_file}}" ``` **Important**: If the value is required and cannot be determined, instruct the focused action to call `declare_task_failed` rather than continuing without setting the variable. ### Array and Object Operations When using `upsert_runtime_values`, you can use MongoDB-style operators to manipulate arrays and objects instead of replacing them entirely: | Operator | Description | Example | | ---------- | -------------------------------- | ---------------------------------------- | | `$append` | Append item to array | `{"items": {"$append": "new_item"}}` | | `$prepend` | Prepend item to array | `{"items": {"$prepend": "first"}}` | | `$concat` | Concatenate arrays | `{"items": {"$concat": ["a", "b"]}}` | | `$merge` | Shallow merge objects | `{"config": {"$merge": {"key": "val"}}}` | | `$remove` | Remove first occurrence by value | `{"tags": {"$remove": "old"}}` | | `$pop` | Remove last element | `{"stack": {"$pop": true}}` | #### Example: Accumulating Items Across Loop Iterations ```text theme={null} "For each order in the list: 1. Use focused_action to extract the order details (order_id, customer_name, total) 2. Use upsert_runtime_values to append to the orders list: text='{\"orders\": {\"$append\": {\"order_id\": \"...\", \"customer_name\": \"...\", \"total\": ...}}}' 3. Navigate to the next order After the loop, {{orders}} will contain all extracted order data." ``` #### Example: Building a Summary Object ```text theme={null} "After extracting each section of the report, use upsert_runtime_values to merge the results: text='{\"report_data\": {\"$merge\": {\"section_name\": {...extracted data...}}}}' This accumulates data from multiple sections into a single {{report_data}} object." ``` Operators work with nested paths too: `{"invoice.items": {"$append": {...}}}` will append to the nested `items` array within `invoice`. ## Sensitive Variables and Focused Actions If your prompt includes `{$variable}` (e.g., `{$password}`), the value is handled securely. The agent passes `{$variable}` verbatim to tools; plaintext is not exposed in thoughts or observations. The actual value is only resolved during the underlying computer action (e.g., when typing) and is deleted from the vault after the run. ### Best Practices * Explicitly remind your agent to not repeat sensitive values in it's thought process * For extra peace of mind, we have org-level data retention rules you can set to delete all run data after a set number of days. ### Example ```text theme={null} "Log in with email {email} and password {$password}. After submission, use focused_action to verify that the dashboard is visible without echoing the password in observations." ``` ## Working with Output Schemas When your workflow has an output schema that expects dynamic data, you have multiple extraction methods: * **[extract\_prompt](/workflow-prompting/extract-prompt) (recommended for large-scale, non-blocking extraction):** Use `screenshot` with `extract_prompt` and `process_async` for vision-based extraction. Supports synchronous, batch-scoped, and run-scoped async modes. Best for: non-copyable content, parallel processing, large extractions that don't affect navigation. * **focused\_action (recommended for dynamic decisions or runtime variables during navigation):** Use when you must make a decision each run (branching, verification) and/or you need to set `{{runtime_variables}}` for later steps that affect workflow flow. * **[copy\_to\_clipboard](/workflow-prompting/copy-to-clipboard) (fastest for copyable text):** Deterministic clipboard-based extraction. Best for: selectable text fields like IDs, numbers, dates. ```text theme={null} "Your output schema expects 'patient_vitals' containing blood pressure, heart rate, and temperature. Option A - Extract Prompt (vision-based, can be async): 'Take screenshot with extract_prompt=\"Extract patient vitals as JSON: {systolic, diastolic, heart_rate, temperature}\" and process_async=\"batch\"' Option B - Focused Action (for decisions + extraction): 'Use focused_action to check if vitals are within normal range, extract the values as JSON, and save heart_rate as {{patient_heart_rate}} for later use' Option C - Copy to Clipboard (fastest for copyable fields): 'Triple-click the heart rate field and use copy_to_clipboard with key name \"heart_rate\" to save as {{heart_rate}}'" # Example: Parallel extraction with scrolling (extract_prompt) "Extract all order data by scrolling through the orders table: - Take screenshot with extract_prompt='Extract all visible orders as JSON {order_id, customer, total, status}' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract all visible orders as JSON {order_id, customer, total, status}' and process_async='batch' - Repeat until bottom All batch extractions complete in parallel before next agent step." ``` See [Extract Prompt](/workflow-prompting/extract-prompt) for comprehensive documentation on vision-based extraction with async processing modes. ## Advanced Patterns ### Chained Focused Actions ```text theme={null} "Use focused_action to count how many unread messages are shown. Then, for each unread message, use focused_action to open it and extract the sender and subject line." ``` ### Chaining Runtime Variables ```text theme={null} "1. Use focused_action to find the file with Type DI4 in the current directory. It should match pattern '*{import_file_name}*.D14'. Save the full path as {{di4_file_path}}. 2. Use focused_action to open {{di4_file_path}} and extract the total amount. Save this as {{total_amount}}. 3. Navigate to the summary page and type 'Total: {{total_amount}}' in the field. 4. Use mark_file_for_export to export {{di4_file_path}}" ``` ### Fallback Handling ```text theme={null} "Use focused_action to look for the 'Download Report' button. If it's not visible, use focused_action to check if there's an error message about report generation still in progress, then wait 10 seconds and try again." ``` ### Multi-Criteria Selection ```text theme={null} "In the appointment list, use focused_action to find an available slot that meets ALL of these criteria: {day_of_week}, after {earliest_time}, before {latest_time}, and with {preferred_doctor}." ``` ## Model Override (Optional) You can optionally specify which model to use for a focused action by adding the `model` parameter: ```text theme={null} Use focused_action with model="Sonnet 4.5 (Thinking)" to find and click on the patient whose name matches {patient_name} ``` **Prefer Computer Use Models**: Focused actions work best with models that are known computer use models. The model picker in the prompt editor warns when you select a model that is not marked for computer use. Non-computer-use models may still be selectable, but they often won't work correctly for actions that require clicking, typing, or navigation. In the prompt editor, type `model=""` or use the `/` slash menu and select "Model Override" to access the model picker. See [Model Configuration](/concepts/model-configuration) for details on per-action model overrides and available models. ## Integration with Other Tools Focused actions often work in conjunction with other workflow tools: ```text theme={null} "Use focused_action to verify the report has finished generating (status shows 'Complete'). Once confirmed, use mark_file_for_export to export the report from C:\Users\Reports\{report_date}.pdf and use save_screenshot_as_run_attachment to capture the summary statistics shown on screen." ``` Runtime variables enable powerful integration patterns: ```text theme={null} "Use focused_action to extract the transaction ID from the confirmation screen and save it as {{transaction_id}}. Then use execute_terminal_command to run 'Add-Content -Path C:\Logs\transactions.log -Value "{{transaction_id}} - Processed at $(Get-Date)"' to log the transaction. Finally, use save_screenshot_as_run_attachment to save the confirmation as 'transaction_{{transaction_id}}.png'." ``` Focused actions work especially well inside loops for per-iteration validation: ```text theme={null} "Use start_loop with text={order_ids}. For each order, navigate to {{loop_item}} and use focused_action to verify the order status is 'Shipped' before continuing. Use end_loop_iteration with text='Verified order {{loop_item}}'." ``` Learn more about looping in [Looping Tools](/workflow-prompting/looping-tools). ## Early Workflow Completion Focused actions have exclusive access to `declare_task_succeeded`, which allows them to signal that the entire workflow is complete: ```text theme={null} "Use focused_action to check if the invoice has already been processed. If the status shows 'Complete' or 'Paid', use declare_task_succeeded with message 'Invoice already processed - no action needed'." ``` This is particularly useful during cached trajectory replay, where the focused action can dynamically determine that no further steps are needed. Because this ends the whole run immediately, include the exact phrase `declare_task_succeeded` in the `focused_action` instructions when you want to enable that early-exit behavior. Learn more about early success detection in [Declare Task Succeeded](/workflow-prompting/declare-task-succeeded). # Get Main Instructions Source: https://docs.cyberdesk.io/workflow-prompting/get-main-instructions Let focused action agents reread the main workflow instructions when you explicitly allow it ## What is Get Main Instructions? `get_main_instructions` is a specialized tool that is available only to a **focused action agent**. It returns the main workflow instructions so the focused agent can recover from an unexpected state and get back to the right place before continuing its focused task. In prompts, refer to this tool as `get_main_instructions` exactly. **Focused Action Only**: This tool is only available inside `focused_action`. The main agent and recovery agent cannot call it directly. In most cases, you should only expect the focused agent to use it when your prompt explicitly tells it to use `get_main_instructions`, or otherwise clearly permits access to the main workflow instructions. For broader guidance on when and how to use focused agents, see [Focused Action](/workflow-prompting/focused-action). ## What It Returns The tool returns the main workflow instructions in text form. * Input values like `{patient_id}` stay in bracket notation * Sensitive values like `{$password}` stay in bracket notation * Runtime values may be shown with their current value when available, for example `{{report_id (currently: 'RPT-123')}}` This is useful because the focused agent can reread the larger workflow context without exposing sensitive plaintext values. ## Why This Tool Exists Sometimes a focused action lands on an unexpected screen, popup, or branch and needs more context than its local focused prompt provides. In those cases, `get_main_instructions` can help the focused agent: 1. Reread the full workflow instructions 2. Understand where it was supposed to be in the larger flow 3. Recover back to the correct screen 4. Continue the original focused task ## When to Use It Use `get_main_instructions` mainly when: * The focused agent hits an unexpected state and needs broader workflow context * The recovery path depends on understanding earlier or later workflow steps * You explicitly want the focused agent to have permission to consult the full workflow prompt ## How to Prompt for It The safest pattern is to mention the tool directly inside the `focused_action` instructions. ```text theme={null} "Use focused_action to verify the payment confirmation screen. If the screen is unexpected and you need broader workflow context, call get_main_instructions, recover back to the intended confirmation flow, then continue the focused task." ``` You can also make the permission even more explicit: ```text theme={null} "Use focused_action to find the correct patient record. If you lose context or land on the wrong screen, you may use get_main_instructions once to reread the main workflow instructions and recover back to the right place." ``` ## Common Pitfalls * Do not treat this as a default tool just because it is available * Do not use it as a substitute for writing a clear `focused_action` prompt * Do not assume it is available to the main agent * Do not rely on it to reveal plaintext sensitive values; sensitive placeholders remain bracketed ## Relationship to Focused Action `get_main_instructions` is a supporting tool for `focused_action`, not a replacement for it. * Use [Focused Action](/workflow-prompting/focused-action) for the main delegation pattern * Use `get_main_instructions` only when that focused agent may need broader workflow context to recover from an unexpected situation # Grab Reference Image Source: https://docs.cyberdesk.io/workflow-prompting/grab-reference-image Use prompt images at runtime by filename ## What is Grab Reference Image? `grab_reference_image` lets the agent retrieve a prompt image at runtime by its filename (tail only). This is ideal for using reference visuals during analysis. This returns an image for analysis only. When focusing on small regions, prefer using a fresh full screenshot with `zoom_bounding_box`; include a small margin in the region so labels/edges aren’t cropped. Before generating coordinate actions, the agent should take a fresh full screenshot (without zoom) and base coordinates on that. ## Full‑Page Screenshot Requirements When providing reference images, always capture the entire screen that the agent sees. * Use full‑page screenshots only (no crops or scaled‑down images). * Match the agent’s viewport exactly: same aspect ratio and pixel resolution. * Cropped or smaller images often lead to misaligned coordinates and targeting errors. * If you need to focus on a region, instruct the agent to take a fresh full screenshot and use zoom features for analysis rather than providing a cropped reference image. ## Parameters * **text** (string, required): The tail filename of the prompt image (e.g., `logo.png`). Include the file extension (e.g., `.png`, `.jpg`). ## Behavior * Returns the image as base64 (PNG or JPEG). Other formats are not supported. * If multiple prompt images share the same tail filename, the call returns an error. Use unique names. * If the filename is not found, the call returns an error. ## Best Practices * Keep filenames unique in a prompt. If you import duplicates in the editor, rename them. * Place the image near the instructions it relates to. * Use full‑page screenshots; avoid cropped images. When you need to focus on a region, rely on zoom features after taking a fresh full screenshot. ## Example Prompt Snippets ```text theme={null} "Use grab_reference_image with text='chart_header.png' to examine the header style. After analysis, take a fresh full screenshot and continue." ``` ```text theme={null} "When comparing icons, call grab_reference_image with text='{icon_name}.png'. Use it to confirm the icon design, then proceed with a fresh screenshot before clicking." ``` ## Related * See `Save Screenshot` for capturing run attachments * See `Prompting Overview` for where to add and reference images # Looping Tools Source: https://docs.cyberdesk.io/workflow-prompting/looping-tools Repeat workflow steps deterministically over arrays or counts Looping tools allow you to repeat the same sequence of actions multiple times over a collection of items or a fixed count. The system learns the pattern once during the first iteration, then deterministically replays it for all remaining iterations. ## Overview Instead of manually repeating actions in a focused\_action step, use `start_loop` and `end_loop_iteration` to automate repetitive tasks: * **Learn once, replay many**: The agent maps out one iteration, then the system replays it automatically * **Access loop items**: Use the special `{{loop_item}}` runtime variable to reference the current item in a focused\_action step * **Structured results**: Get detailed summaries of all iterations including successes and failures * **Cache friendly**: Loops work seamlessly with trajectories for lightning-fast execution * **Error handling**: Loops exit early if errors occur, returning partial results ## Actions ### start\_loop Begin a loop iteration over a non-empty array or positive integer count. **Parameters:** * `text`: JSON array like `["item1", "item2", "item3"]` or integer like `5` for counting loops. If the loop source is a variable, do not wrap the variable in quotes: use `text={items}`, `text={$items}`, or `text={{items}}`. **Usage:** ``` Use start_loop with text=["Alice", "Bob", "Carol"] ``` **What Happens:** 1. System creates a loop context with all items 2. Sets current iteration to 0 (first item) 3. Agent completes ONE full iteration manually 4. Agent calls `end_loop_iteration` when done 5. System automatically replays remaining iterations If the resolved `text` contains no items to process, the loop is skipped instead of started. That includes values like `[]`, `null`, an empty string, or `0` / negative integer counts. In that case there is no active loop context, so you should continue with the steps after the loop and **not** call `end_loop_iteration`. **The `{{loop_item}}` Variable:** Access the current loop item using the special `{{loop_item}}` runtime variable: * Simple items: `Type {{loop_item}}` → types "Alice" during iteration 0 * Nested objects: `{{loop_item.name}}` → accesses name field * Array elements: `{{loop_item[0]}}` → accesses first array element * Deep nesting: `{{loop_item.user.emails[0]}}` → accesses nested data ### end\_loop\_iteration Mark the current iteration as complete and trigger replay of remaining iterations. **Parameters:** * `text`: Summary of this iteration (can use runtime variables for dynamic info) **Usage:** ``` Use end_loop_iteration with text="Processed user {{loop_item}}: status={{status}}" ``` **What Happens:** 1. System evaluates runtime variables in the summary text 2. Adds iteration result to the results array 3. If iteration 0: Captures trajectory steps and replays iterations 1 to N-1 4. If recovery iteration: Replays from current + 1 to N-1 5. Returns JSON summary of all iterations when complete **Summary Format:** ```json theme={null} { "total_iterations": 3, "completed_iterations": 3, "successful": 3, "skipped": 0, "failed": 0, "iteration_results": [ {"iteration": 0, "status": "success", "summary": "Processed user Alice: status=complete"}, {"iteration": 1, "status": "success", "summary": "Processed user Bob: status=complete"}, {"iteration": 2, "status": "success", "summary": "Processed user Carol: status=complete"} ], "message": "Loop completed: 3 successful, 0 skipped, 0 failed" } ``` ### skip\_loop\_iteration Skip the **rest of the current loop iteration** and continue to the next loop item. This is intended for **focused agents inside a loop context** (i.e., when a `focused_action` is running during an active `start_loop`). Use it when the current loop item is not actionable (missing data, not found on screen, wrong state, etc.) and you want to move on gracefully. **Parameters:** * `text`: Brief reason why the iteration is being skipped (this will appear in the loop summary) **What Happens:** 1. The system records the skip reason for the current iteration 2. Any remaining steps for that iteration are not executed 3. The loop proceeds to the next item (remaining iterations continue as usual) 4. The final loop summary includes `status: "skipped"` entries and a `skipped` count **Important (how loop replay works):** The system records what you do in the **first** loop iteration (iteration `0`) and replays those recorded steps for iterations `1..N-1`. If iteration `0` immediately calls `skip_loop_iteration`, then the system didn’t learn any “real work” steps to replay — it mostly learned “skip”, so later iterations may also end up doing nothing useful. **Recommendation:** make sure iteration `0` is a normal “happy path” item (one that should be processed). Put a known‑good item first, or reorder/filter your input list so the first item won’t be skipped. If you're running from a **cached trajectory (muscle memory)**, this specific “iteration 0 teaches the replay” concern is not an issue — it’s fine if iteration `0` ends up being a `skip_loop_iteration`, because our system has already learned the full loop actions. **Example – Skip when the current item is missing:** ``` 1. Use start_loop with text=["Alice", "Bob", "Carol"] 2. Use focused_action: "Check whether {{loop_item}} exists in the list. If not found, skip this loop iteration." 3. (Inside focused_action, if not found) Use skip_loop_iteration with text="User not found: {{loop_item}}" 4. Continue with next item automatically 5. Use end_loop_iteration with text="Processed {{loop_item}}" ``` ## Loop Types ### Array Loop Iterate over a collection of items. **Example - Loop over names:** ``` 1. Use start_loop with text=["Alice", "Bob", "Carol"] 2. Click on user {{loop_item}} in the list 3. Extract their email and save as {{user_email}} 4. Use end_loop_iteration with text="Processed {{loop_item}}: {{user_email}}" ``` **Result:** System executes the pattern for Alice, then automatically replays for Bob and Carol. ### Integer Loop Repeat actions a fixed number of times. The `{{loop_item}}` variable will be 0, 1, 2, etc. Positive integers create that many iterations; `0` or a negative count skips the loop entirely. **Example - Download 5 reports:** ``` 1. Navigate to reports page 2. Use start_loop with text=5 3. Click on report number {{loop_item}} 4. Download the report 5. Use end_loop_iteration with text="Downloaded report {{loop_item}}" ``` **Result:** Executes for items 0, 1, 2, 3, 4. ### Variable-Based Loop Loop over data collected during the workflow using runtime or input variables. **Example - Loop over runtime variable:** ``` 1. Use focused_action to extract all order IDs from the screen. Save as {{order_ids}} as a JSON array 2. Use start_loop with text={{order_ids}} 3. Navigate to order {{loop_item}} 4. Extract order details 5. Use end_loop_iteration with text="Processed order {{loop_item}}" ``` **Example - Loop over input variable:** ``` Input variables: {patient_ids: ["P001", "P002", "P003"]} 1. Use start_loop with text={patient_ids} 2. Search for patient {{loop_item}} 3. Extract medical records 4. Use end_loop_iteration with text="Extracted data for {{loop_item}}" ``` ### Passing Complex JSON Arrays via SDK When creating a run programmatically, you can pass arrays of complex objects as input variables. The workflow can then loop over these objects and access nested fields. **Workflow Prompt:** ``` 1. Use start_loop with text={patients} 2. Search for patient {{loop_item.mrn}} 3. Verify name matches {{loop_item.first_name}} {{loop_item.last_name}} 4. Navigate to the {{loop_item.department}} department 5. Extract the latest lab results 6. Use end_loop_iteration with text="Processed {{loop_item.first_name}} {{loop_item.last_name}}" ``` **SDK Code to Create the Run:** ```typescript theme={null} import { createCyberdeskClient } from 'cyberdesk'; const client = createCyberdeskClient('YOUR_API_KEY'); // Define your complex array of objects const patients = [ { mrn: 'MRN-001', first_name: 'Alice', last_name: 'Smith', department: 'Cardiology' }, { mrn: 'MRN-002', first_name: 'Bob', last_name: 'Johnson', department: 'Neurology' }, { mrn: 'MRN-003', first_name: 'Carol', last_name: 'Williams', department: 'Oncology' } ]; // Create the run with the array as an input variable const { data: run } = await client.runs.create({ workflow_id: 'your-workflow-id', machine_id: 'your-machine-id', input_values: { patients: patients // Pass the array directly - SDK handles JSON serialization } }); console.log('Run created:', run.id); ``` ```python theme={null} from cyberdesk import CyberdeskClient, RunCreate import asyncio async def create_run_with_loop_data(): client = CyberdeskClient('YOUR_API_KEY') # Define your complex array of objects patients = [ {'mrn': 'MRN-001', 'first_name': 'Alice', 'last_name': 'Smith', 'department': 'Cardiology'}, {'mrn': 'MRN-002', 'first_name': 'Bob', 'last_name': 'Johnson', 'department': 'Neurology'}, {'mrn': 'MRN-003', 'first_name': 'Carol', 'last_name': 'Williams', 'department': 'Oncology'} ] # Create the run with the array as an input variable run_data = RunCreate( workflow_id='your-workflow-id', machine_id='your-machine-id', input_values={ 'patients': patients # Pass the array directly - SDK handles JSON serialization } ) response = await client.runs.create(run_data) print(f'Run created: {response.data.id}') asyncio.run(create_run_with_loop_data()) ``` **Accessing Nested Fields:** Inside your loop, use dot notation to access object properties: * `{{loop_item.mrn}}` → `"MRN-001"` * `{{loop_item.first_name}}` → `"Alice"` * `{{loop_item.department}}` → `"Cardiology"` For deeply nested data, chain the accessors: `{{loop_item.address.city}}` or `{{loop_item.contacts[0].email}}` ## Advanced Patterns ### Nested Data Access When looping over objects or arrays, access nested fields: **Example:** ```javascript theme={null} // Input: [{"name": "Alice", "dept": "Sales"}, {"name": "Bob", "dept": "Engineering"}] 1. Use start_loop with text={employees} 2. Search for employee {{loop_item.name}} 3. Navigate to {{loop_item.dept}} department 4. Use end_loop_iteration with text="Processed {{loop_item.name}} from {{loop_item.dept}}" ``` ### Combining With Async Extraction Extract data asynchronously while looping for maximum efficiency: **Example:** ``` 1. Use start_loop with text=["Form1", "Form2", "Form3"] 2. Navigate to {{loop_item}} 3. Take screenshot with extract_prompt="Extract all form data as JSON" and process_async="run" 4. Use end_loop_iteration with text="Queued extraction for {{loop_item}}" 5. All extractions complete at end of run before generating final output ``` ### Error Handling in Loops Loops exit early if errors occur, returning partial results: **Example:** ``` 1. Use start_loop with text={document_names} 2. Open document {{loop_item}} 3. If document fails to open, use declare_task_failed with message "Cannot open {{loop_item}}" 4. Process the document 5. Use end_loop_iteration with text="Completed {{loop_item}}" ``` **If iteration 2 fails:** Loop returns summary with iterations 0-1 successful, iteration 2 failed, and `early_exit: true`. ## Cache Detection in Loops By default, recorded loop steps use cache detection. `end_loop_iteration` and `skip_loop_iteration` are control steps, so they do not use cache detection. You can disable cache detection for an individual recorded loop step if a step is too noisy or causes unnecessary cache misses. ### Enabling Loop Cache Detection In the trajectory editor, cache detection is **on by default** for recorded loop steps that have a reference screenshot. Toggle **Cache detection** off to force replay for that step without a screenshot comparison. `end_loop_iteration` and `skip_loop_iteration` are control steps and do not use cache detection. Steps added later in the UI also cannot use cache detection because they have no reference screenshot. ### Recovery Options If cache detection fails mid-loop, the recovery agent is invoked with loop context and can: | Option | Use When | Result | | --------------------- | ------------------------------------ | ----------------------------------- | | `resume_trajectory` | Minor issue (popup, timing) | Retries cache detection | | `end_loop_iteration` | You completed the iteration manually | Loop continues with remaining items | | `skip_loop_iteration` | This item is problematic | Skips to next item | | `declare_task_failed` | Workflow is broken | Task fails cleanly | If the recovery agent resolves the issue (via `resume_trajectory`, `end_loop_iteration`, or `skip_loop_iteration`), the trajectory is still saved. If the agent calls `declare_task_failed`, no trajectory is saved. Loop cache detection is optional per step. Leave it on for steps where visual state matters, and turn it off for steps that are noisy or do not need a screenshot check. ## Important Limitations **Nested Loops Not Supported**: You cannot call `start_loop` while already in a loop. Complete the current loop first before starting a new one. If you attempt nested loops, the system will return an error: `"Cannot start_loop while already in a loop. Call end_loop_iteration first."` **Unclosed Loop Reminder**: If you haven't called `end_loop_iteration` after 20 steps, the system will append a reminder to your tool results. This prevents accidentally leaving loops open. ## Complete Examples ### Example 1: Discover and Process Patients (Runtime Variable Loop) This example demonstrates a common pattern where the loop items are **discovered during the run** rather than passed as input. The agent extracts a list of patients from the screen and then loops over them. **Workflow Prompt:** ``` 1. Navigate to patient management system 2. Login with {username} and {$password} 3. Navigate to "Today's Appointments" page 4. Use focused_action with instruction "Extract all patient IDs visible in the appointments table. Return them as a JSON array of objects with fields: id, name, appointment_time. Save the result as {{todays_patients}}" 5. Use start_loop with text={{todays_patients}} 6. Click on patient {{loop_item.name}} (ID: {{loop_item.id}}) 7. Use focused_action to verify the patient record loaded successfully 8. Navigate to "Demographics" tab 9. Use screenshot with extract_prompt="Extract patient demographics as JSON with fields: name, dob, address, phone" and process_async="batch" 10. Navigate to "Insurance" tab 11. Use copy_to_clipboard with text="insurance_id" to copy the insurance ID 12. Use end_loop_iteration with text="Completed {{loop_item.name}} ({{loop_item.id}}) - Appointment: {{loop_item.appointment_time}}, Insurance: {{insurance_id}}" 13. System will automatically process remaining patients 14. Export final report with all patient data ``` **Input Variables:** ```json theme={null} { "username": "admin" } ``` **What Happens:** 1. Agent logs in and navigates to the appointments page 2. `focused_action` analyzes the screen and extracts patient data, saving it as `{{todays_patients}}`: ```json theme={null} [ {"id": "P12345", "name": "Alice Smith", "appointment_time": "9:00 AM"}, {"id": "P67890", "name": "Bob Johnson", "appointment_time": "10:30 AM"}, {"id": "P11223", "name": "Carol Williams", "appointment_time": "2:00 PM"} ] ``` 3. `start_loop` begins iterating over this runtime-discovered array 4. Agent completes iteration 0 (Alice Smith) manually 5. System automatically replays iterations 1-2 for Bob and Carol 6. Each iteration accesses nested fields: `{{loop_item.id}}`, `{{loop_item.name}}`, `{{loop_item.appointment_time}}` **Key Pattern:** The loop items aren't known ahead of time—they're discovered by `focused_action` during execution and stored as a runtime variable. This is powerful for workflows where you need to "find what's on the screen, then process each item." ### Example 2: Batch Download Forms **Workflow Prompt:** ``` 1. Navigate to forms portal 2. Use start_loop with text=10 3. Click on form row {{loop_item}} 4. Click "Download PDF" button 5. Wait 2 seconds for download 6. Use end_loop_iteration with text="Downloaded form {{loop_item}}" 7. Use execute_terminal_command to run "Get-ChildItem $env:USERPROFILE\Downloads\*.pdf | Select-Object -First 10 | ConvertTo-Json" to list downloaded files ``` **Execution Flow:** * Iteration 0: Agent downloads form 0 * Iterations 1-9: System automatically downloads forms 1-9 * Terminal command lists all downloaded files * Total time: \~10 seconds (cached) vs \~5 minutes (uncached) ### Example 3: Data Entry From Spreadsheet **Workflow Prompt:** ``` 1. Use execute_terminal_command to run "Import-Csv {csv_file_path} | ConvertTo-Json" to load spreadsheet data 2. Use focused_action to parse the JSON output and save the array as {{records}} 3. Navigate to data entry form 4. Use start_loop with text={{records}} 5. Click "New Record" button 6. Type {{loop_item.name}} in the Name field 7. Type {{loop_item.email}} in the Email field 8. Type {{loop_item.phone}} in the Phone field 9. Click "Save" button 10. Use focused_action to verify "Record saved successfully" message appears 11. Use end_loop_iteration with text="Entered record for {{loop_item.name}}" 12. Click "Close" to exit data entry ``` **Result:** Efficiently processes entire spreadsheet with validation at each step. ## Best Practices Add focused\_action steps within your loop to verify each iteration succeeded: ``` 1. Use start_loop with text={items} 2. Process {{loop_item}} 3. Use focused_action to verify processing succeeded 4. Use end_loop_iteration with text="Verified {{loop_item}}" ``` This ensures errors are caught immediately per iteration. Make iteration summaries dynamic by using runtime variables: ``` Use end_loop_iteration with text="Processed {{loop_item}}: found {{count}} results, status={{status}}" ``` This creates rich per-iteration results you can analyze later. Keep loop iterations concise (5-15 steps). For complex processing: ``` 1. Use start_loop with text={orders} 2. Use focused_action to process order {{loop_item}} completely 3. Use end_loop_iteration with text="Completed {{loop_item}}" ``` Let focused\_action handle complex logic per iteration. Set flags during iterations for later decision-making: ``` 1. Use start_loop with text={invoices} 2. Check invoice {{loop_item}} status 3. If unpaid, save unpaid_count via upsert_runtime_values 4. Use end_loop_iteration with text="Checked {{loop_item}}" 5. After loop: If {{unpaid_count}} > 0, send notification email ``` ## Troubleshooting **Cause:** An error occurred in one of the iterations. **Solution:** Check the loop summary's `iteration_results` array to see which iteration failed and why. Add focused\_action validation steps to catch errors early. **Cause:** The `start_loop` input resolved to no items, such as `[]`, `null`, an empty string, or `0`. **Solution:** Confirm the input or runtime variable really contains a JSON array (or a positive integer for counting loops). If it is intentionally empty, skip the loop body and continue with the steps that come after the loop. Do not call `end_loop_iteration` when no loop was started. **Cause:** Runtime variable replacement failed or loop wasn't started. **Solution:** Ensure `start_loop` was called before using `{{loop_item}}`. Check that the loop items array is valid JSON. **Cause:** Attempted to call `start_loop` while already in a loop. **Solution:** Complete the current loop with `end_loop_iteration` before starting a new loop. Nested loops are not supported in the current version. **Cause:** You haven't called `end_loop_iteration` after 20+ steps. **Solution:** Call `end_loop_iteration` when the iteration is complete. The reminder appears every 20 steps to prevent accidental unclosed loops. ## Related Tools Dynamic validation and data extraction within loop iterations Set and use runtime variables in loop summaries Vision-based extraction with async processing in loops How caching accelerates loop execution # Mark File for Export Source: https://docs.cyberdesk.io/workflow-prompting/mark-file-for-export Export files from the remote machine and attach them to workflow runs ## What is Mark File for Export? Mark File for Export is a specialized tool that **immediately exports** files from the remote machine as run attachments. When the agent calls this tool, the file is read and saved right away, providing instant feedback on whether the export succeeded or failed. In your prompts, always refer to this tool as `mark_file_for_export` (lowercase, with underscores). ## Why This Tool Exists Many workflows generate or manipulate files that need to be retrieved: * Downloaded reports * Generated documents * Processed data files * Exported configurations * Created screenshots or images The `mark_file_for_export` tool ensures these files are immediately exported as Run attachments with clear success/failure feedback, allowing agents to retry or investigate if a file doesn't exist. ## How It Works 1. During workflow execution, the agent identifies files to export 2. The agent calls `mark_file_for_export` with the file path 3. **The file is immediately read from the machine and saved as a run attachment** 4. The agent receives instant feedback: * **Success**: Confirmation that the file was exported as a run attachment * **Failure**: Error message explaining why the file couldn't be read (e.g., file not found) 5. If the file doesn't exist, the agent can retry or investigate the issue 6. The new attachment is added to the Run's `output_attachment_ids` array Since files are exported immediately, you can safely delete the file from the machine after calling `mark_file_for_export` if needed (e.g., using `execute_terminal_command`). **Cached execution behavior**: During cached runs, if a file doesn't exist when `mark_file_for_export` is called, the run will fail immediately with a clear error message. This is because retrying isn't useful in cached mode - the trajectory assumed the file would exist. ## When to Use Mark File for Export ### 1. Report Downloads ```text theme={null} "Navigate to the reports section, generate the monthly sales report for {month}, download it as PDF, then use mark_file_for_export to export the file from D:\Downloads\sales_report_{month}.pdf" ``` ### 2. Data Extraction ```text theme={null} "Export the customer database to CSV format. Save it to C:\Exports\customers.csv, then use mark_file_for_export to mark this file for extraction." ``` ### 3. Generated Documents ```text theme={null} "Fill out the insurance claim form with the provided data, save it as D:\Documents\claim_{claim_number}.pdf, then use mark_file_for_export to ensure we retrieve the completed form." ``` ### 4. Batch Processing ```text theme={null} "Process all invoices in the queue. For each processed invoice, save it to D:\Processed\{invoice_id}.pdf and use mark_file_for_export to mark it for extraction." ``` ## How to Prompt for File Export ### Best Practices 1. **Specify Exact Paths**: Be as specific as possible about file locations 2. **Include File Extensions**: Always specify the expected file type 3. **Use Input Variables**: Leverage variables for dynamic file names 4. **Confirm File Creation**: Ensure the file exists before marking for export ### Prompt Template ```text theme={null} "[Describe action that creates/downloads file], save it to [specific file path], then use mark_file_for_export to mark this file for export." ``` ## Real-World Examples ### Healthcare: Patient Records ```text theme={null} "Search for patient {patient_id} and navigate to their medical records. Download the complete patient history as PDF to D:\PatientRecords\{patient_id}_history.pdf. Use mark_file_for_export to mark this file for extraction." ``` ### Finance: Statement Downloads ```text theme={null} "Log into the banking portal and navigate to statements. Download statements for account {account_number} from {start_date} to {end_date}. Save each as D:\Statements\{account_number}_{month}_{year}.pdf and use mark_file_for_export for each downloaded statement." ``` ### HR: Employee Onboarding ```text theme={null} "Generate the new employee welcome packet for {employee_name}. Include the offer letter, benefits summary, and tax forms. Save the complete packet as D:\HR\Onboarding\{employee_id}_welcome_packet.pdf and use mark_file_for_export to mark it for extraction." ``` ### Legal: Document Compilation ```text theme={null} "Compile all documents for case {case_number}. Merge them into a single PDF at D:\LegalDocs\{case_number}_complete.pdf. After successful merge, use mark_file_for_export to mark the compiled document for extraction." ``` ## Working with Dynamic File Paths ### Using Input Variables ```text theme={null} "Download the {report_type} report for {department} and save it as D:\Reports\{department}_{report_type}_{timestamp}.xlsx. Use mark_file_for_export to mark this file for extraction." ``` ### Pattern-Based Exports ```text theme={null} "After processing all orders, use mark_file_for_export to mark each generated invoice following the pattern D:\Invoices\{order_date}\INV_{order_id}.pdf" ``` ### Using Runtime Variables ```text theme={null} "Use focused_action to find the dynamically generated report file (it will have a timestamp in the name). Save the full path as {{generated_report_path}}. Then use mark_file_for_export to export {{generated_report_path}}." ``` ### Sensitive Variables If your prompt includes `{$variable}`, the value is handled securely and is not exposed to the agent or logs. Avoid including secrets in file paths or filenames. Sensitive values are typically used for actions like login; they should not appear in exported file paths or content. ### Conditional Exports ```text theme={null} "If the analysis finds anomalies, export the detailed report to D:\Anomalies\anomaly_report_{run_date}.csv and use mark_file_for_export to mark it for extraction. If no anomalies, skip the export." ``` ## Common Patterns and Tips ### Immediate Feedback Pattern ```text theme={null} "After clicking 'Generate Report', wait for the download to complete, then use mark_file_for_export on D:\Downloads\report.pdf. If the export fails, verify the file was downloaded correctly and retry." ``` ### Multiple File Exports ```text theme={null} "For each patient in the list {patient_ids}, download their: 1. Medical history to D:\Exports\{patient_id}_history.pdf 2. Recent labs to D:\Exports\{patient_id}_labs.pdf 3. Medications list to D:\Exports\{patient_id}_meds.pdf Use mark_file_for_export for each file after it's downloaded." ``` ### Handling Download Locations ```text theme={null} "The system will download the file to the default downloads folder. After download completes, use mark_file_for_export to mark C:\Users\{username}\Downloads\{expected_filename}.pdf for extraction." ``` ## Error Prevention **Common mistakes to avoid:** 1. Exporting files before they're created (you'll get an immediate error if the file doesn't exist) 2. Using incorrect file paths 3. Forgetting file extensions 4. Not accounting for dynamic file names ### ❌ Incorrect Usage ```text theme={null} "Use mark_file_for_export on the report" // No file path specified "Mark D:\Reports\* for export" // Wildcards not supported "Export the file from somewhere in Documents" // Vague path ``` ### ✅ Correct Usage ```text theme={null} "Use mark_file_for_export to export D:\Reports\quarterly_summary.pdf" "Export each file using mark_file_for_export: D:\Reports\Q1.pdf, D:\Reports\Q2.pdf" "Use mark_file_for_export on C:\Users\admin\Documents\output_{date}.xlsx" ``` ### Handling Export Errors Since `mark_file_for_export` provides immediate feedback, your prompts can instruct the agent how to handle failures: ```text theme={null} "After the download completes, use mark_file_for_export on D:\Downloads\report.pdf. If the export fails because the file doesn't exist, check the Downloads folder for the actual filename (it may include a timestamp) and retry with the correct path." ``` ## Integration with Other Tools ### With Focused Action ```text theme={null} "Use focused_action to identify which reports are marked as 'Ready'. For each ready report, download it and use mark_file_for_export to export D:\Downloads\{report_name}.pdf." ``` ### With Terminal Commands ```text theme={null} "Use execute_terminal_command to run 'dir C:\ProcessedFiles\ /B > C:\file_list.txt' to create a list of processed files. Then use mark_file_for_export to export C:\file_list.txt." ``` ### Cleanup After Export Since files are exported immediately, you can clean up the machine right after: ```text theme={null} "Use mark_file_for_export to export D:\Temp\processed_data.csv. Once exported successfully, use execute_terminal_command to delete the file: 'Remove-Item D:\Temp\processed_data.csv'" ``` ### With Screenshots ```text theme={null} "Generate the analytics dashboard for {client_name}. Export the data as D:\Analytics\{client_name}_data.csv and use mark_file_for_export on it. Also use save_screenshot_as_run_attachment to capture the visual dashboard." ``` ## Advanced Usage ### Batch Operations ```text theme={null} "Process all PDFs in the input folder. For each: 1. Apply OCR and save to D:\OCR_Output\{filename}_ocr.pdf 2. Extract text and save to D:\OCR_Output\{filename}.txt 3. Use mark_file_for_export on both the PDF and TXT files 4. Move the original to D:\Processed\ " ``` ### Computed File Names ```text theme={null} "The system generates files with timestamps. After generation, look in D:\Output\ for the most recent file matching pattern 'report_*.pdf' and use mark_file_for_export on that specific file." ``` ### Workflow Output Integration ```text theme={null} "Your workflow should output a JSON with 'exported_files' array containing the paths of all files marked for export. Each time you use mark_file_for_export on a file, add its path to this array in your output." ``` # Model Parameter Source: https://docs.cyberdesk.io/workflow-prompting/model-parameter Override the AI model for specific actions in your workflow prompts The `model` parameter lets you specify which AI model to use for individual `focused_action` and screenshot actions that include `extract_prompt`, directly in your workflow prompt. For comprehensive documentation on model configuration, including workflow-level settings, custom configurations, and supported providers, see [Model Configuration](/concepts/model-configuration). ## Syntax Add `model="Model Name"` to your action: ```text theme={null} Use focused_action with model="Sonnet 4.5 (Thinking)" to verify the order status Take screenshot with extract_prompt="Extract invoice data" and model="Sonnet 4.5" ``` ## Using the Model Picker In the prompt editor: 1. **Slash menu**: Type `/` → select "Model Override" → inserts `model=""` 2. **Tab autocomplete**: Type `model` → press Tab → inserts `model=""` 3. **Direct typing**: Type `model=""` manually With your cursor inside the quotes, a dropdown appears showing all available models. Hover to see model details including computer use support. The dropdown also includes **System Default**, which clears the override and lets Cyberdesk choose the normal default for that action. ## Computer Use Requirement **For `focused_action`**: Prefer models marked as "computer use" models. The editor warns if you pick another configured model, because non-computer-use models may not handle clicking, typing, and navigation reliably. **For `extract_prompt`**: Any vision-capable model works since it only reads the screen. ## What Happens If You Omit `model` * `focused_action` uses the workflow's main agent model, or Cyberdesk's main default if the workflow does not set one * `extract_prompt` uses Cyberdesk's current extraction default unless you override it ## Example Prompts ### Using Different Models for Different Tasks ```text theme={null} Navigate to the patient records system and log in. Use focused_action with model="Sonnet 4.5 (Thinking)" to find the patient whose name is {patient_name} and MRN ends with {mrn_suffix}. This requires careful reasoning to match the correct record. Once on the patient chart, take a screenshot with extract_prompt="Extract all vital signs as JSON: {blood_pressure, heart_rate, temperature, oxygen_saturation}" and model="Sonnet 4.5" Navigate to the lab results tab and extract data from each page: - Take screenshot with extract_prompt="Extract lab results as JSON array" and model="Sonnet 4.5" and process_async="batch" - Scroll down - Repeat until all results captured ``` ### Cost Optimization with Model Selection ```text theme={null} Open the invoice management portal. Search for invoice {invoice_number} and open the details page. Use focused_action with model="Sonnet 4.5 (Thinking)" to verify this is the correct invoice by checking the customer name matches {customer_name} and the date is {invoice_date}. Take a screenshot with extract_prompt="Extract complete invoice data: invoice_number, date, customer info, all line items with descriptions and amounts, subtotal, tax, and total. Format as detailed JSON." and model="Sonnet 4.5" and process_async="run" Continue to the next invoice while extraction runs in background. ``` ### Complex Decision-Making with Thinking Models ```text theme={null} Navigate to the appointment scheduling system. Use focused_action with model="Sonnet 4.5 (Thinking)" to analyze the calendar and find an available slot that meets ALL criteria: - Day is {preferred_day} - Time is between {earliest_time} and {latest_time} - Provider is {preferred_doctor} - Duration allows for {appointment_length} minutes If no slot meets all criteria, use focused_action with model="Sonnet 4.5 (Thinking)" to find the closest alternative and explain the tradeoff. Click to book the selected slot. ``` ### Batch Extraction with Consistent Model ```text theme={null} Open the product catalog and navigate to category {category_name}. For each page of products: - Take screenshot with extract_prompt="Extract all visible products as JSON: {sku, name, price, stock_status, rating}" and model="Sonnet 4.5" and process_async="batch" - Scroll to next page - Repeat until end of catalog All extractions use the same fast model for consistent results and cost. ``` ### Verification Steps with Reasoning ```text theme={null} Submit the insurance claim form. Use focused_action with model="Sonnet 4.5 (Thinking)" to analyze the confirmation screen: - Verify the claim ID is displayed - Check that all submitted fields match our input values - Identify any warnings or additional steps required - Save the claim ID as {{claim_id}} If any discrepancies are found, use declare_task_failed with details. Take screenshot with extract_prompt="Extract confirmation details: claim_id, submission_date, estimated_processing_time, any reference numbers" and model="Sonnet 4.5" ``` ### Mixed Extraction Strategy ```text theme={null} Open the monthly financial report. Fast clipboard extraction for key fields: - Triple-click Report Date field, use copy_to_clipboard with key "report_date" - Triple-click Total Revenue field, use copy_to_clipboard with key "total_revenue" Vision extraction for complex content: - Take screenshot of the revenue breakdown chart with extract_prompt= "Extract revenue by category from this chart as JSON" and model="Sonnet 4.5" - Take screenshot of the expenses table with extract_prompt= "Extract all expense line items as JSON array" and model="Sonnet 4.5" and process_async="batch" Dynamic verification: - Use focused_action with model="Sonnet 4.5 (Thinking)" to verify the totals in the summary section match the detailed breakdowns ``` ## Model Selection Guidelines | Task Type | Recommended Model | Why | | ------------------------------------------- | ----------------------------------------------- | -------------------------------------- | | Complex decisions, multi-criteria matching | Thinking models (e.g., "Sonnet 4.5 (Thinking)") | Better reasoning for nuanced decisions | | Simple verification, straightforward clicks | Standard computer-use models | Faster, lower cost | | Bulk data extraction | Fast vision models (e.g., "Sonnet 4.5") | Speed and cost efficiency | | Critical accuracy needs | Thinking models with verification | Reduces errors on important data | ## Tips If you don't specify a model, the workflow uses its configured default. Only override when you have a specific reason. Use thinking models for complex reasoning, standard models for bulk extraction. The model picker shows which models support computer use. Required for `focused_action`. Pair `model=` with `process_async` for efficient bulk extraction with your preferred model. ## Related Documentation * [Model Configuration](/concepts/model-configuration) — Full details on workflow-level and per-action model settings * [Focused Action](/workflow-prompting/focused-action) — Dynamic decisions and observations * [Extract Prompt](/workflow-prompting/extract-prompt) — Vision-based data extraction # Prompting Overview Source: https://docs.cyberdesk.io/workflow-prompting/prompting-overview Learn how to write effective prompts and leverage specialized tools for workflow outputs ## Writing Effective Prompts When creating workflows, clarity and specificity are paramount. Your prompts should be: * **Unambiguous**: Leave no room for interpretation * **Specific**: Include exact details like file paths, button names, or expected values * **Actionable**: Use clear, directive language ### Example of Good vs Poor Prompting ```text Poor Prompt theme={null} "Download the report" ``` ```text Good Prompt theme={null} "Click the 'Export' button in the top-right corner, select 'PDF' from the dropdown, and save the file to D:\Reports\{report_date}.pdf" ``` ## Understanding Variables Cyberdesk supports three types of variables in your workflows. All variable types support nested access for structured data. ### Input Variables: `{variable}` * **When defined**: Before the workflow starts * **Purpose**: Pass dynamic data like patient names, dates, or IDs * **Example**: `{patient_name}`, `{report_date}`, `{patient.demographics.dob}`, `{items[0].id}` ### Runtime Variables: `{{variable}}` * **When defined**: During workflow execution * **Purpose**: Store values discovered during the workflow for later use * **Example**: `{{extracted_invoice_id}}`, `{{invoice.line_items[0].amount}}` * **Important**: Can be set by `focused_action`, `copy_to_clipboard`, and `extract_prompt` flows that use `upsert_runtime_values` **Key Difference**: Input variables are replaced before the agent sees them, runtime variables are set during execution and used by subsequent steps, and sensitive variables are passed verbatim (as aliases) to tools but their plaintext values are never exposed to the agent; they are resolved only during actual computer actions. ### Sensitive Variables: `{$variable}` * **When defined**: At run creation (via `sensitive_input_values`) * **Purpose**: Handle secrets like passwords, SSNs, tokens without exposing plaintext to the agent or logs * **Example**: `{$password}`, `{$credentials.api_key}` * **Behavior**: * Kept as `{$variable}` in prompts and passed verbatim to tools * Stored securely in a third‑party vault (Basis Theory) only for the duration of the run * Never logged, never shown on the dashboard, and never sent to LLMs * Resolved to plaintext only at the last mile (e.g., when typing), then deleted from the vault after run completion ### Nested Access for Structured Data Instead of creating many flat variables, pass structured JSON and access nested properties: ```text theme={null} {patient.demographics.first_name} {patient.insurance.policy_number} {patient.emergency_contacts[0].phone} ``` See [Structured Inputs](/concepts/structured-inputs) for full documentation on nested access syntax and error handling. ## Utilizing Prompt Images Images can anchor your instructions with visual context. Add screenshots alongside your text in the dashboard editor to help the agent recognize ambiguous icons, menu locations, or tricky UI flows (especially in legacy apps). * Place images near the paragraph they illustrate for best results * Use images as guidance – the agent will still take fresh screenshots during execution * Keep images focused on the relevant UI area; include multiple images if needed Runtime access to prompt images is available via the `grab_reference_image` tool by filename (tail only). After analyzing any reference or zoomed image, the agent should take a fresh full screenshot before issuing coordinate actions. When specifying zoom regions, consider leaving a small margin so labels/edges aren’t cropped. Use your OS’s native Markup/annotation tools to add red circles and arrows around the exact UI elements you want the AI to focus on. Simple visual callouts dramatically improve recognition. ## Understanding Trajectories and Cached Tools Cyberdesk's intelligent caching system uses **trajectories** to record successful workflow executions and replay them deterministically. When a workflow has an approved trajectory, it can complete in seconds instead of minutes. To maximize this feature while still doing dynamic actions, you must properly prompt the agent to use **special dynamic tools**. **More on Trajectories**: Learn more in [Trajectories 101](/concepts/trajectories). ### Dynamic Tools in Cached Workflows Even during cached trajectory replay, certain tools execute dynamically to capture fresh data: * **[focused\_action](/workflow-prompting/focused-action)**: Focused computer use agent to complete an always dynamic part of the workflow * **[Screenshot with extract\_prompt](/workflow-prompting/extract-prompt)**: Extracts fresh data from current screen, and can set runtime values * **[copy\_to\_clipboard](/workflow-prompting/copy-to-clipboard)**: Captures current clipboard contents, setting a runtime value By explicitly prompting the agent to use these tools, you ensure dynamic data capture even when the rest of the workflow is cached. ## The Essential Workflow Tools ### 1. Focused Action Capture dynamic content or make context-aware decisions during cached runs. * Selecting different items from a list each run * Extracting dynamic data that changes between runs * Making decisions based on screen content **In prompts, refer to it as** `"focused_action"` ### 2. Mark File for Export Export files from the remote machine after workflow completion. * Downloading generated reports * Extracting processed files * Saving workflow outputs **In prompts, refer to it as** `"mark_file_for_export"` ### 3. Save Screenshot as Run Attachment Capture important moments as screenshots accessible after the run. * Documenting visual confirmation * Capturing patient X-rays or medical images * Recording transaction confirmations **In prompts, refer to it as** `"save_screenshot_as_run_attachment"` ### 4. Grab Reference Image Retrieve a user-provided prompt image during execution by filename (tail only). * Provide visual context to disambiguate UI elements * Help the agent recognize specific screens, dialogs, or popups **In prompts, refer to it as** `"grab_reference_image"`. After analyzing, take a fresh full screenshot before coordinate actions. See the dedicated page for details. ### 5. Copy to Clipboard Execute Ctrl+C and automatically save clipboard contents as a runtime variable. * Extracting text from legacy systems * Capturing copyable values from read-only or context-menu-copy fields * Getting data from read-only fields * Quick text extraction with manual selection **In prompts, refer to it as** `"copy_to_clipboard"` ### 6. Screenshot with Extract Prompt Vision-based data extraction with flexible async processing modes (synchronous, batch-scoped, or run-scoped) and ability to set a runtime value * Non-copyable content (images, PDFs, charts, tables) * Parallel extraction across multiple views * When navigation doesn't depend on extraction results * Quickly setting one off runtime values **In prompts, refer to it as** `"extract_prompt"` parameter on `screenshot` action. For parallel processing, use `process_async="batch"` or `process_async="run"`. See the dedicated page for full details on async modes. #### Quick Start: Simple Extraction ```text theme={null} "Take a screenshot with extract_prompt='Extract customer name, email, and phone number as JSON: {name: string, email: string, phone: string}'" ``` #### Parallel Extraction Across Multiple Screens Make extraction 3-5x faster by using `process_async="batch"` when scrolling or navigating: ```text theme={null} "In one batched tool call, extract all product data by scrolling through the catalog: - Take screenshot with extract_prompt='Extract all visible products as JSON array' and process_async='batch' - Scroll down - Take screenshot with extract_prompt='Extract all visible products as JSON array' and process_async='batch' - Repeat until bottom All extractions process in parallel before the next agent step." ``` #### Run-Scoped Extraction (Fully Non-Blocking) For large extractions that don't affect navigation, use `process_async="run"`: ```text theme={null} "Take screenshot with extract_prompt='Extract complete dashboard analytics as detailed JSON, saving as a runtime value.' and process_async='run'. Continue with other workflow tasks. The extraction runs in background and will be included in final output automatically." ``` **Learn More**: See the [Extract Prompt](/workflow-prompting/extract-prompt) page for detailed documentation on async modes, runtime variable integration, and advanced patterns. ### 7. Execute Terminal Command Run PowerShell commands on the Windows machine. * System administration tasks * File operations * API calls or data processing **In prompts, refer to it as** `"execute_terminal_command"` If a workflow only needs a known set of terminal actions, configure the workflow's terminal command allowlist in the editor. When present, Cyberdesk enforces that allowlist server-side before the command is sent to the machine. Allowlist entries can use input, sensitive, runtime, and loop-item placeholders. Cyberdesk accepts either the template form or the resolved command for the current run. ### 8. Declare Task Failed Terminate workflow execution when failure conditions are met. * Error handling * Validation failures * Preventing unnecessary continuation **In prompts, refer to it as** `"declare_task_failed"` ## Additional Advanced Tools These are also important parts of the workflow-prompting surface, even if you do not use them in every workflow. ### Declare Task Succeeded Allow a focused action to end the workflow early when a specific dynamic success condition is met. **In prompts, refer to it as** `"declare_task_succeeded"`. Use it only inside `focused_action` when early success is explicitly intended. ### Get Main Instructions Let a focused action reread the main workflow instructions when it needs broader context to recover from an unexpected state. **In prompts, refer to it as** `"get_main_instructions"`. It is focused-action-only and should be granted deliberately. ### Looping Tools Use `start_loop`, `end_loop_iteration`, and `skip_loop_iteration` for repeated patterns across arrays or counts. Use loops when the workflow genuinely repeats the same block of work across many items. Access the current item via `{{loop_item}}`. ### Upsert Runtime Values Use `upsert_runtime_values` when the agent should write or merge runtime values explicitly instead of only returning observations. This is especially useful in focused actions and extraction flows when later steps depend on discovered values like `{{report_path}}`, `{{missing_file}}`, or accumulated arrays/objects. ### Model Parameter Use `model="..."` when a specific `focused_action` or `extract_prompt` step benefits from a different model than the workflow default. Use model overrides sparingly. They are best for unusually hard reasoning or extraction tasks. ### Complex Drag Actions Use `left_mouse_down` and `left_mouse_up` when `left_click_drag` is not enough, such as drag flows that require hover pauses or multi-step hold behavior. If a simple drag is enough, prefer `left_click_drag`. Reach for the lower-level drag actions only when the UI genuinely needs them. ## Quick Examples ### Using Focused Action ```text theme={null} "When you reach the patient list, use focused_action to select the patient whose name matches {patient_name} and extract their ID number" ``` ### Exporting Files ```text theme={null} "After generating the report, use mark_file_for_export to export the file located at C:\Reports\output.pdf" ``` ### Capturing Screenshots ```text theme={null} "Once the payment confirmation appears, use save_screenshot_as_run_attachment to capture the confirmation screen as 'payment_confirmation.png'" ``` ### Copying to Clipboard ```text theme={null} "Triple-click on the account number field to select it, then use copy_to_clipboard with key name 'account_number' to save it as {{account_number}} for use in the next form" ``` ### Running Terminal Commands ```text theme={null} "Use execute_terminal_command to run 'Get-ChildItem C:\Exports\*.pdf | ConvertTo-Json' to list all PDF files in the exports folder" ``` **For long-running commands:** ```text theme={null} "Use execute_terminal_command with duration=120 (2 minutes) to run the report generation script and wait for completion before proceeding" ``` ### Handling Failures ```text theme={null} "If the login screen shows 'Account Locked' or 'Invalid License', use declare_task_failed to terminate with message 'Unable to access system'" ``` ## Next Steps For detailed information about each tool, including advanced usage patterns and real-world examples, explore some of the individual tool documentation pages: Dynamic observations and decisions File extraction from workflows Visual documentation Vision-based extraction with optional async extraction Copy text and save as runtime variable Use prompt images at runtime PowerShell automation Error handling End early on dynamic success Let focused actions recover with broader context Repeat stable blocks across items Bundle multiple actions into one fast trajectory step Write and merge runtime values explicitly Override models for specific actions Advanced drag and hold behavior # Save Screenshot as Run Attachment Source: https://docs.cyberdesk.io/workflow-prompting/save-screenshot Capture and save important visual moments from your workflows ## What is Save Screenshot? Save Screenshot as Run Attachment is a specialized tool that captures the current screen state and saves it as an attachment to your workflow run. Unlike regular screenshots used for navigation, these are specifically preserved as outputs that can be accessed after the workflow completes. In your prompts, always refer to this tool as `save_screenshot_as_run_attachment` (lowercase, with underscores). ## Why This Tool Exists Visual documentation is crucial for many workflows: * **Proof of Completion**: Transaction confirmations, submission receipts * **Medical Imaging**: X-rays, scans, patient charts * **Quality Assurance**: Visual verification of results * **Compliance**: Audit trails and visual evidence * **Data Capture**: Charts, graphs, or visual data that's hard to extract This tool ensures important visual moments are captured even during cached runs, providing consistent documentation across all executions. ## How It Works 1. When the agent reaches a specified point in the workflow 2. It calls `save_screenshot_as_run_attachment` with a descriptive filename 3. The current screen is captured and saved 4. The screenshot is attached to the Run record 5. The attachment ID is added to the Run's `output_attachment_ids` array Screenshots are captured at full screen resolution and saved in PNG format for maximum quality and compatibility. You can optionally pass a `zoom_bounding_box` parameter to capture a zoomed-in crop for analysis. Format: `[x1, y1, x2, y2]` in pixels relative to the full screenshot. After analyzing a zoomed image, always take a fresh full screenshot before generating coordinate actions. `zoom_bounding_box` is clamped to the screenshot bounds if values go out of range, and coordinates are always interpreted in the full-screenshot pixel space (not the zoomed image). Consider including a small padding margin in your bbox to avoid cropping labels/edges. ## When to Use Screenshot Attachments ### 1. Confirmation Screens ```text theme={null} "After submitting the payment, wait for the confirmation page to load, then use save_screenshot_as_run_attachment to capture the confirmation as 'payment_confirmation_{invoice_number}.png'" ``` ### 2. Medical Images ```text theme={null} "Navigate to the patient's imaging results. Open the chest X-ray dated {scan_date} and use save_screenshot_as_run_attachment to save it as '{patient_id}_chest_xray_{scan_date}.png'" ``` ### 3. Dashboard Captures ```text theme={null} "After the analytics dashboard loads with data for {month}, use save_screenshot_as_run_attachment to capture the full dashboard as '{client_name}_analytics_{month}.png'" ``` ### 4. Error Documentation ```text theme={null} "If any error message appears during processing, use save_screenshot_as_run_attachment to capture it as 'error_{timestamp}.png' before attempting to resolve" ``` ## How to Prompt for Screenshots ### Best Practices 1. **Descriptive Filenames**: Use clear, contextual names for easy identification 2. **Include Context**: Specify when and what to capture 3. **Use Variables**: Incorporate input variables for dynamic naming 4. **Be Specific**: Clearly indicate the exact moment to capture ### Prompt Template ```text theme={null} "[Navigate to specific screen/state], [wait for specific element if needed], then use save_screenshot_as_run_attachment to capture [what's being captured] as '[descriptive_filename].png'" ``` ### Zoomed Captures ```text theme={null} "Zoom into the billing subtotal region by passing zoom_bounding_box [x1, y1, x2, y2], then use save_screenshot_as_run_attachment as '{customer_id}_subtotal.png'. After analysis, take a fresh full screenshot before any clicking." ``` ## Real-World Examples ### Healthcare: Patient Documentation ```text theme={null} "Open patient {patient_name}'s record and navigate to the vitals chart. Ensure the last 30 days of data is visible, then use save_screenshot_as_run_attachment to save the vitals trend as '{patient_id}_vitals_trend_{date}.png'. This screenshot is critical for the physician's review." ``` ### E-commerce: Order Processing ```text theme={null} "Complete the order for customer {customer_id}. On the order confirmation page, use save_screenshot_as_run_attachment to capture the full page including order number, items, and total as 'order_{order_number}_confirmation.png'" ``` ### Finance: Transaction Records ```text theme={null} "After executing the wire transfer of ${amount} to {recipient_bank}, wait for the success screen, then use save_screenshot_as_run_attachment to document the transaction as 'wire_transfer_{transaction_id}.png'. Ensure the reference number is clearly visible." ``` ### Insurance: Claim Evidence ```text theme={null} "For each damage photo in the claim, maximize the image view and use save_screenshot_as_run_attachment to save it as 'claim_{claim_number}_damage_{photo_number}.png'. Capture all 5 required angles." ``` ## Naming Strategies ### Dynamic Naming with Variables ```text theme={null} "Use save_screenshot_as_run_attachment to save the test results as '{patient_lastname}_{test_type}_{test_date}.png'" ``` ### Using Runtime Variables in Names ```text theme={null} "First, use focused_action to read the order number from the confirmation screen and save it as {{order_number}}. Then use save_screenshot_as_run_attachment to capture the receipt as 'order_{{order_number}}_receipt.png'" ``` ### Sensitive Variables If your prompt includes `{$variable}`, the value is handled securely and never logged or shown. Avoid including secrets in screenshot filenames or on-screen content. The agent will not repeat secrets in observations, but screenshots may still capture whatever is visible on screen; prefer verification that doesn't display the secret. ### Sequential Captures ```text theme={null} "For each step of the procedure: 1. Screenshot the pre-procedure state as 'procedure_{case_id}_step1_before.png' 2. Perform the action 3. Screenshot the result as 'procedure_{case_id}_step1_after.png' Use save_screenshot_as_run_attachment for each capture." ``` ### Timestamped Names ```text theme={null} "Capture the system status dashboard every 5 minutes during the test. Use save_screenshot_as_run_attachment with names like 'system_status_{test_id}_{capture_time}.png'" ``` ## Common Patterns ### Wait Before Capture ```text theme={null} "After clicking 'Generate Report', wait for the loading spinner to disappear and the report to fully render (usually 3-5 seconds), then use save_screenshot_as_run_attachment to save it as 'report_{report_id}.png'" ``` ### Conditional Screenshots ```text theme={null} "Check if the prescription includes controlled substances. If yes, use save_screenshot_as_run_attachment to capture the prescription as 'controlled_rx_{patient_id}_{date}.png' for compliance records." ``` ### Multiple Page Documentation ```text theme={null} "The report spans 3 pages. For each page: 1. Use save_screenshot_as_run_attachment to save as 'report_{report_id}_page{n}.png' 2. Click 'Next Page' Repeat for all pages." ``` ## Quality Considerations **Tips for high-quality screenshots:** 1. Ensure the relevant content is fully visible (not cut off) 2. Wait for all elements to load before capturing 3. Maximize windows when capturing detailed information 4. Consider zoom levels for readability 5. For small UI elements, consider `zoom_bounding_box` to improve clarity ### Ensuring Content Visibility ```text theme={null} "Scroll to ensure the entire form is visible on screen (all fields from 'Patient Name' to 'Signature' should be in view), then use save_screenshot_as_run_attachment to capture as 'completed_form_{form_id}.png'" ``` ### Handling Pop-ups and Overlays ```text theme={null} "If a confirmation dialog appears, use save_screenshot_as_run_attachment to capture it as 'confirmation_dialog.png' before clicking OK. Make sure the dialog is centered and fully visible." ``` ## Integration with Other Tools ### With Focused Action ```text theme={null} "Use focused_action to verify the chart has loaded completely with all data points visible. Once confirmed, use save_screenshot_as_run_attachment to capture the analytics chart as '{metric_name}_trend_{date_range}.png'" ``` ### With File Export ```text theme={null} "Generate the PDF report and save to D:\Reports\{report_id}.pdf. Use mark_file_for_export on the PDF file. Also use save_screenshot_as_run_attachment to capture the report summary screen as '{report_id}_summary.png' for quick reference." ``` ### Documentation Workflow ```text theme={null} "For the audit trail: 1. Screenshot the initial state: 'audit_{audit_id}_initial.png' 2. Perform the changes as specified 3. Screenshot each change confirmation 4. Screenshot the final state: 'audit_{audit_id}_final.png' Use save_screenshot_as_run_attachment for all captures." ``` ## Advanced Usage ### Creating Visual Logs ```text theme={null} "During the 10-minute system test, use save_screenshot_as_run_attachment every minute to create a visual log. Name files as 'test_{test_id}_minute_{n}.png' where n goes from 1 to 10." ``` ### Comparison Documentation ```text theme={null} "Open the previous version in the left panel and current version in the right panel. Use save_screenshot_as_run_attachment to capture the side-by-side comparison as 'document_comparison_{doc_id}_v{old}_vs_v{new}.png'" ``` ### Error State Collection ```text theme={null} "If the workflow encounters any unexpected screens or error messages: 1. Use save_screenshot_as_run_attachment to capture as 'unexpected_state_{timestamp}.png' 2. Include these screenshots in the error report 3. Continue with error recovery steps This helps with debugging failed runs." ``` ## Best Practices Summary 1. **Always specify clear, descriptive filenames** 2. **Ensure content is fully loaded before capturing** 3. **Use input variables for dynamic naming** 4. **Capture at moments that provide maximum value** 5. **Consider the screenshot's purpose when framing the capture** 6. **Document both success and failure states when relevant** # Upsert Runtime Values Source: https://docs.cyberdesk.io/workflow-prompting/upsert-runtime-values Store, merge, and accumulate runtime variables during workflow execution ## What is Upsert Runtime Values? `upsert_runtime_values` is a specialized tool for writing runtime variables explicitly during workflow execution. It is useful when the agent should do more than simply observe or extract text. Instead, it should store structured values that later workflow steps can reuse. In prompts, always refer to this tool as `upsert_runtime_values` exactly. ## When to Use It Use `upsert_runtime_values` when you want to: * store a discovered value like `{{report_path}}` * set a failure or status flag like `{{missing_file}}` * accumulate arrays or objects across repeated steps * merge structured data discovered in different phases of a workflow Examples: * saving a discovered export path for later `mark_file_for_export` * recording that a required file or patient was not found * building up an `{{orders}}` array across loop iterations * merging multiple extracted sections into one `{{report_data}}` object ## Where It Is Available `upsert_runtime_values` is not a general-purpose main-agent action. You will most commonly use it in: * [`focused_action`](/workflow-prompting/focused-action) * [`extract_prompt`](/workflow-prompting/extract-prompt) flows, especially async extraction agents That means your top-level workflow prompt usually instructs a focused or extraction agent to call `upsert_runtime_values` as part of its work. ## Basic Usage The tool accepts JSON-like structured values keyed by runtime-variable name. Conceptually: ```json theme={null} { "report_path": "C:\\Reports\\output.pdf", "missing_file": true } ``` After that, later steps can use: * `{{report_path}}` * `{{missing_file}}` ## Common Prompt Patterns ### Save a Discovered Value ```text theme={null} Use focused_action to find the exported report file in Downloads. Save the full path as {{report_path}} using upsert_runtime_values. ``` ### Record a Failure Flag ```text theme={null} Use focused_action to check whether invoice {invoice_id} exists in the list. If it is missing, use upsert_runtime_values to set {{missing_invoice}} to true, then declare_task_failed. ``` ### Store Data During Extract Prompt ```text theme={null} Take screenshot with extract_prompt="Extract customer_id and order_total and store them as runtime values using upsert_runtime_values. Then provide a short summary of the visible order." and process_async="batch" ``` ## Array and Object Operations `upsert_runtime_values` also supports Mongo-style operators for accumulating structured data instead of replacing it wholesale. | Operator | Description | Example | | ---------- | -------------------------------- | ---------------------------------------- | | `$append` | Append item to array | `{"items": {"$append": "new_item"}}` | | `$prepend` | Prepend item to array | `{"items": {"$prepend": "first"}}` | | `$concat` | Concatenate arrays | `{"items": {"$concat": ["a", "b"]}}` | | `$merge` | Shallow merge objects | `{"config": {"$merge": {"key": "val"}}}` | | `$remove` | Remove first occurrence by value | `{"tags": {"$remove": "old"}}` | | `$pop` | Remove last element | `{"stack": {"$pop": true}}` | ### Example: Accumulating Across a Loop ```text theme={null} For each order: 1. Use focused_action to extract order_id, customer_name, and total. 2. Use upsert_runtime_values with text='{"orders": {"$append": {"order_id": "...", "customer_name": "...", "total": ...}}}' 3. Use end_loop_iteration when finished. ``` ### Example: Merging Structured Sections ```text theme={null} After extracting each section, use upsert_runtime_values with text='{"report_data": {"$merge": {"section_name": {...extracted data...}}}}' ``` ## Best Practices 1. Use clear runtime variable names like `{{report_path}}` or `{{missing_file}}`. 2. Prefer booleans for simple failure flags. 3. Use `$append` and `$merge` instead of replacing large objects repeatedly. 4. Only store values you actually need later in the workflow or output. 5. If a missing value should stop the run, pair `upsert_runtime_values` with `declare_task_failed`. ## Related Docs * [Focused Action](/workflow-prompting/focused-action) * [Extract Prompt](/workflow-prompting/extract-prompt) * [Looping Tools](/workflow-prompting/looping-tools) * [Generating Output Data](/concepts/generating-output-data)