What is Focused Action?
Focused Action is a specialized tool that enables your agent to make observations and decisions that need to be evaluated dynamically during every workflow run, even when the workflow is cached and running deterministically.
In your prompts, always refer to this tool as focused_action (lowercase, with underscore).
Why Focused Action Exists
In Cyberdesk’s caching system, workflows are recorded and can be replayed deterministically. However, certain actions require real-time evaluation:
- Dynamic Selection: Choosing different items from a list based on runtime criteria
- Data Extraction: Capturing values that change between runs (prices, dates, patient data)
- Conditional Logic: Making decisions based on current screen content
Without focused_action, these dynamic elements would be frozen to their first recorded values during cached runs.
How It Works
When you instruct the agent to use focused_action, it:
- Takes a screenshot at that exact moment
- Analyzes the current screen content
- Makes a decision or extracts data based on your instructions
- Continues the workflow with the dynamically obtained information
This happens every time the workflow runs, ensuring fresh data capture even in cached executions.
When to Use Focused Action
1. Dynamic List Selection
"Navigate to the patient list, then use focused_action to find and click on
the patient whose name is {patient_name} and date of birth is {patient_dob}"
"Once you reach the invoice details page, use focused_action to extract
the total amount, invoice number, and due date. These values will be
different for each run and should be included in the output."
3. Conditional Workflow Branching
"After submitting the form, use focused_action to check if an error message
appears. If it says 'Invalid credentials', try the alternate login method.
If it says 'Account locked', stop the workflow."
4. Verification Steps
"After clicking submit, use focused_action to verify that the success
message appears and contains the confirmation number starting with 'CNF-'"
How to Prompt for Focused Action
Best Practices
- Be Explicit: Always use the exact phrase “use focused_action” in your prompt
- Provide Context: Explain what the agent should look for and why
- Specify the Action: Clearly state what decision or extraction should occur
- Think Repeatability: Frame instructions for what should happen EVERY time
Prompt Template
"[Navigate to specific screen], then use focused_action to [specific observation/action].
[Explain what to look for and what to do with the information]"
Real-World Examples
Healthcare: Patient Record Selection
"Go to the patient search page and search for the last name {patient_last_name}.
Once the results load, use focused_action to find the patient whose full name
is {patient_full_name} and medical record number ends with {mrn_last_4}, saving their full MRN as {{patient_mrn}}.
Click on that patient's record to open their details. Later, when saving reports, use {{patient_mrn}}
in the filename."
"Search for product SKU {product_sku} and open the product page. Use focused_action
to extract the current price, availability status, and any promotional discount
percentage. These values should be captured in the output schema as 'current_price',
'in_stock', and 'discount_percent' respectively."
Finance: Transaction Verification
"After initiating the transfer of ${amount} to account {recipient_account},
use focused_action to verify the confirmation screen shows the correct amount
and recipient details. Extract the transaction ID for the output."
"Open the claim form for claim number {claim_id}. Use focused_action to
identify which optional sections are present (Medical History, Prior Authorization,
or Specialist Referral) and fill only those sections that appear."
Common Pitfalls to Avoid
Don’t use focused_action for static actionsIf the action is always the same (e.g., “click the Submit button”), you don’t need focused_action. It’s only for dynamic decisions and observations.
❌ Incorrect Usage
"Use focused_action to click the Submit button" // Static action, always the same
"Use focused_action to wait 3 seconds" // Static wait, not dynamic
✅ Correct Usage
"Use focused_action to click on the appointment slot that shows {preferred_time}"
"Use focused_action to wait until the processing message changes to 'Complete'"
Working with Runtime Variables
Important: Focused actions are the ONLY way to set runtime variables. No other tool or action can create or update runtime variables.
{{variable}}. Once set, these variables can be used by any subsequent steps in the workflow.
Setting Runtime Variables
When you need to capture a value for later use, instruct the focused action to save it:
"Use focused_action to find the generated report file and note its full path,
something like 'C:\Reports\2024\Report_12345.pdf'. Save this as {{report_path}}."
Using Runtime Variables
Once set, runtime variables can be used in any subsequent action:
"First, use focused_action to find the invoice number on the screen and save it as {{invoice_id}}.
Then navigate to the search page and type {{invoice_id}} in the search box.
Finally, use mark_file_for_export to export the file at C:\Invoices\{{invoice_id}}.pdf"
{{invoice_id}} → {{invoice_id (currently: 'INV-12345')}}
This makes it clear what the current value is without confusion about assignment.
Real-World Example: Dynamic File Discovery
"1. Use focused_action to look at the downloads folder and find the most recent
file matching pattern 'Export_*.xlsx'. Save the full file path as {{export_file}}.
2. Use execute_terminal_command to run 'Copy-Item {{export_file}} D:\ProcessedReports\'
3. Use mark_file_for_export to export {{export_file}}"
Important: If the focused action cannot determine the value to save, it will call declare_task_failed rather than continuing without setting the variable. This ensures your workflow doesn’t fail silently with missing data.
Sensitive Variables and Focused Actions
If your prompt includes {$variable} (e.g., {$password}), the value is handled securely. The agent passes {$variable} verbatim to tools; plaintext is not exposed in thoughts or observations. The actual value is only resolved during the underlying computer action (e.g., when typing) and is deleted from the vault after the run.
Best Practices
- Explicitly remind your agent to not repeat sensitive values in it’s thought process
- For extra peace of mind, we have org-level data retention rules you can set to delete all run data after a set number of days.
Example
"Log in with email {email} and password {$password}. After submission, use focused_action to verify that the dashboard is visible without echoing the password in observations."
Working with Output Schemas
When your workflow has an output schema that expects dynamic data, you now have two options:
- Just ask to extract (recommended for simple reads): Simply tell the agent what to extract (and optionally the format, e.g., JSON). Under the hood, the agent will take a fresh screenshot and use a strong vision model to read the screen. The extracted text is automatically piped into the output transformation step. For multiple screens, you can request “parallel extraction” to process all extractions simultaneously.
- focused_action (recommended for dynamic decisions or runtime variables): Use when you must make a decision each run (branching, verification) and/or you need to set
{{runtime_variables}} for later steps.
"Your output schema expects 'patient_vitals' containing blood pressure, heart rate, and temperature.
Option A: Tell the agent to extract: {systolic:int, diastolic:int, heart_rate:int, temperature:float} as JSON.
Option B: Use focused_action to read and (optionally) save these values as runtime variables for later reuse."
# Example: Parallel extraction with scrolling
"Extract all order data from the orders table. Start at the top, extract all visible
orders as JSON {order_id, customer, total, status}, then scroll down and repeat.
Continue until you reach the bottom of the list. Do all extractions in parallel
for faster processing."
Advanced Patterns
Chained Focused Actions
"Use focused_action to count how many unread messages are shown. Then,
for each unread message, use focused_action to open it and extract the
sender and subject line."
Chaining Runtime Variables
"1. Use focused_action to find the file with Type DI4 in the current directory.
It should match pattern '*{import_file_name}*.D14'. Save the full path as {{di4_file_path}}.
2. Use focused_action to open {{di4_file_path}} and extract the total amount.
Save this as {{total_amount}}.
3. Navigate to the summary page and type 'Total: {{total_amount}}' in the field.
4. Use mark_file_for_export to export {{di4_file_path}}"
Fallback Handling
"Use focused_action to look for the 'Download Report' button. If it's not
visible, use focused_action to check if there's an error message about
report generation still in progress, then wait 10 seconds and try again."
Multi-Criteria Selection
"In the appointment list, use focused_action to find an available slot that
meets ALL of these criteria: {day_of_week}, after {earliest_time},
before {latest_time}, and with {preferred_doctor}."
"Use focused_action to verify the report has finished generating (status shows 'Complete').
Once confirmed, use mark_file_for_export to export the report from
C:\Users\Reports\{report_date}.pdf and use save_screenshot_as_run_attachment
to capture the summary statistics shown on screen."
"Use focused_action to extract the transaction ID from the confirmation screen and
save it as {{transaction_id}}. Then use execute_terminal_command to run
'Add-Content -Path C:\Logs\transactions.log -Value "{{transaction_id}} - Processed at $(Get-Date)"'
to log the transaction. Finally, use save_screenshot_as_run_attachment to save
the confirmation as 'transaction_{{transaction_id}}.png'."
