Focused Action

"Navigate to the patient list, then use focused_action to find and click on 
the patient whose name is {patient.name.full} and date of birth is {patient.demographics.dob}"

"Once you reach the invoice details page, use focused_action to extract 
the total amount, invoice number, and due date. These values will be 
different for each run and should be included in the output."

"After submitting the form, use focused_action to check if an error message 
appears. If it says 'Invalid credentials', try the alternate login method. 
If it says 'Account locked', stop the workflow."

"After clicking submit, use focused_action to verify that the success 
message appears and contains the confirmation number starting with 'CNF-'"

"[Navigate to specific screen], then use focused_action to [specific observation/action]. 
[Explain what to look for and what to do with the information]"

"Go to the patient search page and search for the last name {patient_last_name}. 
Once the results load, use focused_action to find the patient whose full name 
is {patient_full_name} and medical record number ends with {mrn_last_4}, saving their full MRN as {{patient_mrn}}. 
Click on that patient's record to open their details. Later, when saving reports, use {{patient_mrn}} 
in the filename."

"Search for product SKU {product_sku} and open the product page. Use focused_action 
to extract the current price, availability status, and any promotional discount 
percentage. These values should be captured in the output schema as 'current_price', 
'in_stock', and 'discount_percent' respectively."

"After initiating the transfer of ${amount} to account {recipient_account}, 
use focused_action to verify the confirmation screen shows the correct amount 
and recipient details. Extract the transaction ID for the output."

"Open the claim form for claim number {claim_id}. Use focused_action to 
identify which optional sections are present (Medical History, Prior Authorization, 
or Specialist Referral) and fill only those sections that appear."

"Use focused_action to click the Submit button"  // Static action, always the same
"Use focused_action to wait 3 seconds"           // Static wait, not dynamic

"Use focused_action to click on the appointment slot that shows {preferred_time}"
"Use focused_action to wait until the processing message changes to 'Complete'"

"Use focused_action to find the generated report file and note its full path, 
something like 'C:\Reports\2024\Report_12345.pdf'. Save this as {{report_path}}."

"First, use focused_action to find the invoice number on the screen and save it as {{invoice_id}}.
Then navigate to the search page and type {{invoice_id}} in the search box.
Finally, use mark_file_for_export to export the file at C:\Invoices\{{invoice_id}}.pdf"

"1. Use focused_action to look at the downloads folder and find the most recent 
   file matching pattern 'Export_*.xlsx'. Save the full file path as {{export_file}}.
2. Use execute_terminal_command to run 'Copy-Item {{export_file}} D:\ProcessedReports\'
3. Use mark_file_for_export to export {{export_file}}"

"For each order in the list:
1. Use focused_action to extract the order details (order_id, customer_name, total)
2. Use upsert_runtime_values to append to the orders list:
   text='{\"orders\": {\"$append\": {\"order_id\": \"...\", \"customer_name\": \"...\", \"total\": ...}}}'
3. Navigate to the next order

After the loop, {{orders}} will contain all extracted order data."

"After extracting each section of the report, use upsert_runtime_values to merge 
the results: text='{\"report_data\": {\"$merge\": {\"section_name\": {...extracted data...}}}}'

This accumulates data from multiple sections into a single {{report_data}} object."

"Log in with email {email} and password {$password}. After submission, use focused_action to verify that the dashboard is visible without echoing the password in observations."

"Your output schema expects 'patient_vitals' containing blood pressure, heart rate, and temperature.

Option A - Extract Prompt (vision-based, can be async):
'Take screenshot with extract_prompt=\"Extract patient vitals as JSON: 
{systolic, diastolic, heart_rate, temperature}\" and process_async=\"batch\"'

Option B - Focused Action (for decisions + extraction):
'Use focused_action to check if vitals are within normal range, extract the 
values as JSON, and save heart_rate as {{patient_heart_rate}} for later use'

Option C - Copy to Clipboard (fastest for copyable fields):
'Triple-click the heart rate field and use copy_to_clipboard with key name 
\"heart_rate\" to save as {{heart_rate}}'"

# Example: Parallel extraction with scrolling (extract_prompt)
"Extract all order data by scrolling through the orders table:
- Take screenshot with extract_prompt='Extract all visible orders as JSON 
  {order_id, customer, total, status}' and process_async='batch'
- Scroll down
- Take screenshot with extract_prompt='Extract all visible orders as JSON 
  {order_id, customer, total, status}' and process_async='batch'
- Repeat until bottom

All batch extractions complete in parallel before next agent step."

"Use focused_action to count how many unread messages are shown. Then, 
for each unread message, use focused_action to open it and extract the 
sender and subject line."

"1. Use focused_action to find the file with Type DI4 in the current directory. 
   It should match pattern '*{import_file_name}*.D14'. Save the full path as {{di4_file_path}}.
2. Use focused_action to open {{di4_file_path}} and extract the total amount. 
   Save this as {{total_amount}}.
3. Navigate to the summary page and type 'Total: {{total_amount}}' in the field.
4. Use mark_file_for_export to export {{di4_file_path}}"

"Use focused_action to look for the 'Download Report' button. If it's not 
visible, use focused_action to check if there's an error message about 
report generation still in progress, then wait 10 seconds and try again."

"In the appointment list, use focused_action to find an available slot that 
meets ALL of these criteria: {day_of_week}, after {earliest_time}, 
before {latest_time}, and with {preferred_doctor}."

Use focused_action with model="Sonnet 4.5 (Thinking)" to find and click on 
the patient whose name matches {patient_name}

"Use focused_action to verify the report has finished generating (status shows 'Complete'). 
Once confirmed, use mark_file_for_export to export the report from 
C:\Users\Reports\{report_date}.pdf and use save_screenshot_as_run_attachment 
to capture the summary statistics shown on screen."

"Use focused_action to extract the transaction ID from the confirmation screen and 
save it as {{transaction_id}}. Then use execute_terminal_command to run 
'Add-Content -Path C:\Logs\transactions.log -Value "{{transaction_id}} - Processed at $(Get-Date)"' 
to log the transaction. Finally, use save_screenshot_as_run_attachment to save 
the confirmation as 'transaction_{{transaction_id}}.png'."

"Use start_loop with text={order_ids}. For each order, navigate to {{loop_item}} 
and use focused_action to verify the order status is 'Shipped' before continuing. 
Use end_loop_iteration with text='Verified order {{loop_item}}'."

"Use focused_action to check if the invoice has already been processed. 
If the status shows 'Complete' or 'Paid', use declare_task_succeeded 
with message 'Invoice already processed - no action needed'."