Async Extraction Patterns

"Navigate to order details. Take a screenshot with extract_prompt='Extract 
the order status as one word: Pending, Processing, or Shipped' to determine 
if the order needs manual intervention."

Agent Step 1 → Extract (3s) → Agent Step 2 → Extract (3s) → Agent Step 3
Total: 6 seconds of extraction time

"Scroll through the product catalog and extract all data:
- Take screenshot with extract_prompt='Extract visible products as JSON' 
  and process_async='batch'
- Scroll down
- Take screenshot with extract_prompt='Extract visible products as JSON' 
  and process_async='batch'
- Scroll down
- Take screenshot with extract_prompt='Extract visible products as JSON' 
  and process_async='batch'

All extractions complete in parallel before next agent turn."

Agent Step: [Screenshot + Extract] → [Scroll] → [Screenshot + Extract] → [Scroll] → [Screenshot + Extract]
            └─────────── All 3 extractions run in parallel (3s total) ──────────┘
                                                                                  ↓
                                                                          Agent Step 2
Total: ~3 seconds for all extractions combined

"Navigate to analytics dashboard. Take screenshot with extract_prompt='Extract 
complete analytics data: all metrics, charts, KPIs, trends as detailed JSON' 
and process_async='run'

Continue with report generation workflow. The analytics extraction will complete 
in the background and be included automatically in final output."

Agent Step 1 → [Start Extract (non-blocking)] → Agent Step 2 → Agent Step 3 → Agent Step 4
                        ↓ (running in background)
                        ↓
                        ↓
               [Extraction completes while workflow continues]
                        ↓
                        └─────────────────────────────────────→ Final Output Generation
                                                                 (waits for completion)
Total: 0 seconds of blocking time, extraction happens in parallel with workflow

Page 1: Extract (3s) → Navigate
Page 2: Extract (3s) → Navigate
...
Page 10: Extract (3s)
Total: 30 seconds of extraction time

Batch 1: [Page 1 Extract, Navigate, Page 2 Extract, Navigate, Page 3 Extract]
         All 3 extractions in parallel: 3s
Batch 2: [Page 4 Extract, Navigate, Page 5 Extract, Navigate, Page 6 Extract]
         All 3 extractions in parallel: 3s
Batch 3: [Page 7 Extract, Navigate, Page 8 Extract, Navigate, Page 9 Extract]
         All 3 extractions in parallel: 3s
Batch 4: [Page 10 Extract]
         1 extraction: 3s
Total: 12 seconds of extraction time (2.5x faster)

Start: Launch extraction for all pages → Continue workflow
       All 10 extractions run in background while workflow completes
Total: 0 seconds of blocking time (10x faster)

"Navigate to customer portal.

Step 1 - Quick Decision (Synchronous):
Take screenshot with extract_prompt='Extract customer account status: 
Active, Suspended, or Closed' to determine workflow path.

Step 2 - List Processing (Batch-Scoped):
If status is Active, extract recent transactions:
- Take screenshot with extract_prompt='Extract visible transactions as JSON' 
  and process_async='batch'
- Scroll down
- Take screenshot with extract_prompt='Extract visible transactions as JSON' 
  and process_async='batch'
- Repeat for all pages

Step 3 - Detailed Analytics (Run-Scoped):
Take screenshot with extract_prompt='Extract complete account analytics: 
spending patterns, category breakdown, payment history' and process_async='run'

Continue with report generation. The analytics extraction completes in background."

You are an extraction assistant. You have two capabilities:

1. Call upsert_runtime_values to store specific extracted values that should 
   be available throughout the workflow as {{key_name}} placeholders.

2. Provide final observations as text describing what you see on screen.

You can do BOTH: store specific values AND provide observations, or just do 
one or the other. Your final text message will be recorded as the extraction result.

"Scroll through order list:
- Take screenshot with extract_prompt='Extract order_id as runtime variable 
  using upsert_runtime_values, then describe order status and customer name' 
  and process_async='batch'
- Use {{order_id}} to determine if this order needs special handling
- If yes, click on order
- Repeat for next order"

"Open invoice {invoice_number}.

Take screenshot with extract_prompt='Extract the invoice_date and total_amount 
and store them as runtime variables using upsert_runtime_values. Then provide 
a detailed description of all line items, tax breakdown, and payment terms.' 
and process_async='run'

Continue with payment workflow. The {{invoice_date}} and {{total_amount}} 
variables will be available once extraction completes in background, and the 
full invoice details will be in the final output."

"Take screenshot with extract_prompt='Describe all customer information 
visible on screen including contact details, order history, and preferences. 
Format as detailed JSON.' and process_async='run'

This large extraction runs in background and will be in final output."

"Take screenshot with extract_prompt='Extract customer_id, order_total, and 
expected_delivery_date as runtime variables using upsert_runtime_values. Then 
provide a summary of order contents, shipping address, and special instructions.' 
and process_async='run'

Use {{customer_id}} and {{order_total}} in subsequent workflow steps once available."

Do you need the extraction result to make the next decision?
├─ YES → Use Synchronous
│         Example: "Extract status to determine next action"
│
└─ NO → Is this a list/multiple views?
    ├─ YES → Use Batch-Scoped Async
    │         Example: "Scroll and extract from each page"
    │
    └─ NO → Is this only for final output?
        ├─ YES → Use Run-Scoped Async
        │         Example: "Extract analytics for report"
        │
        └─ DEPENDS → Does it need to set runtime variables?
            ├─ YES → Use Run-Scoped Async (with upsert_runtime_values)
            │         Example: "Extract invoice_number for later use"
            │
            └─ NO → Use Synchronous if < 3 extractions
                    Use Batch-Scoped if >= 3 extractions

"Navigate to patient {patient_mrn}.

Quick Check (Synchronous):
Take screenshot with extract_prompt='Extract patient age as number' to 
determine if pediatric workflow is needed.

Medical History (Batch-Scoped):
Navigate through each section of medical history:
- Take screenshot with extract_prompt='Extract diagnosis history' and process_async='batch'
- Go to medications tab
- Take screenshot with extract_prompt='Extract current medications' and process_async='batch'
- Go to allergies tab
- Take screenshot with extract_prompt='Extract allergies' and process_async='batch'

Comprehensive Record (Run-Scoped):
Take screenshot of complete patient chart with extract_prompt='Extract full 
patient record including all demographics, vitals, lab results, imaging reports, 
and visit notes. Store patient_mrn as runtime variable.' and process_async='run'

Continue with documentation workflow using {{patient_mrn}} for file naming."

"Log into inventory system.

Category Check (Synchronous):
Take screenshot with extract_prompt='Count visible categories' to determine 
navigation depth.

Per-Category Extraction (Batch-Scoped):
For each category:
- Navigate to category
- Take screenshot with extract_prompt='Extract all products as JSON' and process_async='batch'
- Repeat

Full Catalog Analytics (Run-Scoped):
Take screenshot with extract_prompt='Extract complete inventory analytics: 
stock levels, trends, low-stock alerts, reorder recommendations. Store 
total_products and low_stock_count as runtime variables.' and process_async='run'

Generate inventory report using {{total_products}} and {{low_stock_count}} in report header."

"Open account {account_number}.

Balance Check (Synchronous):
Take screenshot with extract_prompt='Extract current balance as number' to 
verify sufficient funds.

Recent Transactions (Batch-Scoped):
Scroll through transaction history:
- Take screenshot with extract_prompt='Extract visible transactions' and process_async='batch'
- Scroll down
- Take screenshot with extract_prompt='Extract visible transactions' and process_async='batch'
- Repeat

Account Analysis (Run-Scoped):
Take screenshot with extract_prompt='Extract complete account analysis: 
spending categories, monthly trends, unusual patterns. Store account_type 
and risk_level as runtime variables.' and process_async='run'

Continue with report generation using {{account_type}} and {{risk_level}} for classification."

# Initial (works, but slow)
"Extract field A (sync) → Extract field B (sync) → Extract field C (sync)"

# Optimized (3x faster)
"Extract all three fields with process_async='batch' in one batch"

"For each item in list:
- Extract data with process_async='batch'
- Navigate to next
All extractions complete in parallel"

"Take screenshot with extract_prompt='Extract complete report data' 
and process_async='run'

Continue workflow - extraction completes in background"

"Extract order data:
- Triple-click Order ID, use copy_to_clipboard with key 'order_id' (instant)
- Take screenshot with extract_prompt='Extract order items' and process_async='batch' (parallel)
- Take screenshot with extract_prompt='Extract shipping analytics' and process_async='run' (background)

Optimal performance using all available methods"

"Take screenshot with extract_prompt='Extract customer_id and order_total 
as runtime variables. Then extract complete order details, history, and 
preferences as detailed JSON.' and process_async='run'

The {{customer_id}} and {{order_total}} become available once extraction completes, 
while full details are in final output."

"Extract from page 1
Navigate
Extract from page 2
Navigate
Extract from page 3"

"Extract from page 1 with process_async='batch'
Navigate
Extract from page 2 with process_async='batch'
Navigate
Extract from page 3 with process_async='batch'"

"Extract complete data with process_async='batch'
Continue workflow (must wait for extraction)"

"Extract complete data with process_async='run'
Continue workflow (extraction in background)"