> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cyberdesk.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Trajectories
> Understanding how Cyberdesk records, validates, and replays workflow executions
## What Are Trajectories?
Trajectories are reusable sequences of actions that Cyberdesk learns from workflow runs. Think of them as "learned paths" through your software that can be replayed deterministically to make future runs much faster and more reliable.
Cyberdesk now separates **capture** from **generation**:
* During runs, Cyberdesk may auto-save run-scoped trajectory candidates in the background.
* Candidates become user-visible trajectories only after you click **Generate Trajectory** on the corresponding completed run.
* Generated trajectories still require manual approval before replay.
## How Trajectories Work
### 1. Run Execution: Background Capture
When you run a workflow (new or existing), Cyberdesk captures trajectory candidates for non-cached execution paths:
```
User triggers workflow → Non-cached steps execute → Actions recorded → Candidate saved to run
```
**What Gets Captured**:
* Actions taken (click, type, etc.)
* Screen state before each action (pre-check snapshot)
* Agent's thought process for each step
* Final results and observations
* Display dimensions (screen resolution)
* Input values used during the original run
These run-scoped candidates are not replayable until they are generated/promoted.
**Example Trajectory Step**:
```json theme={null}
{
"func_name": "click",
"args": [850, 450],
"kwargs": {},
"pre_check_snapshot": {
"screenshot": "supabase://trajectory-images/organizations/org_123/workflows/wf_456/step_12.png",
"last_agent_thought": "I need to click the Submit button to proceed",
"custom_cache_detection_instructions": "The Submit button must be visible and enabled"
},
"result": "Clicked at coordinates (850, 450)"
}
```
### 2. Generate from Run Details (Promote to Library)
After a run completes:
```
Open run details → Click Generate Trajectory (if available) → Trajectory appears in Trajectories tab (unapproved)
```
At this stage, the trajectory is user-visible and reviewable, but still not used for replay until approved.
If you see **No new trajectory** on a run, common reasons are:
* The run was fully cached, so no new path needed to be captured.
* The run is legacy data from before this capture model.
* The run ended before a complete candidate trajectory was persisted.
### 3. Subsequent Runs: Replay with Validation
When you run the same workflow again, Cyberdesk attempts to replay the trajectory:
```
Load approved, generated trajectories → Validate screen state → Replay if match → Complete in seconds
```
**Validation Process**:
1. **Load**: System loads approved, generated trajectories for this workflow and exact display resolution
2. **Filter**: Filters trajectories whose already-replayed prefix and current step still line up with the live run
3. **Compare**: Captures the current screen and compares it against candidate snapshots
4. **Decision**: The first passing candidate is used for replay. If an optional **main trajectory** is still in contention, Cyberdesk waits for that comparison and prefers it as the tiebreaker.
5. **Replay**: If match, executes the cached action
6. **Fallback**: If no match, falls back to agent for that step
**Performance Impact**:
* **Cache Hit**: Workflow completes in seconds (no AI reasoning needed)
* **Cache Miss**: Falls back to recovery agent, which may either resume the trajectory or create a new one
### Main Trajectory Tiebreaker
When a workflow has multiple approved/generated trajectories for the same resolution, Cyberdesk can optionally mark one of them as the **main trajectory**.
* A workflow can have **zero or one** main trajectory.
* Only **approved generated** trajectories can be marked main.
* Switching the main checkbox on one trajectory clears it from the previous main automatically.
* Clearing the checkbox leaves the workflow with **no main trajectory**, which is valid and preserves the previous fallback behavior.
The main trajectory only matters when Cyberdesk would otherwise have to break a tie:
* If **multiple** trajectory candidates get a cache hit at the same step, Cyberdesk prefers the main trajectory.
* If **all** trajectory candidates miss at the same step, Cyberdesk prefers the main trajectory when building recovery context for the recovery agent.
* If **no** trajectory is marked main, Cyberdesk behaves the same way it did before this feature.
Choose a main trajectory when one path is your most stable or most desirable canonical path. If your workflow does not need that extra determinism, it is fine to leave every trajectory unchecked.
## Resume Trajectory: Smart Cache Recovery
When a cache miss occurs, Cyberdesk doesn't immediately abandon the cached trajectory. Instead, the recovery agent has the ability to **resume the trajectory** after handling minor disruptions.
### How Resume Works
When the screen state doesn't match the cached snapshot (cache miss), the recovery agent:
1. **Analyzes the situation** - Determines if this is a minor disruption or a major deviation
2. **Takes recovery actions** - Dismisses popups, closes dialogs, or fixes the screen state
3. **Decides whether to resume** - Calls `resume_trajectory` to continue the cached path, or proceeds with full recovery
```
Cache Miss → Recovery Agent → Quick Fix → resume_trajectory → Continue Cached Trajectory
↓
Major Issue → Full Recovery → New Trajectory Created
```
### When Resume is Used
The recovery agent calls `resume_trajectory` for:
* **Popups and dialogs** - Unexpected alerts, cookie banners, notifications
* **Minor UI shifts** - Elements that moved slightly but are still accessible
* **Transient states** - Loading spinners that completed, tooltips that appeared
The recovery agent proceeds with full recovery for:
* **Major UI changes** - Different page loaded, workflow state changed significantly
* **Blocked paths** - Required elements missing or inaccessible
* **Uncertain situations** - When resuming might cause incorrect behavior
### How Resume is Called
When the recovery agent determines the situation is recoverable, it calls `resume_trajectory()` to signal that the system should continue replaying the cached trajectory. The system will then retry the step that originally failed cache detection.
### Automatic Retry Logic
If `resume_trajectory` is called but the next cache check still fails:
* The system retries with the recovery agent again
* The recovery prompt includes how many prior `resume_trajectory` attempts have already failed
* The agent can then either try to restore the exact cached state again or proceed with full recovery instead
### Benefits of Resume
* **Preserves the approved trajectory** - No new trajectory is created for minor disruptions
* **Faster recovery** - Quick fixes complete in seconds vs. full re-recording
* **Consistent execution** - Continues the validated path rather than an untested new one
* **Recovery actions are not saved** - The popup dismissal or dialog close isn't added to the trajectory
Recovery actions taken before `resume_trajectory` are logged for debugging but
are not saved to the trajectory. The cached trajectory remains unchanged.
## The Approval Process
**Critical**: Newly created trajectories are **NOT automatically approved**.
They must be manually reviewed and approved before they can be used in future
workflow executions.
### Why Approval is Required
Trajectories capture exact coordinates, keyboard inputs, and action sequences. Before allowing automatic replay:
* **Verify Correctness**: Ensure the recorded actions actually worked
* **Check for Errors**: Confirm no mistakes were captured in the trajectory
* **Review Observations**: Validate that focused actions extracted correct data
* **Inspect Coordinates**: Ensure clicks hit the right UI elements
**Safety First**: Unapproved trajectories sit in your trajectory library but are never used for replay, preventing potentially incorrect actions from running automatically.
### How to Approve Trajectories
**In the Dashboard**:
1. Navigate to **Runs** (or the workflow's **Runs** section) and open a completed run.
2. Click **Generate Trajectory** on the run details panel (if available).
3. Navigate to **Workflows** → Select your workflow → **Trajectories** tab.
4. Review the trajectory:
* Check each step's screenshot and action
* Verify focused action observations
* Inspect coordinates and inputs
5. Click the **Approval** toggle or checkbox.
6. Optionally mark one approved/generated trajectory as **Main** if you want deterministic tie-breaking.
7. Trajectory is now available for future runs.
**Best Practice**: Generate and approve trajectories promptly after good runs to maximize cache hit rates.
## Editing Trajectories
You can edit trajectories in the dashboard to fix coordinates, update observations, or refine cache detection.
### What You Can Edit
**Step Actions**:
* Coordinates for clicks and drags
* Text to type
* Scroll amounts
* Screenshot zoom areas
**Pre-Check Snapshots**:
* Custom cache detection instructions
* Expected screen states
**Focused Action Observations**:
* Extracted data and observations
* Runtime variable assignments
### How to Edit
1. Open a trajectory in the trajectory viewer
2. Expand the step you want to edit
3. Click the edit icon on specific fields
4. Make your changes
5. Save the trajectory
**Important**: Changes take effect immediately on approved trajectories. If
you're making significant changes, consider unapproving the trajectory first,
testing it, then re-approving.
### Adding Wait Steps
You can add wait steps directly in the trajectory editor to introduce delays between actions. This is useful when:
* The application needs time to process between steps
* Animations or loading states need to complete
* You're experiencing timing issues during replay
**How to Add a Wait**:
1. Hover over any trajectory step
2. A **+ Wait** button appears at the bottom-right of the step
3. Click it to insert a 2-second wait after that step
4. The new wait step expands automatically so you can adjust the duration
5. Click **Save Changes** to persist
**Cache Detection Disabled**: Steps added via UI cannot use cache detection
because there's no reference screenshot. They will always replay
deterministically with the specified duration.
After adding waits to a trajectory, consider updating your workflow prompt to
include the same waits. This ensures new trajectory recordings will also
include the delays.
## Editing Focused Actions: Critical Workflow Update
**⚠️ HIGHLY RECOMMENDED**: When you edit a `focused_action` observation or
instruction in a trajectory, you should **also update the underlying workflow
prompt** to match.
### Why This Matters
When a cache miss occurs, the system falls back to running the AI agent using your workflow's main prompt. If the trajectory has diverged from the prompt, you'll get inconsistent behavior:
**Scenario**:
* **Trajectory**: `focused_action` extracts "patient\_mrn, date\_of\_birth, primary\_diagnosis"
* **Workflow Prompt**: Says to extract "patient\_mrn and date\_of\_birth" only
* **Cache Hit**: Works perfectly (uses trajectory)
* **Cache Miss**: Falls back to prompt, extracts different fields, causes output schema mismatch
### The Right Approach
When editing focused actions in trajectories:
1. **Edit the Trajectory**:
* Update the focused action instruction
* Modify the expected observation
* Adjust runtime variables if needed
2. **Edit the Workflow Prompt**:
* Update the same focused\_action instruction in the main prompt
* Ensure consistency between trajectory and prompt
* Test with cache disabled to verify prompt works
3. **Test Both Paths**:
* Run with cache hit (uses trajectory)
* Run with cache disabled (uses prompt)
* Verify both produce same results
**Example**:
**If you change trajectory from**:
```
focused_action: "Extract patient MRN only"
```
**To**:
```
focused_action: "Extract patient MRN and date of birth"
```
**Also update workflow prompt from**:
```
"Use focused_action to extract the patient MRN"
```
**To**:
```
"Use focused_action to extract the patient MRN and date of birth"
```
**Pro Tip**: When making significant trajectory edits, temporarily disable
cache detection to test the workflow with the updated prompt before
re-enabling caching.
## Cache Detection: Enabling and Disabling
Cache detection is active whenever Cyberdesk has an approved, generated trajectory for the workflow and current screen resolution. There is not currently a dedicated per-run "disable cache" toggle in the dashboard.
### When Cache Detection is Active (Default)
Every workflow run attempts to use approved, generated trajectories:
```
Run Start → Load Approved Trajectories → Validate Each Step → Replay if Match → Agent if Miss
```
**Benefits**:
* ⚡ Massive speed improvements (cache hits complete instantly)
* 🎯 Consistent execution (same actions every time)
### Disabling Cache Detection
To force fresh agent execution from the start of a workflow, temporarily unapprove the relevant trajectories in the dashboard.
**Use Cases for Disabling**:
* Testing workflow prompt changes
* Debugging trajectory mismatches
* Verifying prompt works without cache
* Development and testing
**Re-Enabling**:
* Re-approve the trajectories you want Cyberdesk to consider again
* You can also disable cache detection on individual trajectory steps inside the trajectory editor when only certain steps should always replay deterministically
## Cache Detection in Loops
By default, recorded loop steps use cache detection.
Control steps such as `end_loop_iteration` and `skip_loop_iteration` do not use cache detection, and UI-added steps also skip cache detection because they do not have reference screenshots.
You can disable cache detection on individual loop steps in the trajectory editor to make them deterministic.
If cache detection fails mid-loop, the recovery agent can:
* `resume_trajectory` for minor mismatches
* `end_loop_iteration` or `skip_loop_iteration` to continue the loop
* `declare_task_failed` if the workflow is unsafe to recover
See [Looping Tools](/workflow-prompting/looping-tools) for full details and examples.
## Custom Cache Detection Instructions
To improve cache hit rates and reduce false negatives/positives, you can add [Custom Cache Detection Instructions](/concepts/custom-cache-detection) to trajectory steps.
### What They Do
Custom instructions guide the cache validation AI by providing human context:
```
"The Submit button must be visible and enabled.
Ignore the order number displayed, as it will differ between runs."
```
### Where to Add Them
1. Open a trajectory in the trajectory viewer
2. Expand a step to view details
3. Find "Custom Cache Detection Instructions" in the Pre-check Snapshot
4. Add your guidance
5. Save changes
### When to Use Them
Add custom instructions when:
* ✅ A step frequently has false cache misses due to minor differences
* ✅ Dynamic content (timestamps, IDs) causes unnecessary validation failures
* ✅ Specific UI elements are critical while others are cosmetic
* ✅ You need to specify tolerances for acceptable variations
* ✅ Recovery agent needs context about validation criteria
**Example**:
```
"The patient list must show at least 5 entries. Patient names will vary
based on {search_term}. Focus on the list structure, not specific names."
```
### Impact on Cache Hit Rates
Well-written custom instructions can:
* Reduce false negatives (rejecting valid matches)
* Reduce false positives (accepting invalid matches)
* Improve recovery agent performance
* Provide valuable context for debugging
See the full [Custom Cache Detection Instructions](/concepts/custom-cache-detection) guide for detailed examples and best practices.
## Trajectory Lifecycle
### Phase 1: Capture (Auto-Saved)
* Workflow run executes non-cached steps
* Actions are recorded in real-time
* Candidate trajectory is saved and linked to the run
* **Status**: Pending generation (`is_generated=false`), not visible in default trajectory list
### Phase 2: Generation (Manual Promote)
* Open run details and click **Generate Trajectory**
* Candidate is promoted into the trajectory library
* **Status**: Generated but unapproved (`is_generated=true`, `is_approved=false`)
### Phase 3: Review
* View trajectory in the workflow's **Trajectories** tab
* Inspect each step and screenshot
* Verify focused action observations
* Check coordinates and inputs
### Phase 4: Approval
* Approve if trajectory is correct
* **Status**: Approved, ready for replay
### Phase 5: Usage
* Future runs load this approved trajectory
* Each step validated before replay
* Cache hits = fast execution
* Cache misses = recovery agent attempts to resume or records a new run-scoped candidate
### Phase 6: Maintenance
* Edit as needed (coordinates, instructions)
* Update workflow prompt if editing focused actions
* Test with cache disabled after significant changes
* Unapprove if major edits needed, re-approve after testing
## Multiple Trajectories Per Workflow
A single workflow can have multiple approved, generated trajectories:
**Why Multiple Trajectories?**
* **Different Input Patterns**: Trajectory for "new patient" vs "existing patient"
* **UI Variations**: Trajectory for each possible screen layout
* **Resolution Differences**: Trajectory for 1920x1080 vs 1280x720
* **Branching Paths**: Different trajectories for different workflow branches
**How Selection Works**:
1. System loads approved, generated trajectories for this workflow
2. Filters by display resolution (must match exactly)
3. Keeps only candidates whose current replay position and tool sequence still line up
4. Compares the current screen against candidate snapshots, in parallel when needed
5. Selects the first candidate that passes cache detection, with the optional main trajectory acting as the tiebreaker when it is still in contention
6. Replays selected trajectory step by step
7. Falls back to agent if no trajectory matches
**Parallel Filtering**:
* Multiple candidate comparisons can run simultaneously against the same captured screen
* This keeps selection fast even when several approved trajectories share the same step
* The main trajectory only affects ties; it does not force selection when it misses
## Best Practices
### 1. Approve Promptly
Approve successful trajectories quickly to start benefiting from cache hits on
subsequent runs.
### 2. Keep Prompts in Sync
When editing trajectories (especially focused actions), update the workflow prompt to match:
```
Trajectory Edit: focused_action observation changed
↓
Workflow Prompt: Update focused_action instruction
↓
Test: Run with cache disabled
↓
Verify: Both paths produce same results
↓
Re-approve: Trajectory ready for use
```
### 3. Use Custom Instructions Strategically
Add custom instructions to steps that:
* Frequently fail validation unnecessarily
* Have dynamic content that's acceptable to ignore
* Require nuanced validation logic
### 4. Test Before Approving
Run the workflow at least once successfully before approving the trajectory:
* Verify all actions completed correctly
* Check output data matches expectations
* Ensure focused actions extracted correct values
* Inspect coordinates hit the right UI elements
### 5. Maintain Trajectory Hygiene
* **Unapprove** trajectories that are outdated or incorrect
* **Delete** trajectories that are no longer relevant
* **Edit** trajectories when UI changes are minor (coordinate adjustments)
* **Duplicate** trajectories to create a copy for experimentation (all images are copied to new storage paths)
* **Re-record** when UI changes are major (new flow needed)
### 6. Monitor Cache Hit Rates
Inspect cache behavior in recent runs using the run message history:
* Look for cache hit, cache miss, and trajectory-selection messages
* Check which steps frequently miss cache
* Update custom instructions, edit the trajectory, or regenerate it when misses cluster around the same UI change
### 7. Consider Resolution
Trajectories are resolution-specific:
* Record trajectories at your most common resolution
* If you use multiple resolutions, you'll need multiple trajectories
* Display dimensions must match exactly for replay
## Common Scenarios
### Scenario 1: UI Element Moved
**Problem**: Button moved from (850, 450) to (850, 480)
**Solution**:
1. Open trajectory in viewer
2. Find the click step
3. Edit coordinates from (850, 450) to (850, 480)
4. Save trajectory
5. Test with cache enabled
**Quick Fix**: Minor coordinate adjustments don't require prompt changes.
### Scenario 2: Focused Action Needs More Data
**Problem**: Trajectory extracts "customer\_id" but you now need "customer\_id and email"
**Solution**:
1. Edit trajectory focused\_action observation to include email
2. **IMPORTANT**: Update workflow prompt to match:
```
Before: "Use focused_action to extract customer_id"
After: "Use focused_action to extract customer_id and email"
```
3. Test with cache disabled to verify prompt works
4. Test with cache enabled to verify trajectory works
5. Re-approve if unapproved
**Critical**: Prompt must match trajectory for consistent cache miss behavior.
### Scenario 3: Popup Causes Cache Miss (Resume Recovery)
**Problem**: An unexpected popup (cookie banner, notification, etc.) causes cache detection to fail
**What Happens Automatically**:
1. Cache detection fails due to the popup
2. Recovery agent is invoked with context about the expected screen state
3. Agent dismisses the popup
4. Agent calls `resume_trajectory()` to continue
5. Cache detection retries and succeeds
6. Workflow continues on the approved trajectory
**Result**: The popup is handled without creating a new trajectory. The dismissal action is logged but not saved.
If popups frequently cause cache misses, consider adding custom cache
detection instructions to ignore known popup elements, or update your workflow
to handle popups proactively.
### Scenario 4: Dynamic Content Causes False Misses
**Problem**: Cache detection fails because timestamps/IDs differ
**Solution**:
1. Open trajectory step
2. Add custom cache detection instruction:
```
"The form structure must match with 3 input fields.
Ignore the order ID shown - it will differ per run."
```
3. Save trajectory
4. Future runs will ignore order ID differences
See [Custom Cache Detection Instructions](/concepts/custom-cache-detection) for more examples.
### Scenario 5: Workflow Prompt Changed Significantly
**Problem**: You updated the workflow prompt with new steps or different logic
**Solution**:
1. Unapprove all existing trajectories (they're now outdated)
2. Run workflow with cache disabled (force fresh recording)
3. Review new trajectory
4. Approve new trajectory
5. Old trajectories can be deleted or kept for reference
**Don't Edit**: When prompts change significantly, recording a fresh trajectory is better than trying to edit the old one.
### Scenario 6: Want to Test Without Cache
**Problem**: Need to verify prompt works independently of cache
**Solution**:
1. Trigger run with all trajectories unapproved
2. Agent executes from scratch using workflow prompt
3. A new run-scoped trajectory candidate may be captured if non-cached steps executed
4. Open that run and click **Generate Trajectory**
5. Review and approve the generated trajectory if it's better
**Use Case**: Regression testing, prompt validation, debugging
## Trajectory Data Structure
The examples below are abbreviated for readability. Persisted trajectory rows
also include metadata such as `func_hash`, `signature_hash`, `is_method`,
`skip_cache_detection`, `post_check_snapshot`, and argument wrappers like
`{ "static_value": ... }`.
### High-Level Structure
```json theme={null}
{
"id": "uuid",
"workflow_id": "uuid",
"is_approved": false,
"dimensions": {"width": 1920, "height": 1080},
"original_input_values": {"patient_name": "John Doe"},
"trajectory_data": [
{
"func_name": "click",
"args": [x, y],
"kwargs": {},
"pre_check_snapshot": { ... },
"result": "..."
},
{
"func_name": "type",
"args": ["search text"],
"kwargs": {},
"pre_check_snapshot": { ... },
"result": "..."
},
...
]
}
```
### Pre-Check Snapshot
Captured before each action:
```json theme={null}
{
"screenshot": "supabase://trajectory-images/organizations/org_123/workflows/wf_456/step_12.png",
"last_agent_thought": "I can see the login form...",
"custom_cache_detection_instructions": "Login button must be visible",
"coordinates": {"x": 850, "y": 450},
"step_signature_hash": 123456789
}
```
### Focused Action Steps
Special handling for dynamic observations:
```json theme={null}
{
"func_name": "focused_action",
"args": ["Extract the patient MRN"],
"kwargs": {
"is_cached_action": true,
"cached_thought": "I need to find the MRN field"
},
"pre_check_snapshot": { ... },
"result": "Patient MRN: MRN12345"
}
```
When editing focused actions, remember to update the workflow prompt to match!
## Performance Benefits
### Without Trajectories (Every Run Uses AI)
```
Run 1: AI agent from scratch (45 seconds)
Run 2: AI agent from scratch (45 seconds)
Run 3: AI agent from scratch (45 seconds)
Average: 45 seconds per run
```
### With Trajectories (Cache Hits)
```
Run 1: AI agent + record trajectory (45 seconds) → Trajectory approved
Run 2: Replay trajectory (5 seconds) ✅ Cache hit
Run 3: Replay trajectory (5 seconds) ✅ Cache hit
Average: 18 seconds per run, improving to ~5 seconds as cache hits increase
```
**Typical Improvement**: 5-10x faster execution after trajectory approval
### Partial Cache Hits
Even partial cache hits provide benefits:
```
Steps 1-5: Replay from trajectory (2 seconds)
Step 6: Cache miss, use AI agent (8 seconds)
Steps 7-10: Record new actions (continue with agent)
Total: 10 seconds (vs 45 seconds from scratch)
```
### Resume Recovery Performance
When the recovery agent successfully resumes:
```
Steps 1-5: Replay from trajectory (2 seconds)
Step 6: Cache miss, popup appears
Recovery: Dismiss popup (1 second)
resume_trajectory called
Steps 6-10: Continue replay (2 seconds)
Total: 5 seconds (trajectory preserved)
```
Resume recovery is faster than partial cache hits because no new trajectory is recorded.
## Trajectory Strategies
### Strategy 1: Single Golden Trajectory
**Approach**: One approved trajectory per workflow
**Best For**:
* Highly deterministic workflows
* Consistent UI layouts
* Same input pattern every time
* Minimal variation between runs
**Pros**: Simple, easy to maintain
**Cons**: Any variation causes cache miss
### Strategy 2: Multi-Path Trajectories
**Approach**: Multiple approved trajectories for different scenarios
**Best For**:
* Workflows with branching logic
* Different input patterns (new vs existing records)
* UI variations (different screen layouts)
* Different resolution targets
**Pros**: Higher cache hit rates, handles variation
**Cons**: More trajectories to maintain
### Strategy 3: Progressive Refinement
**Approach**: Start with one trajectory, add more as variations are discovered
**Best For**:
* New workflows where patterns emerge over time
* Workflows with occasional edge cases
* Gradual optimization
**Process**:
1. Approve first successful trajectory
2. Monitor cache miss patterns
3. Identify common variations
4. Record and approve trajectories for those variations
5. Cache hit rate improves over time
## Troubleshooting
### Low Cache Hit Rate
**Symptoms**: Most runs fall back to agent instead of using cache
**Possible Causes**:
* No approved trajectories
* Custom instructions too strict
* UI changed since trajectory was recorded
* Resolution mismatch
* Input patterns vary significantly
**Solutions**:
* Approve successful trajectories
* Add custom instructions to tolerate minor differences
* Re-record trajectories after UI changes
* Ensure consistent display resolution
* Consider multiple trajectories for different input patterns
### Trajectory Approval Uncertainty
**Question**: "Should I approve this trajectory?"
**Checklist**:
* ✅ Workflow completed successfully
* ✅ Output data is correct
* ✅ Focused actions extracted expected values
* ✅ All clicks hit the right UI elements
* ✅ No errors in the run logs
* ✅ Screenshots show correct screens at each step
If all checks pass → **Approve**\
If any concern → **Review more carefully** or run again to verify
### Resume Trajectory Not Working
**Symptoms**: Recovery agent keeps creating new trajectories instead of resuming
**Possible Causes**:
* Major UI changes that can't be recovered with simple actions
* Cache detection consistently failing even after recovery
* Screen state too different from expected snapshot
**Solutions**:
* Check if the disruption is truly minor (popups, dialogs) vs major (different page, missing elements)
* Add custom cache detection instructions to be more lenient
* Re-record trajectory if the UI has changed significantly
* Check logs to see if resume was attempted but cache detection kept failing
If repeated `resume_trajectory` attempts keep failing, the recovery prompt will
call that out and suggest taking over fully. If you're seeing this
frequently, the cache detection instructions may need adjustment.
### Trajectory Editing vs Re-Recording
**Edit Trajectory When**:
* Minor coordinate adjustments (button moved slightly)
* Updating custom cache detection instructions
* Refining focused action observations (if prompt also updated)
* Small corrections that don't change workflow logic
**Re-Record Trajectory When**:
* Major UI redesign
* Workflow prompt changed significantly
* Different action sequence needed
* New steps added or removed
* Complete workflow refactor
**Rule of Thumb**: If you're changing more than 3-4 steps or the workflow
logic has changed, re-record instead of editing.
## Advanced Topics
### Parameterized Trajectories
Trajectories store the original input values that were present when the path was recorded:
**Original Input Values**:
```json theme={null}
{
"patient_name": "John Doe",
"date": "2024-01-15"
}
```
Cyberdesk uses this context during cache detection and recovery, and surfaces it in the dashboard so you can see what the trajectory was originally based on.
**What This Does Not Mean**:
```json theme={null}
{
"patient_name": "Jane Smith",
"date": "2024-01-16"
}
```
Different runtime values can help the system understand that dynamic content changed legitimately, but they do not automatically rewrite previously recorded click/type payloads inside the stored trajectory. If new inputs should change the recorded actions themselves, regenerate or edit the trajectory.
### Trajectory Versioning
Trajectories remain editable after approval:
* Saving edits updates the existing trajectory record and changes its `updated_at` timestamp
* Old trajectory data is overwritten rather than stored as a separate built-in version history
* Consider duplicating or unapproving before major edits
### Resolution Filtering
Trajectories are automatically filtered by display resolution:
* **Exact Match Required**: 1920x1080 trajectory won't work on 1280x720
* **Why**: Coordinates are absolute pixel positions
* **Solution**: Record trajectories at each resolution you use
## Summary
Trajectories are Cyberdesk's intelligent caching system that dramatically speeds up workflow execution:
* **Capture**: Run-scoped trajectory candidates are auto-saved in the background
* **Generation**: Use **Generate Trajectory** on a completed run to promote a candidate into the trajectory library
* **Approval**: Required before generated trajectories can be used
* **Validation**: AI compares screen states before each replay
* **Resume Recovery**: Minor disruptions are handled automatically without creating new trajectories
* **Editing**: Supported, but keep prompts in sync (especially for focused actions)
* **Custom Instructions**: Enhance cache detection accuracy
* **Cache Control**: Temporarily unapprove trajectories or disable cache detection on individual steps when you need fresh execution
**Key Takeaway**: Capture happens automatically, but replayable trajectories are created intentionally. Generate from good runs, approve promptly, keep prompts synchronized when editing, and use custom instructions to optimize hit rates.
For advanced cache validation, see [Custom Cache Detection Instructions](/concepts/custom-cache-detection).