What shall we analyze?
Autonomy runs experiments here as it investigates. You can also start one manually.
New sandbox run
What do you want to test?
Sandbox run
Tradeoffs
Static collections of (input, output, eval) rows. Used for golden sets, calibration, annotation, and sandbox runs. Autonomy builds these as it works; you can also create one manually.
Help the analyst understand your agent's structure and goals
Strategy
Describe your agent's goals and what you're optimizing for
Terminology
Define business terms, KPIs, and domain-specific concepts
Data Structure
Span hierarchy and metadata fields (auto-generated from your traces)
Agent Change Log
Record dated changes to your agent (model swaps, prompt updates, deploys) so analysis can tie trends to causes