Lesson overview

Tool trajectories and side effects

Tool trajectories and side effects — Evaluate AI agents in practice. Outcome and route; A deterministic failure scenario; An environment without hidden memory. Practical assignment: Tool trajectories and side effects.

Demo edition. The materials are undergoing editorial review.

Topics covered

  • Outcome and route
  • A deterministic failure scenario
  • An environment without hidden memory

Practical assignment

Tool trajectories and side effects

The full lesson is available as part of the course.

Continue to learning

Course outline

  1. Turn a task into a test case
  2. Deterministic artifact checks
  3. Tool trajectories and side effects
  4. Rubrics and model-judge limits
  5. Regressions and fair version comparison
  6. Cost, latency and an acceptance report