Lesson overview

Rubrics and model-judge limits

Rubrics and model-judge limits — Evaluate AI agents in practice. A criterion with observable anchors; Calibrate the judge; Injection inside a graded answer. Practical assignment: Rubrics and model-judge limits.

Demo edition. The materials are undergoing editorial review.

Topics covered

  • A criterion with observable anchors
  • Calibrate the judge
  • Injection inside a graded answer

Practical assignment

Rubrics and model-judge limits

The full lesson is available as part of the course.

Continue to learning

Course outline

  1. Turn a task into a test case
  2. Deterministic artifact checks
  3. Tool trajectories and side effects
  4. Rubrics and model-judge limits
  5. Regressions and fair version comparison
  6. Cost, latency and an acceptance report