What “the workflow works” means
What “the workflow works” means — Run AI workflows reliably. Process health and task outcome; A job state machine; Validate outcomes without a live model. Practical assignment: What “the workflow works” means.
Demo edition. The materials are undergoing editorial review.
Topics covered
- Process health and task outcome
- A job state machine
- Validate outcomes without a live model
Process health and task outcome
A live process can spend hours completing no work. Start with the result a user should observe. In our teaching workflow, meeting.txt passes through reading, summarization, validation and saving summary.json in a private folder. Success means a valid artifact tied to the input version, not merely exit code 0 or a model saying “done.” Define separate signals: process_alive, job_started, artifact_validated and artifact_persisted. If job_started is old but process_alive is fresh, investigate a stuck job or empty queue instead of automatically restarting everything. An empty queue should be idle. An unfinished task is running, or stuck after its specified deadline. Without this distinction, normal inactivity looks broken while activity producing no useful output looks healthy. Record the artifact path and digest as evidence of persistence.
A job state machine
Define transitions before implementing retries: queued → running → validated → completed. Use retry_wait for transient failures, failed for exhausted attempts and outcome_unknown for an uncertain external write. Never allow completed before artifact validation and persistence of its reference. A generated empty summary.json is not completed merely because a file exists. A worker records a safe errorCode and must complete or release its claimed job. If it crashes while running, another process may inspect and recover the job after its lease expires. A lease is a temporary right to perform work, not evidence that work finished. Store attemptNumber separately from jobId: a retry is another attempt at the same job, so its history must not fragment into unrelated records. Validate allowed transitions centrally rather than in scattered log statements.
Validate outcomes without a live model
Prepare a generator stub with three outputs: valid summary JSON, an empty string and a TIMEOUT exception. Pass them through the actual artifact validator and state machine. Valid JSON should be saved with inputHash; an empty result produces VALIDATION_FAILED without completed; a timeout enters retry_wait under the policy. This tests workflow plumbing, not the summary quality of a real model. Content quality needs a separate factual fixture or manual review. Save observed test results in local report.json with status, attempt and artifactHash. If you only specify the test, use not_run and do not invent hashes. A teaching stub lets you develop reliability without real API access, but establishes neither future provider cost nor latency. Keep the original input files unchanged throughout the test.
Practical assignment
What “the workflow works” means
Continue to learning