xp_work behavior evals
Deterministic, single-turn behavior tests for the xp_work skill. Run via:
./scripts/run_behavior_evals.sh dev/xp_work
Each case lives in evals.json:
id— stable case numbername— human labelprompt— what gets fed toclaude -pfixture(optional) — directory underfixtures/whose contents seed the test workspaceassertions— list of structured checks against the post-run workspace
Assertion types: file_exists, file_absent, file_contains (literal substring, pattern field), file_not_contains, file_matches (multi-line regex, regex field).
Adding a case:
- (Optional) create
fixtures/<case-name>/with the starting state of the test workspace (typically awork/tree). - Append an entry to
evals.jsonwith a freshid, a clearname, theprompt, thefixturereference, and the assertions. - Run
./scripts/run_behavior_evals.sh dev/xp_workand confirm it passes.
Multi-turn flows (e.g. pluck: with its planner-mode gate) are intentionally out of scope for this first cut.