vinny/CMOnet
Watch
1
0
Fork
You've already forked CMOnet
 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Count a composite action through its steps in trial metrics #145

Manually merged
vinny merged 0 commits from vinny/composite-writes into master 2026-10-17 04:32:48 -04:00 AGit
Conversation 0 Commits - Files changed -
vinny commented 2026-10-17 04:32:11 -04:00
Owner
Copy link

Fixes #144.

The trial extractor counted a composite (navigation.go_to, strike packages, library composites) and each step it sent as separate game writes. In trial tpl-subsurface-offensive-patrol (live 2026-10-17) four go_to calls refused at their route step (submarines out of comms) counted as 4 failed + 4 invalid writes; 127 writes with 6 failed and 4 invalid where 117 with 2 failed and 4 invalid happened.

  • evaluation/extract.py: a composite whose steps went out (result.data.children) is not counted again; its steps' own records are the writes. One that stopped before any step counts once. The extraction notes say so.
  • tools/lab_run.py rederive RUN_FOLDER...: derives a recorded run's metrics again from the CMOnet databases it kept; the counted losses and faults stay; a dated note says so (once).
  • Tests for both; the extractor test fails without the fix. Docs: HUMAN_PARITY_EVALUATION.md, laboratory README.

Re-extracted on the kept databases: trial 1 (surface offensive) 142 verified writes; trial 3 117 writes, 111 verified, 4 invalid, 2 failed.

🤖 Generated with Claude Code

Fixes #144. The trial extractor counted a composite (`navigation.go_to`, strike packages, library composites) and each step it sent as separate game writes. In trial tpl-subsurface-offensive-patrol (live 2026-10-17) four go_to calls refused at their route step (submarines out of comms) counted as 4 failed + 4 invalid writes; 127 writes with 6 failed and 4 invalid where 117 with 2 failed and 4 invalid happened. - `evaluation/extract.py`: a composite whose steps went out (`result.data.children`) is not counted again; its steps' own records are the writes. One that stopped before any step counts once. The extraction notes say so. - `tools/lab_run.py rederive RUN_FOLDER...`: derives a recorded run's metrics again from the CMOnet databases it kept; the counted losses and faults stay; a dated note says so (once). - Tests for both; the extractor test fails without the fix. Docs: HUMAN_PARITY_EVALUATION.md, laboratory README. Re-extracted on the kept databases: trial 1 (surface offensive) 142 verified writes; trial 3 117 writes, 111 verified, 4 invalid, 2 failed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) 
A composite and each step it sent were all counted as game writes, so a
go_to refused at its route step was one failed and one invalid write, and
a verified one three writes (live 2026-10-17, trial
tpl-subsurface-offensive-patrol: 127 writes with 6 failed, where 117 with
2 failed happened). A composite now counts through its steps, or once if
it stopped before any went out. tools/lab_run.py rederive derives a
recorded run's metrics again from the databases it kept.

Fixes #144.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Version 0.7.88 (composite-writes)
Some checks failed
check / check (ubuntu-latest) (push) Has been cancelled
Details
check / check (windows-latest) (push) Has been cancelled
Details
check / check (windows-latest) (pull_request) Has been cancelled
Details
check / check (ubuntu-latest) (pull_request) Has been cancelled
Details
55a7d8370d
vinny manually merged commit 55a7d8370d into master 2026-10-17 04:32:48 -04:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
vinny/CMOnet!145
No description provided.