Imported flows: live verification with a real model (FL-8) #8

New issue

Closed

opened 2026-10-13 05:49:51 -04:00 by vinny · 0 comments

vinny commented

2026-10-13 05:49:51 -04:00

Owner

docs/FLOWS.md lists FL-8 (live verification) as planned. The operator runs the imported "Expert commander" flow (revision 72, active since 2026-10-04) every day. But the only live run used a scripted model (docs/verification/OBSERVATION-SPEED-2026-10-03.md).

All the live evidence on model behaviour comes from the built-in General loop, with one local model (Qwen3.8-27B).

Proposal: a bounded set of order tests through the flow path, judged only on ledger-verified effects as tools/llm_order_tests.py does for the built-in loop. Then record the result in the evidence ledger.

Found in the 2026-10-13 review (PR #1).

`docs/FLOWS.md` lists FL-8 (live verification) as planned. The operator runs the imported "Expert commander" flow (revision 72, active since 2026-10-04) every day. But the only live run used a scripted model (`docs/verification/OBSERVATION-SPEED-2026-10-03.md`). All the live evidence on model behaviour comes from the built-in General loop, with one local model (Qwen3.8-27B). Proposal: a bounded set of order tests through the flow path, judged only on ledger-verified effects as `tools/llm_order_tests.py` does for the built-in loop. Then record the result in the evidence ledger. Found in the 2026-10-13 review (PR #1).

vinny referenced this issue

2026-10-13 05:52:19 -04:00

Fixes from the 2026-10-13 review: metrics, General menu, event loop, code health #1

vinny referenced this issue from a pull request that will close it,

2026-10-13 22:17:24 -04:00

LLM order tests on the operator's controller and its flow (issue #8) #26

vinny referenced this issue from a commit

2026-10-13 22:25:09 -04:00

LLM order tests on a running controller: the operator's flow with its own model (issue #8)

vinny referenced this issue from a commit

2026-10-13 22:25:09 -04:00

Flow order tests on the operator's controller: 2 of 4 after re-judging (issue #8)