vinny/CMOnet
Watch
1
0
Fork
You've already forked CMOnet
 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Every General turn re-sends the whole packet; earlier results are re-fetched #4

New issue
Closed
opened 2026-10-13 05:48:36 -04:00 by vinny · 1 comment
vinny commented 2026-10-13 05:48:36 -04:00
Owner
Copy link

Each turn rebuilds and re-sends the 6.8K system prompt plus the decision packet (agents/runtime.py):

  • about 19K prompt tokens per request, and about 135K per order (docs/llm/PROMPT_ANATOMY.md)
  • the tool menu alone is about 27% of the packet

Only the latest turn's results carry forward. Older ones come back only through evidence.recall, which led to re-fetch loops in the live runs.

Directions:

  • Order the packet so the stable part (system prompt, menu, rules) is a fixed prefix. Local servers (llama.cpp, vLLM) can then reuse it from their prefix cache.
  • Send only what changed in later turns.
  • Keep a small working memory of earlier results instead of re-fetching them.

Invariants that must hold (AGENTS.md):

  • intent and hard constraints are never truncated
  • decision briefs come only from complete validated replies

Found in the 2026-10-13 review (PR #1).

Each turn rebuilds and re-sends the 6.8K system prompt plus the decision packet (`agents/runtime.py`): - about 19K prompt tokens per request, and about 135K per order (`docs/llm/PROMPT_ANATOMY.md`) - the tool menu alone is about 27% of the packet Only the latest turn's results carry forward. Older ones come back only through `evidence.recall`, which led to re-fetch loops in the live runs. Directions: - Order the packet so the stable part (system prompt, menu, rules) is a fixed prefix. Local servers (llama.cpp, vLLM) can then reuse it from their prefix cache. - Send only what changed in later turns. - Keep a small working memory of earlier results instead of re-fetching them. Invariants that must hold (AGENTS.md): - intent and hard constraints are never truncated - decision briefs come only from complete validated replies Found in the 2026-10-13 review (PR #1).
vinny commented 2026-10-13 22:26:15 -04:00
Author
Owner
Copy link

Done in #27 (6cf8372, 0.7.13): the packet's stable sections lead it, for the local server's prefix cache, and job_progress.working_memory carries what the job already read (newest first, within 4,000 characters; a large result keeps its size and an evidence.recall pointer).

Done in #27 (6cf8372, 0.7.13): the packet's stable sections lead it, for the local server's prefix cache, and `job_progress.working_memory` carries what the job already read (newest first, within 4,000 characters; a large result keeps its size and an `evidence.recall` pointer).
vinny closed this issue 2026-10-13 22:26:15 -04:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
vinny/CMOnet#4
No description provided.