On this page

Context grows quietly

A session begins with system instructions and project guidance, then adds your messages, file contents, command output, and model responses. Long autonomous runs can revisit that material many times.

That is why one short follow-up can be expensive late in a conversation. The new message is small, but the working history is not.

Caching helps, but does not erase usage

Many providers discount cached input, which can make repeated prefixes cheaper. Cached tokens may still count toward throughput limits, and a cache miss can suddenly change cost. Track cached and uncached input separately where the API exposes both.

Do not assume a long conversation is efficient simply because it once produced a cache hit.

Keep your usage visible while you work. Get Super Notchy for $27 ↗

Control the working set

Give the agent a focused task, point it at the relevant files, keep command output concise, and start a fresh session when the old one has accumulated unrelated work. For local models, watch context percentage and generation speed. For cloud tools, watch the usage pace before and after the cleanup.

A tracker shows the effect. If the same class of task consumes less after tightening context, the change is working.

Sources and further reading