When you use different models/harnesses on your project, how do you perform backpass on AGENTS.md file or simply how do you maintain that file? Is it common across harnesses/models. Codex harness would have caught/missed something else than Claude harness during a task.
Also a better quality model would not miss something which a lower quality model did. How do you determine if it should go to AGENTS.md
Great question. The model matters more than the harness here. And I think the answer is more philosophical rather than technical.
My current stance is: models will keep evolving rapidly. The models I use this month is already very different from the mix last month. We should avoid tying our workflow to any particular model. Rather, we should operate under the assumption that we'll change between different models constantly, and our workflow must survive all these shifts.
The only way that can happen is to continuously refine the setup under new circumstances. So whenever I've run some sessions under new models, I would accumulate some transcripts, and start another backpass to bake learnings into the memory. This is similar to how a team operates - employees may come and go, but the collective culture goes on.
Ahh, i see. So you are mapping your single AGENTS.md file to current mix of models you are using. Some of the stuff in AGENTS.md would be redundant for smarter model in your mix but might be necessary for relatively not so smart model in your mix, but i guess that’s the tradeoff you have to take when you mix models in your workflow.
When asking the question i did not realize this way of looking at things.
Yes that’s exactly it. The beauty here is - if I indeed only end up using the smartest model all the time, then eventually the “dumb” stuff will automatically get pruned by the backward passes.
I would strongly, strongly suggest that you do not get AI to write them. Just like everything else generated by AI, it makes them harder to read and understand.
Example: "A year later it is 900 lines, and every one of those lines is paid for on every single session, forever." Ugh!
You can try running a pass on a repo for which you have a bunch of agent sessions. It won’t change anything - everything will be presented for your review.
One mistake I made on a project was putting project and context in SKILLS.md that probably should have gone in MEMORY.md. It wasn’t checking skills unless prompted. It’s pretty confusing!
Awesome idea, I really like that and for sure I'll try to use it.
Thanks!
When you use different models/harnesses on your project, how do you perform backpass on AGENTS.md file or simply how do you maintain that file? Is it common across harnesses/models. Codex harness would have caught/missed something else than Claude harness during a task.
Also a better quality model would not miss something which a lower quality model did. How do you determine if it should go to AGENTS.md
Great question. The model matters more than the harness here. And I think the answer is more philosophical rather than technical.
My current stance is: models will keep evolving rapidly. The models I use this month is already very different from the mix last month. We should avoid tying our workflow to any particular model. Rather, we should operate under the assumption that we'll change between different models constantly, and our workflow must survive all these shifts.
The only way that can happen is to continuously refine the setup under new circumstances. So whenever I've run some sessions under new models, I would accumulate some transcripts, and start another backpass to bake learnings into the memory. This is similar to how a team operates - employees may come and go, but the collective culture goes on.
Ahh, i see. So you are mapping your single AGENTS.md file to current mix of models you are using. Some of the stuff in AGENTS.md would be redundant for smarter model in your mix but might be necessary for relatively not so smart model in your mix, but i guess that’s the tradeoff you have to take when you mix models in your workflow.
When asking the question i did not realize this way of looking at things.
I hope i understood it correctly.
Yes that’s exactly it. The beauty here is - if I indeed only end up using the smartest model all the time, then eventually the “dumb” stuff will automatically get pruned by the backward passes.
Really excited to read this article,
I would strongly, strongly suggest that you do not get AI to write them. Just like everything else generated by AI, it makes them harder to read and understand.
Example: "A year later it is 900 lines, and every one of those lines is paid for on every single session, forever." Ugh!
Elegant and genius
I’m not understanding what AGENTS.md should look like before/after a “training” pass. Do any examples exist?
You can try running a pass on a repo for which you have a bunch of agent sessions. It won’t change anything - everything will be presented for your review.
This will only get you slop.
One mistake I made on a project was putting project and context in SKILLS.md that probably should have gone in MEMORY.md. It wasn’t checking skills unless prompted. It’s pretty confusing!
Memory is not referenced unless you ask it to be
It has referenced memory without me telling it to, but good to know that it doesn’t read it by default.