Methods
- 01Select & adapt
Choose a parent and use feedback to steer the next attempt.
- 02Generate
Revise a candidate or carry out an investigation.
- 03Evaluate
Measure quality and correctness.
- 04Select & remember
Retain useful candidates and evidence for the next round.
Feedback updates the search state; marked methods also train model parameters.
Read the paper ↗Takeaways
These papers highlight two ways to improve research agents: change what they see in context, or train the roles that decide what to investigate next.
Refresh context
Carry useful evidence forward, then rebuild or selectively retrieve context for the next attempt.
Artifacts from the search so far.
Each branch starts a new conversation.
Reset chat historyRun each branch for a fixed horizon.
The workspace survives the reset; the winning branch supplies the next starting point. Read the paper ↗
Train the research decisions.
Give high-level decisions a dedicated learning role, with feedback on the work they produce.
Propose strategic edits to a parent solution.
Implement the edits as a complete program.
Run the program and measure its quality.
Learning happens on the current task. Only the guidance model changes; the executor stays frozen. Read the paper ↗
What’s the Next Step?
I see evaluation time as a major constraint on self-improving research agents. A promising idea still needs to be tested, and those tests can dominate the research budget. In DuoDiscover’s vLLM study, one full evaluation takes about 85 minutes. This points to two directions.
Use fewer evaluations
How can we find a better solution with fewer calls to the evaluator? Training-free methods can use past results to select more informative experiments and avoid repeating failed attempts. With training, a research policy can learn which directions are worth pursuing. Both aim for sample-efficient search: reaching a stronger verified result within a fixed evaluation budget.
Memento 3 offers a training-free example. The agent turns interaction feedback into a revisable handbook of environment rules, then compiles those rules into an executable world model. It checks the model against past observations and uses it to plan future actions. The LLM stays frozen; the handbook and simulator improve as evidence accumulates. This illustrates an algorithmic route to fewer real interactions: use what the agent has learned to guide its next attempt.
Make evaluations cheaper
Can a cheap approximation tell us which candidates deserve a full evaluation? Smaller test suites, shorter runs or surrogate models could make it possible to explore more candidates with the same resources.
DuoDiscover offers an example: agents develop optimization targets together with cheap, tailored proxy evaluators. They search using these proxies, then confirm improvements with the full evaluator. Fidelity tests also check whether proxy rankings still agree with full evaluations as the search progresses.
Working on self-improving agents, I’m always interested in the models people choose and the tasks they test them on. Some familiar tasks, such as Erdős problems and packing benchmarks, already seem close to saturation. I’ve collected the models and tasks from these papers below as a reference for planning new experiments.