The Reflexion paper gave language agents a verbal self-critique loop

The conversation

On 20 March 2023, Noah Shinn and Beck Labash of Northeastern University and Ashwin Gopinath of MIT posted the first version of a paper then titled 'Reflexion: an autonomous agent with dynamic memory and self-reflection.' The idea: instead of updating weights through reinforcement learning, let an agent write itself a short note in plain language after a failed attempt, store it in memory, and read it back before trying again. Paired with the ReAct prompting technique already on file, the authors called the result a Reflexion agent, tested on AlfWorld and HotPotQA.

What the documents show

The version-one paper reports a ReAct agent with Reflexion reached a 97% success rate on AlfWorld within 12 autonomous trials, against 75% for a baseline ReAct agent without self-reflection. On a 100-question sample of HotPotQA it reports 51% against a 34% baseline, a 17-point gain by the authors' count. Both figures come from the authors' own scoring scripts, not an independent replication, and the models powering the runs were GPT-3.0 and GPT-3.5; this version does not mention GPT-4 or a coding benchmark. That matters because the paper's own submission history shows three later revisions through 10 October 2023, the last under a new title, 'Reflexion: Language Agents with Verbal Reinforcement Learning,' by which point the abstract cites 91% pass@1 on HumanEval against an 80% GPT-4 baseline. Citing 'the Reflexion paper's result' should specify which version.

The system boundary

Reflexion never touches model parameters. The loop runs entirely in text: an evaluator, either a binary success signal from the environment or a heuristic built to catch hallucinated actions and repeated loops, decides whether a trial failed, and only then does the agent write and store a reflection. The technique depends on that external or self-generated judgment; it does not diagnose failure on its own. The memory buffer must also stay within the model's context window, a built-in limit on how much an agent can carry forward as trials accumulate. Where no evaluator exists, or failure is ambiguous, the paper does not say what a Reflexion agent should do.

Where it fails

The authors report a third benchmark, WebShop, where Reflexion did not outperform the baseline; both runs were halted after four trials for lack of improvement. Their error analysis attributes part of this to the shopping site's search engine returning poor results regardless of the agent's plan, a tool limitation rather than a planning one, though they concede the distinction is not airtight. The project repository, MIT licensed and later tied to a NeurIPS 2023 paper, does not claim the method generalizes beyond the settings tested.

• Which evaluator decides a trial has failed, and can it be gamed by the agent's own output?
• Does the benchmark result being cited come from the same paper version as the other reported baselines?
• What happens once accumulated reflections no longer fit inside the model's context window?

Reflexion is best read as a narrow demonstration that a frozen model can improve across repeated attempts at the same task when given a clear failure signal, not as a general-purpose learning method for open-ended deployments.