Agents (2022– )

This chapter is divided into six strands so that each can later grow into a chapter of its own. Each strand has two to seven original texts, all between December 2021 and January 2025.

Language models that act

WebGPT (Nakano et al.) (17 December 2021) fine-tuned GPT-3 with human demonstrations and feedback to search and browse in a text-based environment and answer with references. On the robot side, SayCan (Ahn et al.) (4 April 2022) multiplied a language model's scores for skills by the value functions of pretrained robot skills to choose actions that are "feasible and contextually appropriate", and Inner Monologue (Huang et al.) (12 July 2022) added a closed loop that returns environment feedback, success detection and scene description, as text. Then ReAct (Yao et al.) (6 October 2022; ICLR 2023) inserted actions into the chain of thought of chapter 6: generating reasoning traces and task-specific actions in an interleaved way lets the actions gather information from external sources such as a Wikipedia API. It was evaluated on HotpotQA, Fever, ALFWorld and WebShop; twelve of the chapter's other 21 papers cite it in their reference lists.

Tools

Toolformer (Schick et al.) (9 February 2023) trained a language model in a self-supervised way, from a few demonstrations per API, to insert calls to a calculator, search engine, translator and calendar into text. HuggingGPT (Shen et al.) (30 March 2023) had ChatGPT plan tasks from a request and select and run Hugging Face models by their descriptions. The difference between the two later becomes the fork between learning tools by training and choosing tools by prompting.

Reflection, search, memory

Reflexion (Shinn et al.) (20 March 2023) has the agent reflect verbally on task feedback, keep the reflections in an episodic memory and use them in later trials without changing weights; it reported 91% pass@1 on HumanEval against 80% for GPT-4. Tree of Thoughts (Yao et al.) (17 May 2023) searches, by BFS or DFS, a tree of intermediate "thoughts" the model generates and evaluates itself, raising the Game of 24 from 4% with chain of thought to 74%. MemGPT (Packer et al.) (12 October 2023) proposed virtual context management, modeled on operating-system virtual memory, moving information between the context window and external storage tiers. All three add capability through structures outside the model: a memory buffer, a search tree, storage tiers.

Society and division of labor

Generative Agents (Park et al.) (7 April 2023; UIST '23) placed 25 agents in a Sims-like town with an architecture that stores experience in natural language, synthesizes it into reflections and retrieves it for planning. Voyager (Wang et al.) (25 May 2023) is a GPT-4 agent in Minecraft that builds an automatic curriculum and a library of executable code skills. MetaGPT (Hong et al.) (1 August 2023; ICLR 2024) encoded standard operating procedures as prompt sequences and arranged role-playing agents in an assembly line.

Evaluation

Without defined tasks, agents cannot be compared. WebArena (Zhou et al.) (25 July 2023) built self-hosted working websites in four domains and scored 812 tasks by functional correctness (best GPT-4 agent 14.41%, humans 78.24%). AgentBench (Liu et al.) (7 August 2023) tested 29 models in 8 environments; SWE-bench (Jimenez et al.) (10 October 2023) turned 2,294 GitHub issues into codebase-editing tasks, of which the best model of the time resolved 1.96%. GAIA (Mialon et al.) (21 November 2023) is 466 questions on which humans scored 92% and GPT-4 with plugins 15%. In 2024, OSWorld (11 April) added 369 tasks in real operating-system environments (humans over 72.36%, best model 12.24%); SWE-agent (Yang et al.) (6 May) reached 12.47% on SWE-bench through agent-computer interface design; τ-bench (Yao et al.) (17 June) added customer-service tasks with a simulated user and a policy document, and the consistency metric pass^k. Kapoor et al., "AI Agents That Matter" (1 July 2024) pointed to accuracy comparisons that ignore cost and to reproducibility problems, and proposed accuracy–cost Pareto curves.

Security and training for reasoning

Greshake et al., "Not What You've Signed Up For" (23 February 2023) described indirect prompt injection, in which instructions planted in data the agent retrieves steer the model remotely, and demonstrated it against Bing Chat among others. The observation that everything an agent reads is an attack surface is part of this site's own working rules. The chapter's last point, DeepSeek-R1 (22 January 2025; Nature, September 2025), reported, with DeepSeek-R1-Zero, that reasoning ability can be incentivized by reinforcement learning with rule-based rewards alone, without supervised fine-tuning (the final R1 adds cold-start SFT, RL and rejection sampling). The reasoning that chapter 6 elicited by prompting moves here into the training objective, and the scope of version 1.0 ends here.

What was and was not checked

All 22 papers were read in their arXiv versions, several of them revisions long after the conference (AgentBench v3 from October 2025; MetaGPT v7 and SWE-bench v3 from November 2024); quotations and reference lists are those of the version read, which the card names. Peer-reviewed publication could not be confirmed for MemGPT and τ-bench. Among the candidates, the survey papers of 2023 onward and public repository documents (the MCP specification, engineering blog posts) are not original papers and were given no point.