深入理解 ReAct 模式：让大模型学会推理与行动

1
2022.10: ReAct (Yao et al.)
2
  │  首次提出 Thought-Action-Observation 循环
3
  │
4
2023: AutoGPT / BabyAGI 兴起
5
  │  基于 ReAct 思路的自主 Agent
6
  │
7
2023.06: Toolformer (Schick et al.)
8
  │  让 LLM 学会自动调用工具
9
  │
10
2023.07: LangChain 集成 ReAct
11
  │  主流框架支持
12
  │
13
2023.11: OpenAI Function Calling
14
  │  原生工具调用能力
15
  │
16
2024: Agent 系统爆发
17
  │  GPTs, Claude Tools, Gemini Function Call
18
  │
19
2024-2025: ReAct 的局限反思
20
  │  Plan-and-Execute, Reflexion, ...
21
  │  ReAct 仍是最经典的基础模式

1
遇到问题：我的航班延误了，需要改签
2

3
思考 1: 我需要先了解我的预订信息
4
行动 1: 查看我的预订 (Search Booking)
5
观察 1: 订单 #12345, 7:00 出发
6

7
思考 2: 我需要知道航班状态
8
行动 2: 查询航班状态 (Check Flight Status)
9
观察 2: 航班已取消
10

11
思考 3: 我需要找到替代航班
12
行动 3: 搜索替代航班 (Search Alternatives)
13
观察 3: 有 3 个选择: 9:00, 14:00, 18:00
14

15
思考 4: 让我选 14:00 的，因为时间合适
16
行动 4: 改签航班 (Reschedule Flight)
17
观察 4: 改签成功
18

19
思考 5: 任务完成，告知用户
20
最终答案: 航班已成功改签至 14:00

1
┌─────────────────────────────────────────────────────────────────┐
2
│                        ReAct 循环                                │
3
├─────────────────────────────────────────────────────────────────┤
4
│                                                                  │
5
│   Thought (思考)                                                 │
6
│   "我需要做什么？"                                               │
7
│       │                                                         │
8
│       ▼                                                         │
9
│   Action (行动)                                                  │
10
│   "执行具体的工具调用"                                            │
11
│       │                                                         │
12
│       ▼                                                         │
13
│   Observation (观察)                                             │
14
│   "我得到了什么结果？"                                            │
15
│       │                                                         │
16
│       ▼                                                         │
17
│   Thought (再次思考)                                             │
18
│   "根据结果，下一步怎么做？"                                       │
19
│       │                                                         │
20
│       ▼                                                         │
21
│   ... 循环直到得出答案 ...                                        │
22
│                                                                  │
23
└─────────────────────────────────────────────────────────────────┘

1
REACT_PROMPT_TEMPLATE = """
2
回答以下问题，请使用 Thought-Action-Observation 的方式一步步推理。
3

4
可用工具：
5
{tool_descriptions}
6

7
格式要求：
8
Thought: [你对当前情况的分析]
9
Action: [工具名称][工具输入]
10
Observation: [工具返回的结果]
11
... (重复 Thought/Action/Observation 直到有足够信息)
12
Thought: 我现在可以给出最终答案了
13
Final Answer: [你的最终答案]
14

15
开始！
16

17
问题：{question}
18
历史：
19
{history}
20
Thought:
21
"""

1
Thought 1: 我需要先了解用户的预订信息
2
类型: INFORMATION_SEEKING
3

4
Thought 2: 用户预订在 7:00，因此需要查询这个时间的航班
5
类型: PLANNING
6

7
Thought 3: 7:00 的航班是 AA123，让我计算是否会延误
8
类型: CALCULATION
9

10
Thought 4: 根据观察，航班延误 2 小时，我需要找替代方案
11
类型: REASONING
12

13
Thought 5: 让我验证一下这个替代航班是否还有座位
14
类型: VERIFICATION
15

16
Thought 6: 现有 5 个座位，可以预订，给用户最终建议
17
类型: FINAL_ANSWER

1
def is_finished(response):
2
    """
3
    检测 LLM 是否给出最终答案
4
    """
5
    indicators = [
6
        "Final Answer:",
7
        "Finish[",
8
        "答案：",
9
        "<final_answer>",
10
        "<invoke name=\"Finish\">",
11
    ]
12

13
    for indicator in indicators:
14
        if indicator in response:
15
            return True
16

17
    return False

1
from typing import Callable, Dict, Any
2
from dataclasses import dataclass
3

4
@dataclass
5
class Tool:
6
    """工具定义"""
7
    name: str
8
    description: str
9
    func: Callable
10
    parameters: Dict[str, Any] = None
11

12
    def to_schema(self):
13
        """转换为 OpenAI function calling 格式"""
14
        return {
15
            "type": "function",
16
            "function": {
17
                "name": self.name,
18
                "description": self.description,
19
                "parameters": self.parameters
20
            }
21
        }
22

23
    def execute(self, **kwargs):
24
        """执行工具"""
25
        try:
26
            result = self.func(**kwargs)
27
            return f"Observation: {result}"
28
        except Exception as e:
29
            return f"Observation: Error - {str(e)}"

1
class ToolRegistry:
2
    """工具注册表"""
3

4
    def __init__(self):
5
        self.tools: Dict[str, Tool] = {}
6

7
    def register(self, tool: Tool):
8
        """注册工具"""
9
        self.tools[tool.name] = tool
10

11
    def get(self, name: str) -> Tool:
12
        """获取工具"""
13
        return self.tools.get(name)
14

15
    def to_prompt(self) -> str:
16
        """生成工具描述（用于 prompt）"""
17
        descriptions = []
18
        for tool in self.tools.values():
19
            desc = f"- {tool.name}: {tool.description}"
20
            if tool.parameters:
21
                params = ", ".join(
22
                    f"{k}: {v.get('type', 'string')}"
23
                    for k, v in tool.parameters.get("properties", {}).items()
24
                )
25
                desc += f"\n  参数: {params}"
26
            descriptions.append(desc)
27
        return "\n".join(descriptions)
28

29
    def to_openai_schema(self):
30
        """生成 OpenAI function calling schema"""
31
        return [tool.to_schema() for tool in self.tools.values()]

1
# 1. 搜索工具
2
def search(query: str) -> str:
3
    """搜索信息"""
4
    # 实际实现可能调用 Google API
5
    return f"搜索结果: {query} 的相关信息..."
6

7
# 2. 计算工具
8
def calculator(expression: str) -> str:
9
    """计算数学表达式"""
10
    try:
11
        result = eval(expression)
12
        return f"计算结果: {result}"
13
    except Exception as e:
14
        return f"计算错误: {e}"
15

16
# 3. 数据库查询工具
17
def query_database(sql: str) -> str:
18
    """查询数据库"""
19
    # 实际执行 SQL
20
    return f"查询结果: ..."
21

22
# 4. 文件读取工具
23
def read_file(path: str) -> str:
24
    """读取文件内容"""
25
    with open(path, 'r', encoding='utf-8') as f:
26
        return f.read()
27

28
# 5. API 调用工具
29
def call_api(endpoint: str, params: dict) -> str:
30
    """调用 HTTP API"""
31
    import requests
32
    response = requests.get(endpoint, params=params)
33
    return response.text
34

35
# 注册工具
36
registry = ToolRegistry()
37
registry.register(Tool(
38
    name="Search",
39
    description="搜索网络信息",
40
    func=search,
41
    parameters={
42
        "type": "object",
43
        "properties": {
44
            "query": {"type": "string", "description": "搜索关键词"}
45
        },
46
        "required": ["query"]
47
    }
48
))
49
registry.register(Tool(
50
    name="Calculator",
51
    description="计算数学表达式",
52
    func=calculator,
53
    parameters={
54
        "type": "object",
55
        "properties": {
56
            "expression": {"type": "string", "description": "数学表达式"}
57
        },
58
        "required": ["expression"]
59
    }
60
))

1
import re
2
from typing import List, Tuple
3
from openai import OpenAI
4

5
class ReActAgent:
6
    """
7
    完整的 ReAct Agent 实现
8
    """
9
    def __init__(self, llm, tools: ToolRegistry, max_steps=10, max_context=5):
10
        self.llm = llm
11
        self.tools = tools
12
        self.max_steps = max_steps
13
        self.max_context = max_context
14

15
    def run(self, question: str) -> str:
16
        """执行 ReAct 循环"""
17
        history = []
18

19
        for step in range(self.max_steps):
20
            # 1. 构建 prompt
21
            prompt = self._build_prompt(question, history)
22

23
            # 2. 调用 LLM
24
            response = self.llm.generate(prompt)
25

26
            # 3. 解析响应
27
            thought, action = self._parse_response(response)
28

29
            # 4. 检查是否完成
30
            if self._is_finished(response):
31
                return self._extract_final_answer(response)
32

33
            # 5. 检查是否有有效行动
34
            if action is None:
35
                # LLM 没有产生有效行动，反馈给 LLM
36
                history.append({
37
                    'thought': thought,
38
                    'action': None,
39
                    'observation': '请选择一个有效的工具或给出最终答案。'
40
                })
41
                continue
42

43
            # 6. 执行工具
44
            tool_name, tool_input = action
45
            observation = self._execute_tool(tool_name, tool_input)
46

47
            # 7. 记录历史
48
            history.append({
49
                'thought': thought,
50
                'action': action,
51
                'observation': observation
52
            })
53

54
        # 达到最大步数
55
        return "达到最大步数，无法完成任务。"
56

57
    def _build_prompt(self, question, history):
58
        """构建 prompt"""
59
        tools_desc = self.tools.to_prompt()
60

61
        history_str = ""
62
        for h in history[-self.max_context:]:
63
            history_str += f"\nThought: {h['thought']}"
64
            if h['action']:
65
                history_str += f"\nAction: {h['action'][0]}[{h['action'][1]}]"
66
                history_str += f"\nObservation: {h['observation']}"
67
            elif h.get('observation'):
68
                history_str += f"\nObservation: {h['observation']}"
69

70
        return REACT_PROMPT_TEMPLATE.format(
71
            tool_descriptions=tools_desc,
72
            question=question,
73
            history=history_str
74
        )
75

76
    def _parse_response(self, response):
77
        """解析 Thought 和 Action"""
78
        thought_match = re.search(r'Thought:\s*(.+?)(?=Action:|$)', response, re.DOTALL)
79
        action_match = re.search(r'Action:\s*(\w+)\[(.+?)\]', response)
80

81
        thought = thought_match.group(1).strip() if thought_match else ""
82

83
        action = None
84
        if action_match:
85
            tool_name = action_match.group(1)
86
            tool_input = action_match.group(2)
87
            action = (tool_name, tool_input)
88

89
        return thought, action
90

91
    def _is_finished(self, response):
92
        """检测是否完成"""
93
        return "Final Answer:" in response or "Finish[" in response
94

95
    def _extract_final_answer(self, response):
96
        """提取最终答案"""
97
        match = re.search(r'Final Answer:\s*(.+)', response, re.DOTALL)
98
        if match:
99
            return match.group(1).strip()
100
        return response
101

102
    def _execute_tool(self, name, input_str):
103
        """执行工具"""
104
        tool = self.tools.get(name)
105
        if not tool:
106
            return f"错误：未知工具 {name}"
107

108
        try:
109
            # 简单的字符串输入处理
110
            # 实际实现可能需要更复杂的参数解析
111
            return tool.execute(input=input_str)
112
        except Exception as e:
113
            return f"执行错误：{e}"

1
class ReActAgentFunctionCall(ReActAgent):
2
    """
3
    使用 OpenAI Function Calling 的 ReAct Agent
4
    """
5
    def __init__(self, llm, tools: ToolRegistry, max_steps=10):
6
        super().__init__(llm, tools, max_steps)
7
        self.client = OpenAI()
8

9
    def run(self, question: str) -> str:
10
        """使用 Function Calling 运行"""
11
        messages = [{"role": "user", "content": question}]
12
        tool_schemas = self.tools.to_openai_schema()
13

14
        for step in range(self.max_steps):
15
            # 1. 调用 LLM
16
            response = self.client.chat.completions.create(
17
                model="gpt-4",
18
                messages=messages,
19
                tools=tool_schemas,
20
            )
21

22
            message = response.choices[0].message
23
            messages.append(message)
24

25
            # 2. 检查是否有工具调用
26
            if message.tool_calls:
27
                for tool_call in message.tool_calls:
28
                    tool_name = tool_call.function.name
29
                    tool_args = json.loads(tool_call.function.arguments)
30

31
                    # 执行工具
32
                    tool = self.tools.get(tool_name)
33
                    result = tool.execute(**tool_args)
34

35
                    # 添加结果
36
                    messages.append({
37
                        "role": "tool",
38
                        "tool_call_id": tool_call.id,
39
                        "content": str(result)
40
                    })
41
            else:
42
                # 没有工具调用，返回最终答案
43
                return message.content
44

45
        return "达到最大步数"

1
class CustomerServiceAgent(ReActAgent):
2
    """
3
    智能客服 Agent
4
    工具集：订单查询、退款、推荐等
5
    """
6
    def __init__(self, llm):
7
        tools = ToolRegistry()
8
        tools.register(Tool(
9
            name="query_order",
10
            description="查询订单状态",
11
            func=self._query_order,
12
            parameters={
13
                "type": "object",
14
                "properties": {
15
                    "order_id": {"type": "string"}
16
                }
17
            }
18
        ))
19
        tools.register(Tool(
20
            name="process_refund",
21
            description="处理退款",
22
            func=self._process_refund,
23
            parameters={
24
                "type": "object",
25
                "properties": {
26
                    "order_id": {"type": "string"},
27
                    "reason": {"type": "string"}
28
                }
29
            }
30
        ))
31
        tools.register(Tool(
32
            name="recommend_product",
33
            description="推荐相关商品",
34
            func=self._recommend_product,
35
            parameters={
36
                "type": "object",
37
                "properties": {
38
                    "category": {"type": "string"}
39
                }
40
            }
41
        ))
42

43
        super().__init__(llm, tools)
44

45
    def _query_order(self, order_id: str) -> str:
46
        # 实际查询订单
47
        return f"订单 {order_id}: 已发货，预计明天送达"
48

49
    def _process_refund(self, order_id: str, reason: str) -> str:
50
        return f"订单 {order_id} 的退款已处理，原因：{reason}"
51

52
    def _recommend_product(self, category: str) -> str:
53
        return f"在 {category} 类别下，推荐: 商品A, 商品B, 商品C"

1
class ResearchAgent(ReActAgent):
2
    """
3
    学术研究助手
4
    工具集：论文搜索、摘要、引用查询等
5
    """
6
    def __init__(self, llm):
7
        tools = ToolRegistry()
8

9
        # 注册学术工具
10
        tools.register(Tool(
11
            name="arxiv_search",
12
            description="在 arXiv 搜索论文",
13
            func=self._arxiv_search
14
        ))
15
        tools.register(Tool(
16
            name="summarize_paper",
17
            description="总结论文内容",
18
            func=self._summarize_paper
19
        ))
20
        tools.register(Tool(
21
            name="find_citations",
22
            description="查找引用",
23
            func=self._find_citations
24
        ))
25
        tools.register(Tool(
26
            name="compare_methods",
27
            description="比较方法",
28
            func=self._compare_methods
29
        ))
30

31
        super().__init__(llm, tools)
32

33
    def research_question(self, question: str) -> str:
34
        return self.run(question)

1
# 完整的 ReAct 旅行规划示例
2

3
travel_tools = ToolRegistry()
4

5
# 注册工具
6
travel_tools.register(Tool(
7
    name="SearchFlights",
8
    description="搜索航班",
9
    func=lambda origin, destination, date: f"{origin} 到 {destination} ({date}): 直飞 ¥1500, 9:00-11:00"
10
))
11

12
travel_tools.register(Tool(
13
    name="SearchHotels",
14
    description="搜索酒店",
15
    func=lambda city, checkin, nights: f"{city} 酒店 ({checkin}, {nights}晚): A酒店 ¥500/晚, B酒店 ¥800/晚"
16
))
17

18
travel_tools.register(Tool(
19
    name="GetWeather",
20
    description="查询天气",
21
    func=lambda city, date: f"{city} ({date}): 晴 25°C"
22
))
23

24
travel_tools.register(Tool(
25
    name="BookFlight",
26
    description="预订航班",
27
    func=lambda flight_id: f"已预订航班 {flight_id}"
28
))
29

30
# 运行
31
agent = ReActAgent(llm, travel_tools)
32
result = agent.run("""
33
我想去北京旅行 3 天，从上海出发，明天出发。
34
帮我安排：
35
1. 合适的航班
36
2. 性价比高的酒店
37
3. 查看当时的天气
38
""")
39

40
# ReAct 执行的可能轨迹：
41
# Thought: 需要先搜索航班
42
# Action: SearchFlights[上海, 北京, 明天]
43
# Observation: 直飞 ¥1500, 9:00-11:00
44
#
45
# Thought: 现在搜索酒店
46
# Action: SearchHotels[北京, 2026-07-17, 3]
47
# Observation: A酒店 ¥500/晚
48
#
49
# Thought: 查看天气
50
# Action: GetWeather[北京, 2026-07-17]
51
# Observation: 晴 25°C
52
#
53
# Thought: 天气好，选择经济型酒店，给出总结
54
# Final Answer: 推荐方案...

1
class ReflexionAgent(ReActAgent):
2
    """
3
    Reflexion: 反思增强的 ReAct
4
    """
5
    def __init__(self, llm, tools, max_reflections=3):
6
        super().__init__(llm, tools)
7
        self.max_reflections = max_reflections
8
        self.reflections = []
9

10
    def run(self, question: str) -> str:
11
        """带反思的运行"""
12
        for attempt in range(self.max_reflections + 1):
13
            # 1. 执行 ReAct
14
            result = super().run(question)
15

16
            # 2. 评估结果
17
            evaluation = self._evaluate_result(question, result)
18

19
            # 3. 如果成功，返回
20
            if evaluation['success']:
21
                return result
22

23
            # 4. 否则，反思
24
            if attempt < self.max_reflections:
25
                reflection = self._reflect(question, result, evaluation)
26
                self.reflections.append(reflection)
27

28
        return result
29

30
    def _reflect(self, question, result, evaluation):
31
        """反思失败原因"""
32
        prompt = f"""
33
        问题：{question}
34
        结果：{result}
35
        评估：{evaluation['reason']}
36

37
        请反思：
38
        1. 哪里出错了？
39
        2. 如何改进？
40
        3. 下次应该采取什么不同的策略？
41

42
        反思：
43
        """
44
        return self.llm.generate(prompt)
45

46
    def _build_prompt(self, question, history):
47
        """构建带反思历史的 prompt"""
48
        base_prompt = super()._build_prompt(question, history)
49

50
        if self.reflections:
51
            reflection_text = "\n\n之前的反思：\n" + "\n".join(self.reflections)
52
            return base_prompt + reflection_text
53
        return base_prompt

1
class PlanAndExecuteAgent:
2
    """
3
    Plan-and-Execute: 先计划后执行
4
    """
5
    def __init__(self, llm, tools):
6
        self.llm = llm
7
        self.tools = tools
8

9
    def run(self, question: str) -> str:
10
        # 1. 计划阶段
11
        plan = self._create_plan(question)
12

13
        # 2. 执行阶段
14
        results = []
15
        for step in plan:
16
            result = self._execute_step(step, results)
17
            results.append(result)
18

19
        # 3. 总结
20
        return self._summarize(question, plan, results)
21

22
    def _create_plan(self, question):
23
        """创建执行计划"""
24
        prompt = f"""
25
        问题：{question}
26

27
        请创建一个详细的执行计划，列出所有需要完成的步骤。
28
        每个步骤应该是一个可以执行的操作。
29

30
        计划：
31
        """
32
        plan_text = self.llm.generate(prompt)
33

34
        # 解析计划
35
        steps = []
36
        for line in plan_text.split('\n'):
37
            if re.match(r'\d+\.', line.strip()):
38
                steps.append(line.strip())
39

40
        return steps

1
class MultiAgentReAct:
2
    """
3
    多 Agent 协作系统
4
    """
5
    def __init__(self, llm, agents: Dict[str, ReActAgent]):
6
        self.llm = llm
7
        self.agents = agents
8
        self.supervisor = ReActAgent(llm, self._supervisor_tools())
9

10
    def run(self, question: str) -> str:
11
        """多 Agent 协作"""
12
        # 1. Supervisor 决定分配
13
        decision = self._assign_agent(question)
14
        agent_name = decision['agent']
15

16
        # 2. 调用专门的 Agent
17
        agent = self.agents[agent_name]
18
        result = agent.run(decision['subtask'])
19

20
        # 3. Supervisor 整合结果
21
        final = self._integrate(question, result)
22

23
        return final

1
class ReWOOAgent:
2
    """
3
    ReWOO: Reason WithOut Observation
4
    一次性规划，减少观察次数
5
    """
6
    def __init__(self, llm, tools):
7
        self.llm = llm
8
        self.tools = tools
9

10
    def run(self, question):
11
        # 1. 规划阶段：一次性生成所有步骤
12
        plan = self._create_full_plan(question)
13

14
        # 2. 执行阶段：执行所有工具调用
15
        evidence = {}
16
        for step in plan.steps:
17
            result = self._execute_tool_call(step)
18
            evidence[step.id] = result
19

20
        # 3. 总结阶段：基于所有证据回答
21
        return self._synthesize(plan, evidence, question)

1
class ReActErrorHandler:
2
    """
3
    ReAct 错误处理器
4
    """
5

6
    ERROR_TYPES = {
7
        'invalid_action': 'LLM 生成了无效的工具调用',
8
        'tool_failure': '工具执行失败',
9
        'no_progress': '多次循环无进展',
10
        'max_steps': '达到最大步数',
11
        'context_overflow': '上下文超过限制',
12
        'infinite_loop': '陷入循环'
13
    }
14

15
    def handle_error(self, history, error_type):
16
        """处理错误"""
17
        if error_type == 'invalid_action':
18
            return self._handle_invalid_action(history)
19
        elif error_type == 'tool_failure':
20
            return self._handle_tool_failure(history)
21
        elif error_type == 'infinite_loop':
22
            return self._handle_infinite_loop(history)
23
        # ...
24

25
    def _handle_infinite_loop(self, history):
26
        """处理循环"""
27
        # 检测最近的步骤是否重复
28
        if len(history) >= 3:
29
            last_three = history[-3:]
30
            if (last_three[0]['thought'] == last_three[2]['thought'] and
31
                last_three[0]['action'] == last_three[2]['action']):
32
                # 强制给予反馈
33
                return "警告：检测到循环，请尝试不同的方法。"
34
        return None

1
class RobustReActAgent(ReActAgent):
2
    """
3
    健壮的 ReAct Agent
4
    """
5
    def __init__(self, *args, max_retries=3, **kwargs):
6
        super().__init__(*args, **kwargs)
7
        self.max_retries = max_retries
8
        self.visited_states = set()
9

10
    def run(self, question):
11
        """带错误处理的运行"""
12
        for attempt in range(self.max_retries):
13
            try:
14
                return super().run(question)
15
            except Exception as e:
16
                if attempt < self.max_retries - 1:
17
                    print(f"出错：{e}, 重试...")
18
                    self.visited_states.clear()
19
                else:
20
                    return f"多次失败: {e}"

1
class LoopDetector:
2
    """循环检测器"""
3

4
    def __init__(self, threshold=3):
5
        self.threshold = threshold
6
        self.recent_states = []
7

8
    def record(self, thought, action):
9
        """记录状态"""
10
        state = (thought[:100], action)  # 截断以处理长 thought
11
        self.recent_states.append(state)
12

13
        if len(self.recent_states) > self.threshold:
14
            self.recent_states.pop(0)
15

16
    def is_looping(self):
17
        """检测是否循环"""
18
        if len(self.recent_states) < self.threshold:
19
            return False
20

21
        # 检查是否有重复
22
        states = self.recent_states[-self.threshold:]
23
        return len(set(map(str, states))) < len(states)

1
class ReActEvaluator:
2
    """
3
    ReAct Agent 评估器
4
    """
5

6
    METRICS = [
7
        'task_success_rate',       # 任务成功率
8
        'average_steps',           # 平均步数
9
        'tool_accuracy',           # 工具调用准确率
10
        'thought_quality',         # 思考质量
11
        'efficiency',              # 效率
12
        'hallucination_rate',      # 幻觉率
13
        'recovery_rate',           # 错误恢复率
14
    ]
15

16
    def evaluate(self, agent, dataset):
17
        """评估 Agent"""
18
        results = {
19
            'success_count': 0,
20
            'total_count': 0,
21
            'total_steps': 0,
22
            'tool_calls': [],
23
            'thoughts': [],
24
        }
25

26
        for item in dataset:
27
            result = agent.run(item.question)
28
            metrics = self._compute_metrics(result, item.expected)
29
            self._update_results(results, metrics)
30

31
        # 汇总
32
        total = results['total_count']
33
        return {
34
            'task_success_rate': results['success_count'] / total,
35
            'average_steps': results['total_steps'] / total,
36
            # ... 其他指标
37
        }
38

39
    def _compute_metrics(self, result, expected):
40
        """计算单个样本的指标"""
41
        return {
42
            'success': self._check_success(result, expected),
43
            'num_steps': self._count_steps(result),
44
        }

1
class ReActBenchmark:
2
    """ReAct 标准基准"""
3

4
    BENCHMARKS = {
5
        'HotpotQA': '多跳问答',
6
        'FEVER': '事实核查',
7
        'AlfWorld': '文本游戏',
8
        'WebShop': '网页购物',
9
        'Mind2Web': '网页操作',
10
        'SWE-bench': '代码任务',
11
    }
12

13
    def run_benchmark(self, agent, benchmark_name):
14
        """在基准上评估"""
15
        if benchmark_name == 'HotpotQA':
16
            return self._eval_hotpotqa(agent)
17
        elif benchmark_name == 'FEVER':
18
            return self._eval_fever(agent)
19
        # ...