Q1 给出了"Loop ≠ ReAct + 4 机制 + 4 失败 + 5 repo + 量化数据"。Q2 进入下一步:具体怎么做?8 个 best practice 按"影响度 + 实现成本 + 可替代性"3 维加权排序。
| 必做 / nice | Practice | 何时做 | 投入产出比 |
|---|---|---|---|
| 必做 | Max-iter + Stuck Detection | 任何 loop 项目 | ★ × 5 |
| 必做 | Scratchpad Reset | loop > 5 round | ★ × 5 |
| 必做 | Cost / Token Limits | 任何 production loop | ★ × 4 |
| nice | Reflection with Change Path | 任务容易卡住 | ★ × 4 |
| nice | Skill Library | loop > 100 round / 长期 agent | ★ × 4 |
| nice | Multi-Path Exploration | 可靠性要求 > 98% | ★ × 3 |
| nice | Hierarchical Loop | 复杂 multi-step 任务 | ★ × 3 |
| nice | Auto-Optimizer Loop | 已有 eval + 想自动化 | ★ × 2 |
经典 ReAct + 我加的 stuck detection 实战版:
def react_loop_with_safety(question: str, max_iter: int = 10): scratchpad = f"Question: {question}\n" action_history = [] # 用于 stuck detection total_tokens = 0 TOKEN_BUDGET = 50000 for i in range(max_iter): # Cost / Token limit if total_tokens > TOKEN_BUDGET: return "❌ Token budget exceeded" # 1. Think response = llm.invoke(f"{scratchpad}\n\nNext step:") thought = response.content total_tokens += response.usage.total_tokens # 2. Stuck detection (连续 3 次相同 action) action = extract_action(thought) action_history.append(action) if len(action_history) > 3 and action_history[-3:] == [action] * 3: # 强制换 path thought += "\n\nWait — you've tried this 3 times. Try a different approach." response = llm.invoke(f"{scratchpad}\n\n{thought}\n\nNew approach:") # 3. Act + Observe ...
关键设计:3 层防护 — max_iter (硬上限) + stuck detection (软提示) + token budget (成本上限)。这 3 层是 loop 工程化的核心。
Voyager 论文的核心 — 把每轮成功路径沉淀成 skill:
class SkillLibrary: def __init__(self): self.skills = {} # name → {code, description} def add_skill(self, name, code, description): """Loop 成功后, 把成功路径存为 skill.""" self.skills[name] = {"code": code, "description": description} def get_relevant_skills(self, task: str) -> list: """根据任务描述, 找最相关的 skill 列表.""" # 用 LLM 做 semantic search ... return relevant_skills def execute_skill(self, name): return self.skills[name]["code"] # Loop 中使用 def loop_with_skills(task: str): library = SkillLibrary() for i in range(100): # 1. 找相关 skill skills = library.get_relevant_skills(task) # 2. 用 skill + think/act prompt = f"Available skills: {skills}\nTask: {task}\n\nNext step:" response = llm.invoke(prompt) # 3. 成功后存 skill if "success" in response: library.add_skill(f"skill_{i}", response.tool_code, response.description)
核心洞察:loop 越跑越聪明,因为每次成功都沉淀为 skill。这是 Voyager 排 8.05 第一的关键。
Q2 给出了"8 best practice + 3 必做 + 5 nice-to-have + 代码示例"。Q3 进入 项目中应用 — 5 种典型项目 × 4 时间窗 · 何时应用哪个 best practice。