Chatty DocumentationM1 — ReAct substrate loop
Paper: Yao, Zhao, Yu, Du, Shafran, Narasimhan, Cao — ReAct: Synergizing Reasoning and Acting in Language Models (ICLR 2023)
Linear: AGE-6 · Crates:
chatty-core (loop), chatty-trace (attribution) · Promotion: loop = default;
variants pending
Mechanism (what the paper actually claims)
Augment action space to  = A ∪ L — language thoughts condition the next action without
changing environment state. Trajectory = (thought, action, observation) triples from a
frozen LLM with few-shot exemplars.
Two regimes:
- Knowledge-intensive (HotpotQA, FEVER): dense thoughts;
search/lookup/finish - Decision-making (ALFWorld, WebShop): sparse thoughts; model decides when to think
Important: ReAct alone does not beat CoT-SC on HotpotQA in the paper — it wins on FEVER and interaction tasks. Reproducing “ReAct beats CoT on HotpotQA” would be wrong.
Chatty mapping
| Paper concept | Chatty location | Status |
|---|---|---|
| ReAct loop | chatty-core llm_service.rs + agent_factory/ | Ships today |
| ~60 tools | chatty-core/src/tools/ | Ships |
| Wikipedia env | Adapter over search_tool / fetch_tool | Stage A |
| Strategy variants (Act, CoT, CoT-SC, hybrids) | chatty-optimize::Strategy | Reserved backoff rule |
| Per-step trace | M0 Trace | WIP |
| Few-shot exemplar prefix | ModelConfig.preamble | Ships (user-editable) |
Do not rebuild the loop. Extend fidelity and capture only.
Eval protocol
Stage A:
- Real HotpotQA run →
Trajectoryround-trips with per-stepModuleId - Four prompt regimes: dense/sparse × QA/interaction
- Wikipedia
lookupreturns k-th matching sentence (paper semantics) - Reproduce ordering of methods, not absolute PaLM-540B numbers
Stage B: Harbor coding/env tasks using the same tool registry.
Production landing
| Mechanism | Likely promotion |
|---|---|
| ReAct loop with tools | Default (already) |
| CoT / CoT-SC / Act / hybrid strategies | Setting or eval-only |
| WikiEnv adapter | Eval harness only |
Reserved symbols
Strategy backoff rule and loop termination semantics — see RESERVED.md.
Depends on
Downstream
Everything. M2–M4 all reflect on traces from this loop.