Research notes
When to read this: You want to know what the research crates in this workspace are for, how they relate to the product, and where the decisions behind them are recorded.
Chatty ships a working agent: a ReAct-shaped tool loop, memory and skills, a token budget with compaction, ATIF trace export. The research track takes a handful of agentic-self-improvement papers and asks, per mechanism, whether it earns a place in that product — as a default, as an opt-in setting, or not at all. Papers inform the design; experiments decide what ships. The full frame is Paper → experiment → product.
Work is tracked in the Linear project
Self-improving chatty2.
A small set of symbols in the research crates is reserved for the human to write; see
RESERVED.md before touching those crates.
The modules
| Module | Paper | Lands in | What it adds |
|---|---|---|---|
| M0 Trace | — (contract layer) | chatty-trace | The Trajectory every optimizer reflects on; per-step attribution; ATIF round-trip; FeedbackFn |
| M1 ReAct | Yao et al., ICLR 2023 | chatty-core (loop already ships) + chatty-trace | Fidelity to the paper’s action space; strategy variants for eval |
| M2 AFlow | Zhang et al., ICLR 2025 | chatty-flow (IR + interpreter), chatty-optimize (MCTS) | Workflow topology search offline; saved workflows run in-app |
| M3 GEPA | Agrawal et al., ICLR 2026 | chatty-optimize | Reflective prompt optimization of ModelConfig.preamble |
| M4 ACE | Zhang et al., ICLR 2026 | chatty-playbook | Evolving playbook over the memory/skills store with deterministic merge |
| M6 Retrieval | Retrieval literature (BM25, RRF, rerankers, context packing) | chatty-core (search_web, search_memory), chatty-optimize (eval) | One retrieval pipeline over two corpora, open web and agent memory, measured on SimpleQA and FRAMES |
Per-module pages, dependency graph and status: modules/index.md. The fifth paper (DGM) self-modifies its target repo and therefore lives in a separate repo, never against chatty2.
How research meets the product
- Stage A (fidelity) runs inside this workspace’s research crates. Stage B (task
value) runs in the sibling repo
harbor-chatty— see Harbor pivot for why. Neither stage promotes anything by itself; the bar is in Experiment protocol. - Shipping crates (
chatty-trace,chatty-playbook,chatty-flow) sit on the request path and are held to the same production bar aschatty-core;chatty-optimizeis build-time tooling and never blocks a chat. - Where mechanisms land: App ↔ research bridge maps production components (agent loop, memory, context window, traces, sub-agents) to the modules that touch them; Settings integration map lists the settings each promoted mechanism would surface through.
Experiment reports
Run reports, pre-registrations, spikes and their harnesses live in the private
harbor-chatty repo under
experiments/, not here. This folder keeps what documents chatty2’s own code:
the module notes, the process docs above, and the benchmark baselines that
chatty2’s benches check against
(fabric-codec-baseline.md,
fabric-hop-latency-baseline.md). A result a
shipped feature relies on is summarised in that feature’s user guide.
Decision records (ADRs)
| Doc | Decision |
|---|---|
| harbor-pivot.md | Stage B evaluation lives in Harbor, not in chatty2 |
| appworld-decision.md | Which AppWorld sandbox to evaluate against |
| cost-model.md | How optimizer runs are costed from live ModelConfig prices |
Each research crate’s scope is stated in its own README.md (linked in the table above).
Archived: the empty promotion-log template and the pre-Stage-A crate-promise pages
were kept under docs/archive/research/ for their
reasoning only; they are not maintained.