Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Research notes

When to read this: You want to know what the research crates in this workspace are for, how they relate to the product, and where the decisions behind them are recorded.

Chatty ships a working agent: a ReAct-shaped tool loop, memory and skills, a token budget with compaction, ATIF trace export. The research track takes a handful of agentic-self-improvement papers and asks, per mechanism, whether it earns a place in that product — as a default, as an opt-in setting, or not at all. Papers inform the design; experiments decide what ships. The full frame is Paper → experiment → product.

Work is tracked in the Linear project Self-improving chatty2. A small set of symbols in the research crates is reserved for the human to write; see RESERVED.md before touching those crates.

The modules

ModulePaperLands inWhat it adds
M0 Trace— (contract layer)chatty-traceThe Trajectory every optimizer reflects on; per-step attribution; ATIF round-trip; FeedbackFn
M1 ReActYao et al., ICLR 2023chatty-core (loop already ships) + chatty-traceFidelity to the paper’s action space; strategy variants for eval
M2 AFlowZhang et al., ICLR 2025chatty-flow (IR + interpreter), chatty-optimize (MCTS)Workflow topology search offline; saved workflows run in-app
M3 GEPAAgrawal et al., ICLR 2026chatty-optimizeReflective prompt optimization of ModelConfig.preamble
M4 ACEZhang et al., ICLR 2026chatty-playbookEvolving playbook over the memory/skills store with deterministic merge
M6 RetrievalRetrieval literature (BM25, RRF, rerankers, context packing)chatty-core (search_web, search_memory), chatty-optimize (eval)One retrieval pipeline over two corpora, open web and agent memory, measured on SimpleQA and FRAMES

Per-module pages, dependency graph and status: modules/index.md. The fifth paper (DGM) self-modifies its target repo and therefore lives in a separate repo, never against chatty2.

How research meets the product

  • Stage A (fidelity) runs inside this workspace’s research crates. Stage B (task value) runs in the sibling repo harbor-chatty — see Harbor pivot for why. Neither stage promotes anything by itself; the bar is in Experiment protocol.
  • Shipping crates (chatty-trace, chatty-playbook, chatty-flow) sit on the request path and are held to the same production bar as chatty-core; chatty-optimize is build-time tooling and never blocks a chat.
  • Where mechanisms land: App ↔ research bridge maps production components (agent loop, memory, context window, traces, sub-agents) to the modules that touch them; Settings integration map lists the settings each promoted mechanism would surface through.

Experiment reports

Run reports, pre-registrations, spikes and their harnesses live in the private harbor-chatty repo under experiments/, not here. This folder keeps what documents chatty2’s own code: the module notes, the process docs above, and the benchmark baselines that chatty2’s benches check against (fabric-codec-baseline.md, fabric-hop-latency-baseline.md). A result a shipped feature relies on is summarised in that feature’s user guide.

Decision records (ADRs)

DocDecision
harbor-pivot.mdStage B evaluation lives in Harbor, not in chatty2
appworld-decision.mdWhich AppWorld sandbox to evaluate against
cost-model.mdHow optimizer runs are costed from live ModelConfig prices

Each research crate’s scope is stated in its own README.md (linked in the table above).

Archived: the empty promotion-log template and the pre-Stage-A crate-promise pages were kept under docs/archive/research/ for their reasoning only; they are not maintained.