Test

When to read this: You want to run the right part of the suite for the change you made, understand why CI runs tests the way it does, or add a test in the place the next reader will look.

Goal

Know which make target or cargo test invocation covers your change, and be able to update a characterization golden deliberately rather than by accident.

Prerequisites

• Build and run: make setup done.
• make wasm-modules has been run (scripts/build-wasm-fixtures.sh). It stages every fixture and the echo and benford plugins under target/wasm-fixtures/<name>/, which is git-ignored; the integration and conformance suites fail with a missing-file error without it.

How the suite is organised

Steps

1. Inner loop

make test-fast                    # cargo test -p chatty-core --lib
cargo test -p chatty-core --lib my_module::   # one module

2. The crate you touched

make test-tui       # cargo test -p chatty-tui
make test-gpui      # cargo test -p chatty-gpui
make test-gateway   # cargo test -p chatty-protocol-gateway

3. Everything, the way CI runs it

make test           # cargo test --all-features

--all-features matters: the excel, docx, pdf, pptx, math-render, mermaid, duckdb and browser tool groups only compile with their feature on, so a plain cargo test never sees them. make ci adds formatting, clippy and the repository check scripts; see Make targets & CI workflows.

Note

Tests run in parallel. CI used to pass --test-threads=1 because chatty-core tests intermittently died with SIGTRAP. The cause was pdfium, which is not thread-safe, being used from several test threads at once; create_pdfium() now hands out a PdfiumHandle that serialises every use (AGE-176). If a SIGTRAP, SIGSEGV or SIGABRT comes back under cargo test, look for a new pdfium call that bypasses create_pdfium() rather than re-adding the flag.

4. Characterization goldens

Turn orchestration — which chunk produces which event, which chunks end a turn, that cancellation produces a cancelled outcome, that sub-agent progress is drained ahead of stream chunks — is pinned by golden files rather than by assertions scattered across tests.

The scripted streams live in chatty_core::services::stream_fixtures (scenarios(), clarification_scenario(), scripted_stream, assert_golden). The module is compiled under #[cfg(any(test, feature = "test-support"))]; chatty-tui and chatty-gpui enable chatty-core/test-support from their dev-dependencies only, so it never reaches a release build.

Both frontends and the core loop drive the same scenarios and record their own event type, so a divergence between the two UIs shows up as a difference between two golden directories:

Each directory holds one file per scenario: text_only, tool_call_then_result, tool_error, approval_granted, approval_denied, clarification_requested, provider_error_mid_stream, cancelled_mid_stream, sub_agent_progress, token_usage_on_done.

When a golden fails, the assertion prints both sequences. If the change is deliberate, rewrite the files and explain the diff in the PR:

UPDATE_GOLDENS=1 cargo test -p chatty-core --lib loop_callback_sequence
UPDATE_GOLDENS=1 cargo test -p chatty-core session::
UPDATE_GOLDENS=1 cargo test -p chatty-tui characterization
UPDATE_GOLDENS=1 cargo test -p chatty-gpui characterization

Deliberately not pinned: the real-time interleaving of progress events with chunks (racy in production, so scenarios queue progress before the chunks it accompanies), and anything wall-clock based — the stall watchdog has its own tests in stream_processor.rs.

5. Where mocks live

• crates/chatty-core/src/tools/test_helpers.rs — MockMcpRepository and MockA2aRepository, in-memory repositories with injectable load/save errors, #[cfg(test)] only. Used by the MCP and A2A tool tests.
• Providers are not mocked; the scripted streams above stand in for a provider whenever a test needs “what the model sent”.
• The desktop characterization test talks to real entities (a ChatView in a headless test window, a live StreamManager), not stubs, so what it records is what the UI receives.
• browser_live.rs builds a throwaway workspace with tempfile rather than fixtures on disk.

Verify

• make test-fast is green after a core change; the crate target is green after a frontend change.
• make test (full, serialised) is green before you push.
• A golden you changed on purpose has its diff explained in the PR.

Checklist

• make wasm-modules has been run on this checkout
• New logic in chatty-core has a unit test beside it
• A new tool has a gemini_compat guard (see Your first change)
• Turn-lifecycle changes re-recorded all three golden directories, not one
• make test passes

Common mistakes