WHISPAR matches strangers for a single 20-minute anonymous conversation based on emotional compatibility, then deletes everything. That privacy promise is the entire product — which makes safety a harder problem than in a platform that logs and reviews everything by default. You can't bolt on "have a human read flagged messages later" when the design goal is that nothing survives past the timer.
Layer 1: behavioral signals, no content read at all
The first layer never looks at what anyone actually wrote. It scores a conversation on structural signals — message-length trajectory, exchange-ratio balance between the two participants, abandonment risk — into a single behavioralScore. This runs on every conversation, continuously, without needing to read a single word of content. It's cheap, fast, and privacy-preserving by construction, because there's no content to protect if none was read.
Layer 2: a scoped, ephemeral content check — only when Layer 1 says to look
Only when the behavioral score crosses a threshold (0.30) does Layer 2 activate: a Claude-based check against the last five messages, looking specifically for crisis signals. The result is scored and the content is discarded immediately after — never stored, never logged. This is the core design principle: broad, content-free monitoring runs always; narrow, content-aware analysis runs only when justified, and never persists.
Four response tiers, not a binary ban switch
A composite score above 0.50/0.70/0.85 triggers one of three escalating responses — a soft signal, a warm handoff to resources, or direct intervention — rather than a blunt block-or-allow decision. Most safety concerns in an anonymous peer-support context aren't malicious; they're someone in real distress who needs a resource, not a ban. Building four tiers instead of one threshold is what lets the system distinguish those cases.
What TDD caught that a code review would have missed
We built this test-first — the test suite existed before the implementation. That caught two bugs a manual review plausibly wouldn't have: the safety-tier values being written directly into a database column with a CHECK constraint that didn't include those exact strings (meaning every safety event would have silently failed to insert, wrapped by a try/catch that swallowed the error — zero events logged, no error raised, no signal anything was wrong), and a hook placement bug where the safety check ran before a message was actually stored, meaning the lookback window would have missed the very message that triggered the check. Both are the kind of bug that passes a demo and fails in production exactly when it matters most.
Why this matters beyond one product
Trust-and-safety architecture is the same problem whether it's an anonymous chat app, a citizen complaint portal, or an AI agent with tool access: you need a cheap always-on layer to decide what deserves a closer look, a narrow expensive layer for the cases that pass that bar, and a response model with more than two settings. We built this once for WHISPAR; it's the same shape of problem we'd bring to any system that needs to catch bad outcomes without reading everything, all the time, by default.