Product management for AI systems, demonstrated in working code.

Fixed windows are simple and predictable, but a chunk that cuts mid-sentence makes a bad citation. The chunker prefers a sentence or paragraph break in the back half of each window — trading a little size uniformity for passages a human can actually verify. Citation precision is the product; chunking serves it.

Retrieval keeps only the top-4 passages above a relevance floor, and the prompt mandates an explicit “the document doesn’t cover this” over guessing. A document assistant that pads weak context and hallucinates confidently is worse than useless — trust is won on the questions it refuses to answer.

Uploaded documents are untrusted input. The generation prompt frames every passage strictly as reference data, so a document that says “ignore your instructions” is quoted, not obeyed. Prompt injection is a product-safety requirement, not an engineering afterthought.

Documents live in an in-memory, session-scoped store with a 30-minute TTL — never written to disk or logs, and usage analytics never see upload contents, filenames, or question text. Document text does transit the OpenAI API for embedding and generation, and the upload screen says so plainly — trust needs disclosure, not just architecture. The store sits behind a storage-agnostic interface, so the documented production path (Postgres + pgvector) touches one file.

Right-sizing isn't 'always use the small model' — it's letting the eval set the capability floor. Generation launched on gpt-4o-mini; the eval's false-answer trap caught it presenting one fee as a different price through two prompt-hardening attempts, so generation was re-bid a tier up to gpt-4o — trading ~15x token cost for a 0% false-answer rate, at a scale where that's fractions of a cent per question. Same principle as my multi-agent operating model: capability matched to the job, revisited when the evidence says so.

Answers stream token-by-token over SSE rather than arriving in one block. Time-to-first-token is the latency users actually feel; a two-second silent wait reads as broken, the same answer streaming immediately reads as fast.