Blog

Notes from the studio

Field notes from Syed Sartaj on RAG, agents, and the path from pilot to production. Written for people who ship systems, not slide decks.

All notes

7–12 of 14

Demo traffic is cheap. Production traffic is not. Routing, caching, and hard limits that keep quality inside a real budget.

An agent looped for forty minutes and nobody saw it. Instrument traces, tool calls, and failure modes before users do.

Bolt-on chat that ignores your auth model will not survive GA. Design the thin slice, integrate properly, harden, then expand.

Agents are easy to demo and easy to over-apply. Choose rules, forms, classifiers, RAG, or agents based on risk and variability.

Evals in a slide deck do not count. Build a small suite, automate it, and make regression a release blocker.

A notebook demo that answers three questions is not a RAG system. Architecture, retrieval quality, and evaluation that survive real users.