Chipotle’s chatbot becoming an unofficial coding assistant wasn’t just a funny internet moment. It was a clear signal that most chatbot “guardrails” are still too shallow for systems that are optimized to be helpful, not correct, and definitely not restrained. In this episode of Pop Goes the Stack, #F5's Lori MacVittie, Joel Moses, and Emmet McGinnity unpack why jailbreaks and topic drift keep happening, and what teams can do to keep chatbots focused on the job they were actually deployed to do. Emmet’s core point is that safety measures have to start with narrowing scope. A chatbot should operate like a laser pointer, not a flashlight: it should ignore 99% of what the base model can do and stay inside a tight slice of allowed behavior. That begins with a system prompt, but it can’t end there. Naive keyword and regex filtering is easy to bypass with encoding tricks and prompt manipulation, so stronger approaches include adding a verifier or judge agent that evaluates the conversation holistically to detect when it’s drifting out of bounds. They also highlight that long conversations are a common failure mode. As context grows, it becomes easier for the model to veer into capabilities it shouldn’t use, including writing code or pulling sensitive data. Practical controls include summarizing and “squashing” sessions, pruning context when drift begins, rolling back to a safe point in the conversation, or forcing a full reset when needed. A key theme is permissioning: the chatbot must honor what the user is allowed to do, not what the chatbot can access. The real measure of a safe, successful chatbot isn’t the breadth of its knowledge, it’s what it reliably chooses not to do. If you’re deploying chatbots in production, this episode is a useful blueprint for focusing scope, monitoring drift, and enforcing boundaries before someone else does it for you. Chapters: 00:00 Welcome to Pop Goes the Stack 00:23 Why chatbot guardrails aren’t working, and jailbreaks are easy 01:26 Chipotle chatbot: From burritos to Python scripts 02:36 Laser pointer vs flashlight: Why focusing the #LLM matters 03:32 Guardrails basics: Narrow the bot’s allowed knowledge 05:29 Grounding with FAQs and “fit for purpose” prompts 06:24 Regex guardrails are naive, ROT13 and other tricks 07:20 Judge/verifier agent: Scan the whole conversation for drift 08:31 System prompt tuning: Positive + negative examples 11:05 Persona drift: Bots “remember,” contexts grow, focus erodes 15:05 Contextual permissioning: User’s rights over bot’s access 18:04 Fixing drift: Summarize, prune context, or hard reset sessions 20:11 Optimizing a successful chatbot: Quiet side of AI safety Learn how you can stay ahead of the curve and keep your stack whole with additional insights on app security, multicloud, AI, and emerging tech: https://go.f5.net/47eag5t8 More about F5: https://go.f5.net/tmx3l9li Read our blog: https://go.f5.net/8oxkpokp Follow us on LinkedIn: https://go.f5.net/4oa2vufj

 79      1