If you’ve been treating “garbage in, garbage out” as a metaphor, this episode turns it into a live-fire scenario. Lori MacVittie and Joel Moses are joined by Dmitry Kit on this week's episode of #F5's Pop Goes the Stack podcast to unpack what happens when #AI systems ingest misinformation that looks legitimate, and why “just patch it” doesn’t work the way it does in traditional software. They start with a real experiment: researchers fabricated a fake medical condition, complete with fake papers, authors, and supporting citations, and watched it propagate. Within weeks, major AI systems began surfacing and citing it as real. The uncomfortable point is that once false knowledge gets embedded, you can’t reliably roll it back. Retraining is expensive, fine-tuning doesn’t truly excise the information, and even “fixes” can create unintended side effects because the bad pattern can be distributed throughout the network. The conversation reframes the core issue as trust and weighting. Models don’t learn from “the internet” evenly; they learn from sources that are implicitly ranked as more authoritative, which means poisoning a trusted channel can have outsized impact. Even without a trusted source, rare or highly specific topics are vulnerable because the model has so little competing context that a small amount of misinformation can dominate. So what can teams do? The practical guidance is to reduce the attack surface by curating the data set and narrowing scope. For enterprise use cases, that means constraining responses to approved, maintained knowledge, applying strong governance to RAG sources, and using additional validation layers, including “#LLMs as judges,” to screen what gets added. The takeaway is simple: you can’t rely on cleanup after contamination. Prevention, curation, and constraint are the only scalable strategies. Chapters: 00:00 Welcome to Pop Goes the Stack 00:35 “Poisoning the well” is real: A fake disease, Bixonimania, fooled AI fast 02:10 What do you do if someone poisoned your model? 03:16 Old problem, new scale: Data-driven ML and noisy truth 05:01 Trust-weighted sources: Why “credible” outlets amplify poison 07:21 “Don’t trust us” disclaimers aren’t a safety strategy 09:01 How big models “fix” it: System prompts + post-model controls 11:57 Enterprise reality: Small/local models can get poisoned too 12:44 Why retraining/rollback is expensive—and can break other behaviors 14:42 Not a software bug but creeping system corruption 16:26 Next attack surface: Poisoning feedback/teacher systems 18:57 Best defense → curated datasets + constrained answers + LLM judges 22:52 Key takeaway: Zero trust for AI Read the Bixonimania article: https://go.f5.net/l7le97p6 Learn how you can stay ahead of the curve and keep your stack whole with additional insights on app security, multicloud, AI, and emerging tech: https://go.f5.net/r7suf5ky More about F5: https://go.f5.net/6ce4atvu Read our blog: https://go.f5.net/eoms1eek Follow us on LinkedIn: https://go.f5.net/dpt50gjx

 67      3