Production LLMs with Vaishnavi Gudur

About Show #1036

What does a production-grade large language model look like? While at NDC Sydney, Richard talked with Vaishnavi Gudur from Microsoft about her work scaling LLMs for Teams transcriptions, summaries, and more! Vaishnavi discusses the underlying complexities of operating the Teams LLM infrastructure for a large array of customers across different countries and regulatory regimes. Data sovereignty also plays a large role: different countries have specific rules on where data must reside and how it can be accessed. As the scale increases and the tail gets longer, the rules set gets more complex! Lots of great thinking about what LLMs look like in a production environment.

Links

Transcripts in Microsoft Teams

Recorded April 24, 2026

Vaishnavi Gudur is a Senior Software Engineer at Microsoft with 8+ years of full-stack and distributed systems experience. She currently works on the Teams Meetings and Calling AI platform, building the surface where Copilot integrates into meetings - recordings, transcripts, AI-generated summaries, artifact management, and agentic workflows - at the scale of 300M+ Teams users worldwide. Previously, she built full-stack features in the Teams messaging and security space, including an impersonation detection capability that protected users from a growing class of social-engineering attacks. She holds an AI Frontier Ambassador role at Microsoft. Vaishnavi served as lead editor and co-author of Trustworthy AI Systems: Engineering Secure, Scalable, and Responsible Intelligence for Real Applications, published by Springer Nature. She has also contributed chapters to other Springer Nature and Wiley publications, including AI for Compliance in Cloud Environments and Security and Compliance in Cloud-Native Applications. She is a peer reviewer at multiple IEEE and ACM conferences, serves on the program committees of AIES and ICAIC, and has published research papers of her own on agentic AI and trustworthy systems. She has also served as a panelist on National Science Foundation and various other US govt project evaluations for their funding. Vaishnavi recently spoke at NDC Sydney 2026 on production-grade LLM architecture, along with sessions at SecurityWeek's AI Security Summit, IEEE Cloud Summit. She was nominated for AI & Data Science Leader of the Year at the 2025 Women in Tech Global Awards. Outside of work she's a photographer, traveler, and mindfulness practitioner who brings a human-centric lens to how she thinks about AI safety and design.

About Show #1036

Show Comments