A Practical Guide to Running LLMs on Kubernetes
Serving an LLM on Kubernetes has sharp edges. Includes a live demo deploying an open-weight LLM on Kubernetes and serving inference traffic with real-time GPU metrics. That is the entry point for "A Practical Guide to Running LLMs on Kubernetes", where Rohit Mishra shows how to handle GPU scheduling, model weights, health checks, autoscaling, and live inference in practice.
Session details
The path from “I have a model” to “it’s serving production traffic on my cluster” is full of sharp edges. This talk is an end-to-end walk-through: GPU device plugins and node scheduling, model weight distribution strategies for 10–140 GB artifacts, health checks that survive multi-minute model loads, and GPU-aware auto-scaling. Includes a live demo deploying an open-weight LLM on Kubernetes and serving inference traffic with real-time GPU metrics.
Primary action
Share the public session page with the tracked partner link, then use the copy variants below for LinkedIn, short social posts, and newsletter or email mentions.
Agenda facts
- Format
- Talk / Presentation
- Kind
- Conference Session
- Topic
- Artificial Intelligence and Machine Learning
- Level
- Intermediate
- Time
- Oct 29, 2026, 11:15 AM
- Room
- Room A
- Duration
- 60 min
Speakers
Link partners to speaker kits when available, or to the public speaker profile on mitechcon.net.
Rohit Mishra
Copy to paste
LinkedIn, short social, and newsletter or email copy variants for this session.
LinkedIn
Session LinkedIn Post
Serving an LLM on Kubernetes has sharp edges. Includes a live demo deploying an open-weight LLM on Kubernetes and serving inference traffic with real-time GPU metrics. That is the entry point for "A Practical Guide to Running LLMs on Kubernetes", where Rohit Mishra shows how to handle GPU scheduling, model weights, health checks, autoscaling, and live inference in practice. 📅 October 28–30, 2026 📍 Oakland University, Rochester, MI 🔗 https://www.mitechcon.net/sessions/a-practical-guide-to-running-llms-on-kubernetes/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-a-practical-guide-to-running-llms-on-kubernetes #MITechCon #Kubernetes #LLM #MLOps
Short social
Short Session Post
Serving an LLM on Kubernetes has sharp edges. Learn how to handle GPU scheduling, model weights, health checks, autoscaling, and live inference. https://www.mitechcon.net/sessions/a-practical-guide-to-running-llms-on-kubernetes/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-a-practical-guide-to-running-llms-on-kubernetes #MITechCon #Kubernetes
Newsletter / email
Newsletter Or Email Blurb
Feature this MITechCon 2026 session in your newsletter or email: "A Practical Guide to Running LLMs on Kubernetes". Serving an LLM on Kubernetes has sharp edges. Includes a live demo deploying an open-weight LLM on Kubernetes and serving inference traffic with real-time GPU metrics. That is the entry point for "A Practical Guide to Running LLMs on Kubernetes", where Rohit Mishra shows how to handle GPU scheduling, model weights, health checks, autoscaling, and live inference in practice. Format: Conference Session. Topic: Artificial Intelligence and Machine Learning. Level: Intermediate. Scheduled for Oct 29, 2026, 11:15 AM in Room A. Learn more and share the session: https://www.mitechcon.net/sessions/a-practical-guide-to-running-llms-on-kubernetes/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-a-practical-guide-to-running-llms-on-kubernetes