Designing Data Pipelines That Don’t Hate You Six Months Later
Data pipelines rarely fail on day one. The failure usually arrives later, when schema drift, volume growth, and ownership changes expose early shortcuts. Chris Birie uses "Designing Data Pipelines That Don’t Hate You Six Months Later" to follow that thread into design patterns for schema change, idempotency, observability, and growth before handoff gets painful, giving attendees a practical way to bring the lesson back to their own systems.
Session details
Most data pipelines don’t fail on day one. They fail months later — when requirements change, data volume grows, or the original developer is no longer around. Much like martial arts training, pipelines built on shaky fundamentals tend to collapse under pressure. This session focuses on practical data pipeline design patterns that emphasize durability, maintainability, and operational confidence. Drawing from experience modernizing enterprise data platforms, I’ll walk through common failure modes in batch and streaming pipelines and how disciplined upfront design decisions can prevent them. We’ll cover topics such as schema evolution, idempotency, observability, testing strategies, and scaling — grounded in real examples of what worked, what broke, and what we had to correct after the fact. There are no silver bullets here — just repeatable patterns and lessons learned from systems that had to stay running while everything around them changed. This talk is tool-agnostic and focused on principles that apply whether you’re using Python, Spark, Kafka, or traditional ETL platforms. What’s in it for the attendee - How to design pipelines that withstand change and growth - Patterns for building observability and testability into data systems - Common anti-patterns that lead to brittle pipelines - Practical guidance rooted in real production experience
Primary action
Share the public session page with the tracked partner link, then use the copy variants below for LinkedIn, short social posts, and newsletter or email mentions.
Agenda facts
- Format
- Talk / Presentation
- Kind
- Conference Session
- Topic
- Software Engineering and Architecture
- Level
- Beginner
- Time
- Schedule pending
- Room
- Room pending
Speakers
Link partners to speaker kits when available, or to the public speaker profile on mitechcon.net.
Chris Birie
Copy to paste
LinkedIn, short social, and newsletter or email copy variants for this session.
LinkedIn
Session LinkedIn Post
Data pipelines rarely fail on day one. The failure usually arrives later, when schema drift, volume growth, and ownership changes expose early shortcuts. Chris Birie uses "Designing Data Pipelines That Don’t Hate You Six Months Later" to follow that thread into design patterns for schema change, idempotency, observability, and growth before handoff gets painful, giving attendees a practical way to bring the lesson back to their own systems. 📅 October 28–30, 2026 📍 Oakland University, Rochester, MI 🔗 https://www.mitechcon.net/sessions/designing-data-pipelines-that-don-t-hate-you-six-months-later/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-designing-data-pipelines-that-don-t-hate-you-six-months-later #MITechCon #DataEngineering #DataPipelines #Observability
Short social
Short Session Post
Data pipelines rarely fail on day one. Learn design patterns for schema change, idempotency, observability, and growth before handoff gets painful. https://www.mitechcon.net/sessions/designing-data-pipelines-that-don-t-hate-you-six-months-later/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-designing-data-pipelines-that-don-t-hate-you-six-months-later #MITechCon #DataEngineering
Newsletter / email
Newsletter Or Email Blurb
Feature this MITechCon 2026 session in your newsletter or email: "Designing Data Pipelines That Don’t Hate You Six Months Later". Data pipelines rarely fail on day one. The failure usually arrives later, when schema drift, volume growth, and ownership changes expose early shortcuts. Chris Birie uses "Designing Data Pipelines That Don’t Hate You Six Months Later" to follow that thread into design patterns for schema change, idempotency, observability, and growth before handoff gets painful, giving attendees a practical way to bring the lesson back to their own systems. Format: Conference Session. Topic: Software Engineering and Architecture. Level: Beginner. Schedule details are being finalized. Learn more and share the session: https://www.mitechcon.net/sessions/designing-data-pipelines-that-don-t-hate-you-six-months-later/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-designing-data-pipelines-that-don-t-hate-you-six-months-later