⏰Attention - You’re never too old to go back to school at ODSC West’s Bootcamp ⏰
In-Person or Virtual, you’ll not only upskill with the best instructors in data science, but you’ll also be able to harness on-demand education with your complimentary 1 year subscription to Ai+.
🚀 Uncover the power of data orchestration in "Orchestrating Data Assets instead of Tasks, with Dagster," with Sandy Ryza. Sandy is lading Elementl's Dagster project and with a background in machine learning and data science leadership at KeepTruckin and Clover Health.
Explore how orchestrators serve as the backbone for keeping data assets, from datasets to ML models, up-to-date and synchronized. If you're fascinated by data engineering, machine learning, and the pivotal role of orchestrators.
#DataOrchestration #DagsterProject #DataAssets #MachineLearning #DataEngineering #Spark #Hadoop #DataScience #AI #ExpertInsights #DataOptimization #DataSynchronization #TechTalks #DataEmpowerment #AdvancedAnalytics #Leadership #Innovation
Key Topics:
0:00 - Introductions
1:26 - What is a data pipeline?
3:15 - Apache Airflow
5:55 - Building a pipeline
9:51 - The development lifecycle
11:35 - Local development
13:55 - Unit/regression testing
16:28 - Review and staging
18:18 - Deploying
21:06 - Debugging/monitoring
22:55 - To sum up
24:03 - Q&A
Follow ODSC on:
Playlists:
Related Blogs: