Dagster is a declarative, asset-based orchestrator that redefines the way we think about managing workflows. In this video, we’ll learn about features of Dagster including Assets, Resources, Jobs, Schedules, Partitions and more. We’ll do this by creating an End to End project from scratch using Dagster, data load tool (dlt) and Snowflake. I’m super excited about this one. Hope you’ll learn something new!
Timestamps ⏰
0:00 - Intro
1:45 - System Design
6:30 - Setup Dagster and data load tool (dlt)
21:40 - How to define Assets
40:08 - Resources, code refactor
43:22 - Schedules and Jobs
48:42 - Backfill DAGs using Partitions
54:39 - Sensors and Automaterialization policy
1:09:31 - Final thoughts
Who am I? 🙋🏻‍♂️
I'm Jay, I love making videos about travel, self-help and tech. I currently work in New York City as a data engineer, but I grew up in Malaysia and lived in the UK when I was 19. Back then, I had no idea what life was about, moving to so many places, navigating career in Tech. Today, I've learned a lot and wanna share my perspective through filmmaking.
Socials 📱
Sub Count: 11,390