Nick Schrock is the CEO @ Elementl. This is his session for the Open Source Data Stack Conference 2021.
Bio
Nick is the founder of Elementl. Prior to Elementl he was a Principal Engineer and Director at Facebook between 2009-17, where he founded the Product Infrastructure team and co-created GraphQL.
Orchestration: Dagster
You’ve got all the parts of your data stack set up -- now you have to make sure they work together. How do you make sure that upstream ingest jobs complete in time for your warehouse transforms to run? How do you integrate external tools with the Python notebooks that build your ML models? What happens when something breaks, and how do you find out? We’ll show you how an orchestrator lets you tame the dependencies between your tools and gives you visibility into your entire data platform.
About the Event:
The emergence of the modern data stack has seen a rapid spike in the number of data tools an organization can use to drive better decision making. Each tool has become highly specialized in its portion of the data lifecycle, plus tools that are open source are a powerful way for technology buyers to reduce exposure to vendor lock-in.
Control the end-to-end flow of customer data:
To guarantee data auditability
To allow data governance
To support consumer data privacy
To enable productive engineer workflows
By taking ownership and control over your data pipeline through open source, you can reduce the “trust surface area” for your customer data.
We are bringing together the building blocks of the open source data stack to demonstrate how teams can build a data stack that reflects their needs.
Learn about the data flow lifecycle, the modern data stack pipeline, how open source tools integrate and why data teams need help to move and transform data efficiently. Discover gems about ETL, ELT, reverse-ETL, and more with 7 speakers who are passionate about the solutions that data engineers, data analysts, data scientists, data managers and marketers need when dealing with customer data and customer success.