Skip to main content

Dagster & Embedded ELT

The dagster-embedded-elt package provides a framework for building ELT pipelines with Dagster through helpful asset decorators and resources. It includes the dagster-dlt and dagster-sling packages, which you can also use on their own. To get started,

This package includes two integrations:

  • Sling provides a simple way to sync data between databases and file systems.
  • data Load Tool (dlt) easily loads data from external systems and APIs.
note

This integration is currently experimental.

Installation​

pip install dagster-embedded-elt

Sling​

Sling provides an easy-to-use YAML configuration layer for loading data from files, replicating data between databases, exporting custom SQL queries to cloud storage, and much more. The Dagster integration allows you to derive Dagster assets from a replication configuration file.

For more information, see the Sling integration docs.

dlt​

With the ability to leverage pre-made verified sources like Hubspot and Notion, and destinations like Databricks and Snowflake, integrating dlt into your Dagster project enables you to load a data in an easy and structured way.

For more information, see the dlt integration docs.