Code locations#

Before the introduction of the DefinitionsAPI, definitions were grouped into repositories, and there could be many repostories in a particular code location. Refer to the Repositories documentation for info on this previous API and mental model.

A code location is a collection of Dagster definitions loadable and accessible by Dagster's tools, such as the CLI, UI, and Dagster+. A code location comprises:

A reference to a Python module that has an instance of Definitions in a top-level variable
A Python environment that can successfully load that module

Definitions within a code location have a common namespace and must have unique names. This allows them to be grouped and organized by code location in tools.

A single deployment can have one or multiple code locations.

Code locations are loaded in a different process and communicate with Dagster system processes over an RPC mechanism. This architecture provides several advantages:

When there is an update to user code, the Dagster webserver/UI can pick up the change without a restart.
You can use multiple code locations to organize jobs, but still work on all of your code locations using a single instance of the webserver/UI.
The Dagster webserver process can run in a separate Python environment from user code so job dependencies don't need to be installed into the webserver environment.
Each code location can be sourced from a separate Python environment, so teams can manage their dependencies (or even their Python versions) separately.

Relevant APIs#

Name	Description
`Definitions`	The object that contains all the definitions defined within a code location. Definitions include assets, jobs, resources, schedules, and sensors.

Defining code locations#

To define a code location, create a top-level variable that contains a Definitions object in a Python module. For example:


# definitions.py

defs = Definitions(
    assets=[dbt_customers_asset, dbt_orders_asset],
    schedules=[bi_weekly_schedule],
    sensors=[new_data_sensor],
    resources=[dbt_resource],
)

It is recommenced to include definitions in a Python module named definitions.py.

Local development#

Dagster can load a file directly as a code location. In the following example, we used the -f argument to supply the name of the file:


dagster dev -f my_file.py

This command loads the definitions in my_file.py as a code location in the current Python environment.

You can also include multiple files at a time, where each file will be loaded as a code location:


dagster dev -f my_file.py -f my_second_file.py

Dagster can also load Python modules as code locations. When this approach is used, Dagster loads the definitions defined in the module passed to the command line.

We recommend defining a variable containing the Definitions object in a submodule named definitions inside the Python module. In practice, the submodule can be created by adding a file named definitions.py at the root level of the Python module.

As this style of development eliminates an entire class of Python import errors, we strongly recommend it for Dagster projects deployed to production.

In the following example, we used the -m argument to supply the name of the module and where to find the definitions:


dagster dev -m your_module_name.definitions

This command loads the definitions in the variable containing the Definitions object in the definitions submodule in the current Python environment.

You can also include multiple modules at a time, where each module will be loaded as a code location:


dagster dev -m your_module_name.definitions -m your_second_module.definitions

To load definitions without supplying command line arguments, you can use the pyproject.toml file. This file, included in all Dagster example projects, contains a tool.dagster section with a module_name variable:


[tool.dagster]
module_name = "your_module_name.definitions"  ## name of project's Python module and where to find the definitions
code_location_name = "your_code_location_name"  ## optional, name of code location to display in the Dagster UI

When defined, you can run this in the same directory as the pyproject.toml file:


dagster dev

Instead of this:


dagster dev -m your_module_name.definitions

You can also include multiple modules at a time using the pyproject.toml file, where each module will be loaded as a code location:


[tool.dagster]
modules = [{ type = "module", name = "foo" }, { type = "module", name = "bar" }]

Refer to the Running Dagster locally guide for more info about local development, including how to configure your local instance.

Dagster+ deployment#

The dagster_cloud.yaml file is used to create and deploy code locations for Dagster+ deployments. Each code location entry in this file has a code_source property, which is used to specify how a code location is sourced. Code locations can be sourced from a Python file or module:

To load a code location from a Python file, use the python_file property in your dagster_cloud.yaml:


# dagster_cloud.yaml

locations:
  - location_name: my-code-location
    code_source:
      python_file: my_file.py

To load a code location from a Python module, use the module_name property in your dagster_cloud.yaml:


# dagster_cloud.yaml

locations:
  - location_name: my-code-location
    code_source:
      module_name: my_module_name.definitions

Open source deployment#

The workspace.yaml file is used to load code locations for open source (OSS) deployments. This file specifies how to load a collection of code locations and is typically used in advanced use cases. Refer to the Workspace files page for more info.

Definitions versus repositories#

If you used @repository in previous Dagster versions, you might be interested in how Definitions and repositories differ. Check out the following table for a high-level comparison:

	Definitions (Recommended)	Repositories
Minimum Dagster version	1.1.7	0.6
Description	Created by using the `Definitions` object assigned to a top-level variable One `Definitions` object allowed per code location	Created by using the `@repository` decorator Multiple `@repository` definitions allowed per code location
Arguments	Enforced typing and naming	No enforced typing and naming
Resources	`resources` argument can accept definitions and raw objects Top-level resources are automatically bound to all assets	Resources are manually bound to assets (`with_resources`)
Multiple Python environments	Supported for OSS deployments (via`workspace.yaml`)	Supported

Troubleshooting#

Error	Description and resolution
Cannot have more than one Definitions object defined at module scope	Dagster found multiple `Definitions` objects in a single Python module. Only one `Definitions` object may be in a single code location.