Managing alert policies with the dagster-cloud CLI#

In this guide, we'll walk you through managing Dagster+ alert policies using the dagster-cloud CLI. Alert policies apply to a full deployment (ex: prod), not Branch Deployments.

Don't want to work in code? You can also define and set alert policies using the Dagster+ UI.

How it works#

Alert policies are defined in a YAML file and then, using the dagster-cloud CLI, set for a deployment. The YAML file should contain a single top-level key (alert_policies) and contain a list of alert policy objects. For example:

# alert_policies.yaml

alert_policies:
  - name: "email-alert-policy"
    description: "An alert policy to email company executives during job failure."
    tags:
      - key: "level"
        value: "critical"
    event_types:
      - "JOB_FAILURE"
    notification_service:
      email:
        email_addresses:
          - "richard.hendricks@hooli.com"
          - "nelson.bighetti@hooli.com"

Compatible event types#

Only certain event_types can be specified together when using the CLI to create alerts. For example, multiple run-based event types can be included in the same policy, but attempting to include a tick event (such as TICK_FAILURE) will result in an error.

Prerequisites#

To follow the steps in this guide, you'll need:

• To install the dagster-cloud CLI. Refer to the dagster-cloud CLI reference for instructions.
• Organization, Admin, or Editor permissions in Dagster+, which are required to create and manage alerts
• Optional: If using an integration like Microsoft Teams or Slack as a notification service, complete the integration setup before proceeding. Refer to the Supported notification services for a complete list of integrations.

Setting policies for a deployment#

In this example, we'll configure an email alert when a job fails. This job, named important_job, has a level tag of "critical":

def important_computation():
    ...


@job(tags={"level": "critical"})
def important_job():
    important_computation()

1. First, define a list of alert policies in a YAML file. For example, the following policy listens for jobs with a tag of level: critical to fail:

   # alert_policies.yaml
   
   alert_policies:
     - name: "email-alert-policy"
       description: "An alert policy to email company executives during job failure."
       tags:
         - key: "level"
           value: "critical"
       event_types:
         - "JOB_FAILURE"
       notification_service:
         email:
           email_addresses:
             - "richard.hendricks@hooli.com"
             - "nelson.bighetti@hooli.com"
   

2. Next, use the following command to set the policies for the deployment:

   dagster-cloud deployment alert-policies sync -a /path/to/alert_policies.yaml
   

Viewing a deployment's policies#

To list the policies currently configured on the deployment, run:

dagster-cloud deployment alert-policies list

Related#