Dagster allows you to represent your Looker project as assets, alongside other your other technologies like dbt and Sling. This allows you to see how your Looker assets are connected to your other data assets, and how changes to other data assets might impact your Looker project.
Here, we provide interfaces to manage Looker projects using the Looker API.
( experimental ) This API may break in future versions, even between dot releases.
Represents a connection to a Looker instance and provides methods to interact with the Looker API.
( deprecated ) This API will be removed in version 1.9.0.
Use dagster_looker.load_looker_asset_specs instead.
Returns a Definitions object which will load structures from the Looker instance and translate it into assets, using the provided translator.
request_start_pdt_builds (Optional[Sequence[RequestStartPdtBuild]]) – A list of requests to start PDT builds. See https://developers.looker.com/api/explorer/4.0/types/DerivedTable/RequestStartPdtBuild?sdk=py for documentation on all available fields.
dagster_looker_translator (Optional[DagsterLookerApiTranslator]) – The translator to use to convert Looker structures into assets. Defaults to DagsterLookerApiTranslator.
A Definitions object which will contain return the Looker structures as assets.
A request to start a PDT build. See https://developers.looker.com/api/explorer/4.0/types/DerivedTable/RequestStartPdtBuild?sdk=py for documentation on all available fields.
model_name – The model of the PDT to start building.
view_name – The view name of the PDT to start building.
force_rebuild – Force rebuild of required dependent PDTs, even if they are already materialized.
force_full_incremental – Force involved incremental PDTs to fully re-materialize.
workspace – Workspace in which to materialize selected PDT (‘dev’ or default ‘production’).
source – The source of this request.
Filters the set of Looker objects to fetch.
dashboard_folders (Optional[List[List[str]]]) – A list of folder paths to fetch dashboards from. Each folder path is a list of folder names, starting from the root folder. All dashboards contained in the specified folders will be fetched. If not provided, all dashboards will be fetched.
only_fetch_explores_used_in_dashboards (bool) – If True, only explores used in the fetched dashboards will be fetched. If False, all explores will be fetched. Defaults to False.
( experimental ) This API may break in future versions, even between dot releases.
Returns a list of AssetSpecs representing the Looker structures.
looker_resource (LookerResource) – The Looker resource to fetch assets from.
dagster_looker_translator (Type[DagsterLookerApiTranslator]) – The translator to use to convert Looker structures into AssetSpecs. Defaults to DagsterLookerApiTranslator.
The set of AssetSpecs representing the Looker structures.
List[AssetSpec]
( experimental ) This API may break in future versions, even between dot releases.
Returns the AssetsDefinitions of the executable assets for the given the list of refreshable PDTs.
resource_key (str) – The resource key to use for the Looker resource.
request_start_pdt_builds (Optional[Sequence[RequestStartPdtBuild]]) – A list of requests to start PDT builds. See https://developers.looker.com/api/explorer/4.0/types/DerivedTable/RequestStartPdtBuild?sdk=py for documentation on all available fields.
dagster_looker_translator (Optional[DagsterLookerApiTranslator]) – The translator to use to convert Looker structures into assets. Defaults to DagsterLookerApiTranslator.
The AssetsDefinitions of the executable assets for the given the list of refreshable PDTs.
Here, we provide interfaces to manage Looker projects defined a set of locally accessible LookML files.
( experimental ) This API may break in future versions, even between dot releases.
Build a list of asset specs from a set of Looker structures defined in a Looker project.
project_dir (Path) – The path to the Looker project directory.
dagster_looker_translator (Optional[DagsterLookerTranslator]) – Allows customizing how to map looker structures to asset keys and asset metadata.
Examples
from pathlib import Path
from dagster import external_assets_from_specs
from dagster_looker import build_looker_asset_specs
looker_specs = build_looker_asset_specs(project_dir=Path("my_looker_project"))
looker_assets = external_assets_from_specs(looker_specs)
( experimental ) This API may break in future versions, even between dot releases.
Holds a set of methods that derive Dagster asset definition metadata given a representation of a LookML structure (dashboards, explores, views).
This class is exposed so that methods can be overriden to customize how Dagster asset metadata is derived.
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster asset key that represents the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide a custom asset key for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
The Dagster asset key that represents the LookML structure.
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster dependencies of that the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide custom dependencies for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
The Dagster dependencies for the LookML structure.
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster description of the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide a custom description for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
The Dagster description for the LookML structure.
Optional[str]
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster group name of the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide a custom group name for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
A Dagster group name for the LookML structure.
Optional[str]
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster metadata of the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide custom metadata for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
LookML structure.
Optional[Mapping[str, Any]]
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster owners of the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide custom owners for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
A sequence of Dagster owners for the LookML structure.
Optional[Sequence[str]]
A method that takes in a LookML structure (dashboards, explores, views) and returns the Dagster tags of the structure.
The LookML structure is parsed using lkml. You can learn more about this here:
https://lkml.readthedocs.io/en/latest/simple.html.
You can learn more about LookML dashboards and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/param-lookml-dashboard.
You can learn more about LookML explores and views and the properties available in this dictionary here: https://cloud.google.com/looker/docs/reference/lookml-quick-reference.
This method can be overriden to provide custom tags for a LookML structure.
lookml_structure (Tuple[Path, str, Mapping[str, Any]]) – A tuple with the path to file defining a LookML structure, the LookML structure type, and a dictionary representing a LookML structure.
LookML structure.
Optional[Mapping[str, str]]