DaskExecutorOptions

kedro_dagster.config.models.DaskExecutorOptions

Bases: BaseModel

Options for the Dask executor.

Parameters

Name	Type	Description	Default
cluster	DaskClusterConfig	Configuration for the Dask cluster.	required

Examples

executors:
    dask_cluster:
        dask_executor:
            cluster:
                local:
                    n_workers: 4
                    threads_per_worker: 2
jobs:
    dask_job:
        pipeline:
            pipeline_name: dask_enabled_pipeline
        executor: dask_cluster

See Also

kedro_dagster.config.models.DaskClusterConfig : Cluster configuration consumed by this executor. kedro_dagster.dagster.ExecutorCreator : Builds Dagster executor definitions from these options.

Source Code

View on GitHub

Show/Hide source

class DaskExecutorOptions(BaseModel):
    """Options for the Dask executor.

    Parameters
    ----------
    cluster : DaskClusterConfig
        Configuration for the Dask cluster.

    Examples
    --------
    ```yaml
    executors:
        dask_cluster:
            dask_executor:
                cluster:
                    local:
                        n_workers: 4
                        threads_per_worker: 2
    jobs:
        dask_job:
            pipeline:
                pipeline_name: dask_enabled_pipeline
            executor: dask_cluster
    ```

    See Also
    --------
    `kedro_dagster.config.models.DaskClusterConfig` :
        Cluster configuration consumed by this executor.
    `kedro_dagster.dagster.ExecutorCreator` :
        Builds Dagster executor definitions from these options.
    """

    cluster: DaskClusterConfig = Field(default=DaskClusterConfig(), description="Configuration for the Dask cluster.")