DaskClusterConfig

kedro_dagster.config.models.DaskClusterConfig

Bases: BaseModel

Configuration for the Dask cluster.

Parameters

Name	Type	Description	Default
existing	dict[str, str] or None	Connect to an existing scheduler.	required
local	dict[str, Any] or None	Local cluster configuration.	required
yarn	dict[str, Any] or None	YARN cluster configuration.	required
ssh	dict[str, Any] or None	SSH cluster configuration.	required
pbs	dict[str, Any] or None	PBS cluster configuration.	required
moab	dict[str, Any] or None	Moab cluster configuration.	required
sge	dict[str, Any] or None	SGE cluster configuration.	required
lsf	dict[str, Any] or None	LSF cluster configuration.	required
slurm	dict[str, Any] or None	SLURM cluster configuration.	required
oar	dict[str, Any] or None	OAR cluster configuration.	required
kube	dict[str, Any] or None	Kubernetes cluster configuration.	required

See Also

kedro_dagster.config.models.DaskExecutorOptions : Uses this as its cluster configuration.

Source Code

View on GitHub

Show/Hide source

class DaskClusterConfig(BaseModel):
    """Configuration for the Dask cluster.

    Parameters
    ----------
    existing : dict[str, str] or None
        Connect to an existing scheduler.
    local : dict[str, Any] or None
        Local cluster configuration.
    yarn : dict[str, Any] or None
        YARN cluster configuration.
    ssh : dict[str, Any] or None
        SSH cluster configuration.
    pbs : dict[str, Any] or None
        PBS cluster configuration.
    moab : dict[str, Any] or None
        Moab cluster configuration.
    sge : dict[str, Any] or None
        SGE cluster configuration.
    lsf : dict[str, Any] or None
        LSF cluster configuration.
    slurm : dict[str, Any] or None
        SLURM cluster configuration.
    oar : dict[str, Any] or None
        OAR cluster configuration.
    kube : dict[str, Any] or None
        Kubernetes cluster configuration.

    See Also
    --------
    `kedro_dagster.config.models.DaskExecutorOptions` :
        Uses this as its cluster configuration.
    """

    existing: dict[str, str] | None = Field(default=None, description="Connect to an existing scheduler.")
    local: dict[str, Any] | None = Field(default=None, description="Local cluster configuration.")
    yarn: dict[str, Any] | None = Field(default=None, description="YARN cluster configuration.")
    ssh: dict[str, Any] | None = Field(default=None, description="SSH cluster configuration.")
    pbs: dict[str, Any] | None = Field(default=None, description="PBS cluster configuration.")
    moab: dict[str, Any] | None = Field(default=None, description="Moab cluster configuration.")
    sge: dict[str, Any] | None = Field(default=None, description="SGE cluster configuration.")
    lsf: dict[str, Any] | None = Field(default=None, description="LSF cluster configuration.")
    slurm: dict[str, Any] | None = Field(default=None, description="SLURM cluster configuration.")
    oar: dict[str, Any] | None = Field(default=None, description="OAR cluster configuration.")
    kube: dict[str, Any] | None = Field(default=None, description="Kubernetes cluster configuration.")