Skip to content

JobOptions

kedro_dagster.config.models.JobOptions

Bases: BaseModel

Configuration options for a Dagster job.

Parameters

Name Type Description Default
pipeline PipelineOptions

Pipeline options specifying which pipeline and nodes to run.

 required
executor ExecutorOptions or str or None

Executor options instance or string key referencing an executor.

 required
schedule ScheduleOptions or str or None

Schedule options instance or string key referencing a schedule.

 required
loggers list[LoggerOptions or str] or None

List of logger configurations (inline LoggerOptions) or logger names (strings) to attach to the job.

 required

Examples

jobs:
  my_data_processing:
    pipeline:
      pipeline_name: data_processing
      node_namespaces: [price_predictor]
      tags: [test1]
    executor: multiprocessing
    schedule: daily_schedule
    loggers: [console, file_logger]

See Also

kedro_dagster.config.models.PipelineOptions : Pipeline filtering options within this job. kedro_dagster.config.models.ScheduleOptions : Schedule options referenced by this job. kedro_dagster.config.models.LoggerOptions : Logger options referenced by this job. kedro_dagster.config.models.KedroDagsterConfig : Top-level config that holds a dict of these job options.

Source Code

View on GitHub

Show/Hide source 
 973
 974
 975
 976
 977
 978
 979
 980
 981
 982
 983
 984
 985
 986
 987
 988
 989
 990
 991
 992
 993
 994
 995
 996
 997
 998
 999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
class JobOptions(BaseModel):
    """Configuration options for a Dagster job.

    Parameters
    ----------
    pipeline : PipelineOptions
        Pipeline options specifying which pipeline and nodes to run.
    executor : ExecutorOptions or str or None
        Executor options instance or string key referencing an executor.
    schedule : ScheduleOptions or str or None
        Schedule options instance or string key referencing a schedule.
    loggers : list[LoggerOptions or str] or None
        List of logger configurations (inline ``LoggerOptions``) or
        logger names (strings) to attach to the job.

    Examples
    --------
    ```yaml
    jobs:
      my_data_processing:
        pipeline:
          pipeline_name: data_processing
          node_namespaces: [price_predictor]
          tags: [test1]
        executor: multiprocessing
        schedule: daily_schedule
        loggers: [console, file_logger]
    ```

    See Also
    --------
    `kedro_dagster.config.models.PipelineOptions` :
        Pipeline filtering options within this job.
    `kedro_dagster.config.models.ScheduleOptions` :
        Schedule options referenced by this job.
    `kedro_dagster.config.models.LoggerOptions` :
        Logger options referenced by this job.
    `kedro_dagster.config.models.KedroDagsterConfig` :
        Top-level config that holds a dict of these job options.
    """

    model_config = ConfigDict(extra="forbid")

    pipeline: PipelineOptions
    executor: ExecutorOptions | str | None = None
    schedule: ScheduleOptions | str | None = None
    loggers: list[LoggerOptions | str] | None = None