step_in_line.pipeline.Pipeline

class documentation

class Pipeline: (source)

Constructor: Pipeline(name, steps, schedule, generate_step_name)

View In Hierarchy

Undocumented

Method __init__ Initialize a Pipeline
Method generate_layers Create indexed sets of steps. This allows steps to be run in parallel, if they don't depend on each other
Method generate_step_functions Create Step Function workflow definition
Method get_steps Gets all steps, guaranteed to be unique.
Method local_run Runs pipeline locally, with no AWS dependency. Returns all intermediary outputs.
Method set_generate_step_name Undocumented
Instance Variable generate_step_name Undocumented
Instance Variable graph Undocumented
Instance Variable name Undocumented
Instance Variable schedule Undocumented
Instance Variable steps Undocumented
Method _check_uniqueness_of_names Undocumented
def __init__(self, name: str = '', steps: Optional[Sequence[Step]] = None, schedule: Optional[str] = None, generate_step_name: Callable[[Step], str] = _default_lambda_name): (source)

Initialize a Pipeline

Args:
name (str): The name of the pipeline. steps (Sequence[Step]): The list of the non-conditional Steps associated with the pipeline.
def generate_layers(self) -> List[List[Step]]: (source)

Create indexed sets of steps. This allows steps to be run in parallel, if they don't depend on each other

def generate_step_functions(self) -> dict: (source)

Create Step Function workflow definition

def get_steps(self) -> List[Step]: (source)

Gets all steps, guaranteed to be unique.

def local_run(self) -> List[List[Tuple[str, Any]]]: (source)

Runs pipeline locally, with no AWS dependency. Returns all intermediary outputs.

def set_generate_step_name(self, generate_step_name: Callable[[Step], str]): (source)

Undocumented

generate_step_name = (source)

Undocumented

graph = (source)

Undocumented

name = (source)

Undocumented

schedule = (source)

Undocumented

steps = (source)

Undocumented

def _check_uniqueness_of_names(self) -> bool: (source)

Undocumented