lifecycle.api.TaskGroupingHook

class hamilton.lifecycle.api.TaskGroupingHook[source]

Implement this to run something after task grouping or task expansion. This will allow you to capture information about the tasks during Parallelize/Collect blocks in dynamic DAG execution.

final post_task_expand(*, run_id: str, task_id: str, parameters: dict[str, Any])[source]

Hook that is called immediately after a task is expanded into parallelizable tasks. Note that this is only useful in dynamic execution.

Parameters:

• run_id – ID of the run, unique in scope of the driver.
• task_id – ID of the task.
• parameters – Parameters that are being passed to each of the expanded tasks.

final post_task_group(*, run_id: str, task_ids: list[str])[source]

Hook that is called immediately after a task group is created. Note that this is only useful in dynamic execution, although we reserve the right to add this back into the standard hamilton execution pattern.

Parameters:

• run_id – ID of the run, unique in scope of the driver.
• task_ids – IDs of tasks that are in the group.

abstractmethod run_after_task_expansion(*, run_id: str, task_id: str, parameters: dict[str, Any], **future_kwargs)[source]

Runs after task expansion in Parallelize/Collect blocks. This allows you to capture information about the task that was expanded.

Parameters:

• run_id – ID of the run, unique in scope of the driver.
• task_id – ID of the task that was expanded.
• parameters – Parameters that were passed to the task.
• future_kwargs – Additional keyword arguments – this is kept for backwards compatibility.

abstractmethod run_after_task_grouping(*, run_id: str, task_ids: list[str], **future_kwargs)[source]

Runs after task grouping. This allows you to capture information about which tasks were created for a given run.

Parameters:

• run_id – ID of the run, unique in scope of the driver.
• task_ids – List of tasks that were grouped together.
• future_kwargs – Additional keyword arguments – this is kept for backwards compatibility.