plugins.h_tqdm.ProgressBar

Provides a progress bar for Apache Hamilton execution. Must have tqdm installed to use it:

pip install apache-hamilton[tqdm] (use quotes if using zsh)

class hamilton.plugins.h_tqdm.ProgressBar(desc: str = 'Graph execution', max_node_name_width: int = 50, **kwargs)[source]

An adapter that uses tqdm to show progress bars for the graph execution.

Note: you need to have tqdm installed for this to work. If you don’t have it installed, you can install it with pip install tqdm (or pip install apache-hamilton[tqdm] – use quotes if you’re using zsh).

from hamilton.plugins import h_tqdm

dr = (
    driver.Builder()
    .with_config({})
    .with_modules(some_modules)
    .with_adapters(h_tqdm.ProgressBar(desc="DAG-NAME"))
    .build()
)
# and then when you call .execute() or .materialize() you'll get a progress bar!

__init__(desc: str = 'Graph execution', max_node_name_width: int = 50, **kwargs)[source]

Create a new Progress Bar adapter.

Parameters:

• desc – The description to show in the progress bar. E.g. DAG Name is a good choice.
• kwargs – Additional kwargs to pass to TQDM. See TQDM docs for more info.
• node_name_target_width – the target width for the node name so that the progress bar is consistent. If this is None, it will take the longest, until it hits max_node_name_width.

post_graph_execute(*, run_id: str, graph: FunctionGraph, success: bool, error: Exception | None, results: dict[str, Any] | None)

Just delegates to the interface method, passing in the right data.

post_node_execute(*, run_id: str, node_: Node, kwargs: dict[str, Any], success: bool, error: Exception | None, result: Any | None, task_id: str | None = None)

Wraps the after_execution method, providing a bridge to an external-facing API. Do not override this!

pre_graph_execute(*, run_id: str, graph: FunctionGraph, final_vars: list[str], inputs: dict[str, Any], overrides: dict[str, Any])

Implementation of the pre_graph_execute hook. This just converts the inputs to the format the user-facing hook is expecting – performing a walk of the DAG to pass in the set of nodes to execute. Delegates to the interface method.

pre_node_execute(*, run_id: str, node_: Node, kwargs: dict[str, Any], task_id: str | None = None)

Wraps the before_execution method, providing a bridge to an external-facing API. Do not override this!

run_after_graph_execution(*, success: bool = True, **future_kwargs)[source]

This is run after graph execution. This allows you to do anything you want after the graph executes, knowing the results of the execution/any errors.

Parameters:

• graph – Graph that is being executed
• results – Results of the graph execution
• error – Error that occurred, None if no error occurred
• success – Whether the graph executed successfully
• run_id – Run ID (unique in process scope) of the current run. Use this to track state.
• future_kwargs – Additional keyword arguments – this is kept for backwards compatibility

run_after_node_execution(**future_kwargs)[source]

Hook that is executed post node execution.

Parameters:

• node_name – Name of the node in question
• node_tags – Tags of the node
• node_kwargs – Keyword arguments passed to the node
• node_return_type – Return type of the node
• result – Output of the node, None if an error occurred
• error – Error that occurred, None if no error occurred
• success – Whether the node executed successfully
• task_id – The ID of the task, none if not in a task-based environment
• run_id – Run ID (unique in process scope) of the current run. Use this to track state.
• future_kwargs – Additional keyword arguments – this is kept for backwards compatibility

run_before_graph_execution(*, graph: HamiltonGraph, final_vars: list[str], inputs: dict[str, Any], overrides: dict[str, Any], execution_path: Collection[str], **future_kwargs: Any)[source]

This is run prior to graph execution. This allows you to do anything you want before the graph executes, knowing the basic information that was passed in.

Parameters:

• graph – Graph that is being executed
• final_vars – Output variables of the graph
• inputs – Input variables passed to the graph
• overrides – Overrides passed to the graph
• execution_path – Collection of nodes that will be executed – these are just the nodes (not input nodes) that will be run during the course of execution.
• run_id – Run ID (unique in process scope) of the current run. Use this to track state.
• future_kwargs – Additional keyword arguments – this is kept for backwards compatibility

run_before_node_execution(*, node_name: str, node_tags: dict[str, Any], node_kwargs: dict[str, Any], node_return_type: type, task_id: str | None, **future_kwargs: Any)[source]

Hook that is executed prior to node execution.

Parameters:

• node_name – Name of the node.
• node_tags – Tags of the node
• node_kwargs – Keyword arguments to pass to the node
• node_return_type – Return type of the node
• task_id – The ID of the task, none if not in a task-based environment
• run_id – Run ID (unique in process scope) of the current run. Use this to track state.
• node_input_types – the input types to the node and what it is expecting
• future_kwargs – Additional keyword arguments – this is kept for backwards compatibility