Apache Hamilton (incubating) is a general-purpose framework to write dataflows using regular Python functions. At the core, each function defines a transformation and its parameters indicates its dependencies. Apache Hamilton automatically connects individual functions into a Directed Acyclic Graph (DAG) that can be executed, visualized, optimized, and reported on. Apache Hamilton also comes with a UI to visualize, catalog, and monitor your dataflows.

Apache Hamilton (incubating)

# Contributing

We are open contributions big and small. See our [contributing guidelines](https://github.com/apache/hamilton/blob/main/CONTRIBUTING.md).

We also operate under a [Code of Conduct](https://www.apache.org/foundation/policies/conduct.html), and expect contributors to do the same.

Contributing

KnowForge 新版本 2026.0.6 正式发布！！！

产品

资源

其他

Why use Apache Hamilton?

Install

Your First Dataflow

Learning Resources

License

Get Started

Glossary

Functions, nodes & dataflow

Driver

Visualization

Materialization

Function modifiers

Builder

Caching

Function modifiers (Advanced)

Dynamic DAGs/Parallel Execution

UI Overview

Function Naming

Migrating to Apache Hamilton

Code Organization

Common Indices

Output Immutability

Using within your ETL System

Loading Data

Best Practices

Concepts

Jupyter notebooks

Loading data

Feature engineering

Reusing functions and nodes

Model training

LLM workflows

Data quality

Testing Apache Hamilton code

Lineage + Apache Hamilton

Scaling computation

Microservice

Extension autoloading

Wrapping the Driver

Command line interface

pre-commit hooks

User Guide

Apache Hamilton UI

Apache Hamilton VSCode

Language Server

IDE extension

FastAPI

Ibis

Streamlit

Integrations

Kedro

Dagster

LangChain

Airflow

Code Comparisons

Meet-ups

Claude Code Plugin for Hamilton

MCP Server

Ecosystem

check_output*

config.when*

dataloader

datasaver

does

unpack_fields

extract_columns

extract_fields

inject

load_from

parameterize

parameterize_extract_columns

parameterize_frame

parameterize_sources

parameterized_subdag

parameterize_values

pipe family

resolve

save_to

subdag

schema

tag*

with_columns

Decorators

AsyncDriver

Custom Driver

Drivers

Caching logic

Data versioning

Stores

SimplePythonDataFrameGraphAdapter

SimplePythonGraphAdapter

HamiltonGraphAdapter

h_async.AsyncGraphAdapter

h_threadpool.FutureAdapter

CachingGraphAdapter

h_dask.DaskGraphAdapter

h_spark.PySparkUDFGraphAdapter

h_ray.RayGraphAdapter

h_spark.SparkKoalasGraphAdapter

GraphAdapters

lifecycle.ResultBuilder

lifecycle.LegacyResultMixin

lifecycle.api.GraphAdapter

lifecycle.NodeExecutionHook

lifecycle.api.GraphExecutionHook

lifecycle.api.EdgeConnectionHook

lifecycle.api.NodeExecutionMethod

lifecycle.api.StaticValidator

lifecycle.api.GraphConstructionHook

lifecycle.api.TaskSubmissionHook

lifecycle.api.TaskReturnHook

lifecycle.api.TaskExecutionHook

lifecycle.api.TaskGroupingHook

lifecycle.PDBDebugger

lifecycle.PrintLn

plugins.h_tqdm.ProgressBar

plugins.h_rich.RichProgressBar

plugins.h_ddog.DDOGTracer

lifecycle.FunctionInputOutputTypeChecker

plugins.h_slack.SlackNotifier

lifecycle.GracefulErrorAdapter

plugins.h_spark.SparkInputValidator

plugins.h_narhwals.NarwhalsAdapter

plugins.h_mlflow.MLFlowTracker

lifecycle.NoEdgeAndInputTypeChecking

plugins.h_openlineage.OpenLineageAdapter

Lifecycle Adapters

Generic

Numpy

Pandas

Polars

Dask

plugins.h_pyarrow.PyarrowTableResult

Custom ResultBuilder

ResultBuilders

Using Data Adapters

Data Adapters

clear_storage()

copy()

find()

import_module()

inspect()

inspect_module()

install_dependencies_string()

latest_commit()

list()

pull_module()

Dataflows

Contributing

Contributing

评论