Apache Hamilton (incubating) is a general-purpose framework to write dataflows using regular Python functions. At the core, each function defines a transformation and its parameters indicates its dependencies. Apache Hamilton automatically connects individual functions into a Directed Acyclic Graph (DAG) that can be executed, visualized, optimized, and reported on. Apache Hamilton also comes with a UI to visualize, catalog, and monitor your dataflows.

Apache Hamilton (incubating)

# Meet-ups

We have an active global meetup group that meets virtually once a month.

You can [sign up for the group here](https://www.meetup.com/global-hamilton-open-source-user-group-meetup/).

## Past Meet-ups

The below will be out of date. Please [see our youtube](https://www.youtube.com/@dagworks-inc) for the latest recordings on past meetups.

### July 2025

- **Topic**: Apache Transition

### December 2024

- **Topic**: Community spotlight by Jernej Frank on decorators framework

### October 2024

- **Topic**: “Building a Decisioning Engine for Data Scientists” by Sholto Armstrong

### August 2024

- **Topic**: “Finding Optimal DAGs for Machine Learning/AI/RAG projects” by Gilad Rubin
- [Recording](https://youtu.be/3LREcaewZbo?feature=shared)

### June 2024

- [Recording](https://youtu.be/SsrIIM1ed4w?feature=shared)

### April 2024

- [Recording](https://youtu.be/_-yXfnBtrlg?feature=shared)

### March 2024

- [Recording](https://youtu.be/IJByeN41xHs?feature=shared)

### February 2024

- [Recording](https://www.youtube.com/watch?v=ks672Lm0CJo)
- [Slides](https://github.com/skrawcz/talks/files/14351139/Hamilton.February.2024.Meetup.pdf)
- Community Spotlight: [Arthur Andres](https://www.linkedin.com/in/0x26res/) and [slides](https://dagworks-inc.github.io/meetups/hamilton-02202024/arthur_tradewell.html#/)

Meet-ups

KnowForge 新版本 2026.0.6 正式发布！！！

产品

资源

其他

Why use Apache Hamilton?

Install

Your First Dataflow

Learning Resources

Contributing

License

Get Started

Glossary

Functions, nodes & dataflow

Driver

Visualization

Materialization

Function modifiers

Builder

Caching

Function modifiers (Advanced)

Dynamic DAGs/Parallel Execution

UI Overview

Function Naming

Migrating to Apache Hamilton

Code Organization

Common Indices

Output Immutability

Using within your ETL System

Loading Data

Best Practices

Concepts

Jupyter notebooks

Loading data

Feature engineering

Reusing functions and nodes

Model training

LLM workflows

Data quality

Testing Apache Hamilton code

Lineage + Apache Hamilton

Scaling computation

Microservice

Extension autoloading

Wrapping the Driver

Command line interface

pre-commit hooks

User Guide

Apache Hamilton UI

Apache Hamilton VSCode

Language Server

IDE extension

FastAPI

Ibis

Streamlit

Integrations

Kedro

Dagster

LangChain

Airflow

Code Comparisons

Claude Code Plugin for Hamilton

MCP Server

Ecosystem

check_output*

config.when*

dataloader

datasaver

does

unpack_fields

extract_columns

extract_fields

inject

load_from

parameterize

parameterize_extract_columns

parameterize_frame

parameterize_sources

parameterized_subdag

parameterize_values

pipe family

resolve

save_to

subdag

schema

tag*

with_columns

Decorators

AsyncDriver

Custom Driver

Drivers

Caching logic

Data versioning

Stores

SimplePythonDataFrameGraphAdapter

SimplePythonGraphAdapter

HamiltonGraphAdapter

h_async.AsyncGraphAdapter

h_threadpool.FutureAdapter

CachingGraphAdapter

h_dask.DaskGraphAdapter

h_spark.PySparkUDFGraphAdapter

h_ray.RayGraphAdapter

h_spark.SparkKoalasGraphAdapter

GraphAdapters

lifecycle.ResultBuilder

lifecycle.LegacyResultMixin

lifecycle.api.GraphAdapter

lifecycle.NodeExecutionHook

lifecycle.api.GraphExecutionHook

lifecycle.api.EdgeConnectionHook

lifecycle.api.NodeExecutionMethod

lifecycle.api.StaticValidator

lifecycle.api.GraphConstructionHook

lifecycle.api.TaskSubmissionHook

lifecycle.api.TaskReturnHook

lifecycle.api.TaskExecutionHook

lifecycle.api.TaskGroupingHook

lifecycle.PDBDebugger

lifecycle.PrintLn

plugins.h_tqdm.ProgressBar

plugins.h_rich.RichProgressBar

plugins.h_ddog.DDOGTracer

lifecycle.FunctionInputOutputTypeChecker

plugins.h_slack.SlackNotifier

lifecycle.GracefulErrorAdapter

plugins.h_spark.SparkInputValidator

plugins.h_narhwals.NarwhalsAdapter

plugins.h_mlflow.MLFlowTracker

lifecycle.NoEdgeAndInputTypeChecking

plugins.h_openlineage.OpenLineageAdapter

Lifecycle Adapters

Generic

Numpy

Pandas

Polars

Dask

plugins.h_pyarrow.PyarrowTableResult

Custom ResultBuilder

ResultBuilders

Using Data Adapters

Data Adapters

clear_storage()

copy()

find()

import_module()

inspect()

inspect_module()

install_dependencies_string()

latest_commit()

list()

pull_module()

Dataflows

Meet-ups

Meet-ups

Past Meet-ups

July 2025

December 2024

October 2024

August 2024

June 2024

April 2024

March 2024

February 2024

评论