Integrations

This section showcases how Apache Hamilton integrates with popular frameworks.

• dlt

  • Extract, Transform, Load (ETL)

    • Extract
    • Transform
    • Load
    • ETL Summary

  • Extract, Load, Transform (ELT)

    • Extract & Load
    • Transform
    • ELT Summary

  • dlt materializer plugin

    • DataLoader
    • DataSaver
    • Combining both

  • Next steps

• FastAPI

  • Challenges

    • 1. Test your FastAPI application
    • 2. Document your API

  • Apache Hamilton + FastAPI

    • Example

      • Client
      • Backend dataflow with Apache Hamilton
      • Server definition with FastAPI
      • Visualize endpoints’ dataflow

    • Benefits

• Ibis

  • Standalone Ibis

    • Challenge 1 - Maintain and test large data transformations codebases
    • Challenge 2 - Orchestrate Ibis code in production

  • How Apache Hamilton complements Ibis

    • Write modular Ibis code

      • Table-level
      • Column-level

    • Orchestrate Ibis anywhere

  • How Ibis complements Apache Hamilton

    • Performance boost
    • Atomic data transformation documentation
    • Working across rows with user-defined functions (UDFs)

  • Ibis + Apache Hamilton - a natural pairing

• Streamlit

  • Challenges

    • 1. Hard to read UI and data flows.
    • 2. Cache and state management

  • Apache Hamilton + Streamlit

    • Example
    • Benefits

• dbt

• MLFlow

• Airflow

• Amazon Web Services

• Burr

• Dagster

• Dask

• Feast

• Metaflow

• Pandera

• Plotly

• Polars

• Prefect

• Ray

• Slack

• Spark

• Vaex

• Narwhals

• OpenLineage