parameterize

Expands a single function into n, each of which correspond to a function in which the parameter value is replaced either by:

1. A specified value value()
2. The value from a specified upstream node source().

Note if you’re confused by the other @paramterize_* decorators, don’t worry, they all delegate to this base decorator.

import pandas as pd
from hamilton.function_modifiers import parameterize
from hamilton.function_modifiers import value, source

@parameterize(
    D_ELECTION_2016_shifted=dict(n_off_date=source('D_ELECTION_2016'), shift_by=value(3)),
    SOME_OUTPUT_NAME=dict(n_off_date=source('SOME_INPUT_NAME'), shift_by=value(1)),
)
def date_shifter(n_off_date: pd.Series, shift_by: int = 1) -> pd.Series:
    """{one_off_date} shifted by shift_by to create {output_name}"""
    return n_off_date.shift(shift_by)

By choosing value() or source(), you can determine the source of your dependency. Note that you can also pass documentation. If you don’t, it will use the parameterized docstring.

@parameterize(
    D_ELECTION_2016_shifted=(dict(n_off_date=source('D_ELECTION_2016'), shift_by=value(3)), "D_ELECTION_2016 shifted by 3"),
    SOME_OUTPUT_NAME=(dict(n_off_date=source('SOME_INPUT_NAME'), shift_by=value(1)),"SOME_INPUT_NAME shifted by 1")
)
def date_shifter(n_off_date: pd.Series, shift_by: int=1) -> pd.Series:
    """{one_off_date} shifted by shift_by to create {output_name}"""
    return n_off_date.shift(shift_by)

Reference Documentation

Classes to help with @parameterize (also can be used with @inject):

class hamilton.function_modifiers.ParameterizedExtract(outputs: tuple[str, ...], input_mapping: dict[str, ParametrizedDependency])[source]

Dataclass to hold inputs for @parameterize and @parameterize_extract_columns.

Parameters:

• outputs – A tuple of strings, each of which is the name of an output.
• input_mapping – A dictionary of string to ParametrizedDependency. The string is the name of the python parameter of the decorated function, and the value is a “source”/”value” which will be passed as input for that parameter to the function.

class hamilton.function_modifiers.source(dependency_on: Any)[source]

Specifies that a parameterized dependency comes from an upstream source.

This means that it comes from a node somewhere else. E.G. source(“foo”) means that it should be assigned the value that “foo” outputs.

Parameters:

dependency_on – Upstream function (i.e. node) to come from.

Returns:

An UpstreamDependency object – a signifier to the internal framework of the dependency type.

class hamilton.function_modifiers.value(literal_value: Any)[source]

Specifies that a parameterized dependency comes from a “literal” source.

E.G. value(“foo”) means that the value is actually the string value “foo”.

Parameters:

literal_value – Python literal value to use.

Returns:

A LiteralDependency object – a signifier to the internal framework of the dependency type.

class hamilton.function_modifiers.group(*dependency_args: ParametrizedDependency, **dependency_kwargs: ParametrizedDependency)[source]

Specifies that a parameterized dependency comes from a “grouped” source.

This means that it gets injected into a list of dependencies that are grouped together. E.G. dep=group(source(“foo”), source(“bar”)) for the function:

@inject(dep=group(source("foo"), source("bar")))
def f(dep: List[pd.Series]) -> pd.Series:
    return ...

Would result in dep getting foo and bar dependencies injected.

Parameters:

• dependency_args – Dependencies, list of dependencies (e.g. source(“foo”), source(“bar”))
• dependency_kwargs – Dependencies, kwarg dependencies (e.g. foo=source(“foo”))

Returns:

Parameterize documentation:

class hamilton.function_modifiers.parameterize(**parametrization: dict[str, ParametrizedDependency] | tuple[dict[str, ParametrizedDependency], str])[source]

Decorator to use to create many functions.

Expands a single function into n, each of which correspond to a function in which the parameter value is replaced either by:

1. A specified literal value, denoted value(‘literal_value’).
2. The output from a specified upstream function (i.e. node), denoted source(‘upstream_function_name’).

Note that parameterize can take the place of @parameterize_sources or @parameterize_values decorators below. In fact, they delegate to this!

Examples expressing different syntax:

@parameterize(
    # tuple of assignments (consisting of literals/upstream specifications), and docstring.
    replace_no_parameters=({}, 'fn with no parameters replaced'),
)
def no_param_function() -> Any:
    ...

@parameterize(
    # tuple of assignments (consisting of literals/upstream specifications), and docstring.
    replace_just_upstream_parameter=(
        {'upstream_source': source('foo_source')},
        'fn with upstream_parameter set to node foo'
    ),
)
def param_is_upstream_function(upstream_source: Any) -> Any:
    '''Doc string that can also be parameterized: {upstream_source}.'''
    ...

@parameterize(
    replace_just_literal_parameter={'literal_parameter': value('bar')},
)
def param_is_literal_value(literal_parameter: Any) -> Any:
    '''Doc string that can also be parameterized: {literal_parameter}.'''
    ...

@parameterize(
    replace_both_parameters={
        'upstream_parameter': source('foo_source'),
        'literal_parameter': value('bar')
    }
)
def concat(upstream_parameter: Any, literal_parameter: str) -> Any:
    '''Adding {literal_parameter} to {upstream_parameter} to create {output_name}.'''
    return upstream_parameter + literal_parameter

You also have the capability to “group” parameters, which will combine them into a list.

@parameterize(
    a_plus_b_plus_c={
        'to_concat' : group(source('a'), value('b'), source('c'))
    }
)
def concat(to_concat: List[str]) -> Any:
    '''Adding {literal_parameter} to {upstream_parameter} to create {output_name}.'''
    return sum(to_concat, '')

__init__(**parametrization: dict[str, ParametrizedDependency] | tuple[dict[str, ParametrizedDependency], str])[source]

Decorator to use to create many functions.

Parameters:

parametrization –

**kwargs with one of two things:

• a tuple of assignments (consisting of literals/upstream specifications), and docstring.
• just assignments, in which case it parametrizes the existing docstring.