timeeval.params package¶

timeeval.params.ParameterConfig¶

class timeeval.params.ParameterConfig¶

Bases: ABC, Sized

Base class for algorithm hyperparameter configurations.

Currently, TimeEval supports three kinds of parameter configurations:

FixedParameters: A single parameter setting with one value for each parameter.
IndependentParameterGrid and FullParameterGrid: Parameter search using the specification of a parameter grid, where each parameter can have multiple values. Depending on the parameter grid, TimeEval will build a parameter search space and test all combinations of parameters.
BayesianParameterSearch: Parameter search using Bayesian optimization.

static defaults() → ParameterConfig¶: Returns the default parameter configuration that has only a single parameter setting with no parameters.

abstract iter(algorithm: Algorithm, dataset: Dataset) → Iterator[Params]¶

Iterate over the points in the grid.

Returns: params – Yields a params object that maps each parameter to a single value.
Return type: iterator over Params

timeeval.params.FixedParameters¶

class timeeval.params.FixedParameters(params: Mapping[str, Any])¶

Bases: ParameterConfig

Single parameters setting with one value for each.

Iterating over this grid yields the input setting as the first and only element.

Parameters: params (dict of str to Any) – The parameter setting to be evaluated, as a dictionary mapping parameters to allowed values. An empty dict signifies default parameters.

Examples

>>> from timeeval.params import FixedParameters
>>> params = {"a": 2, "b": True}
>>> list(FixedParameters(params)) == (
...    [{"a": 2, "b": True}])
True
>>> FixedParameters(params)[0] == {"a": 2, "b": True}
True

iter(algorithm: Algorithm, dataset: Dataset) → Iterator[Params]¶

Iterate over the points in the grid.

Returns: params – Yields a params object that maps each parameter to a single value.
Return type: iterator over Params

timeeval.params.FullParameterGrid¶

class timeeval.params.FullParameterGrid(param_grid: Mapping[str, Any])¶

Bases: ParameterGridConfig

Grid of parameters with a discrete number of values for each.

Iterating over this grid yields the full cartesian product of all available parameter combinations. Uses the sklearn.model_selection.ParameterGrid internally.

Parameters: param_grid (dict of str to sequence) – The parameter grid to explore, as a dictionary mapping parameters to sequences of allowed values. An empty dict signifies default parameters.

Examples

>>> from timeeval.params import FullParameterGrid
>>> params = {"a": [1, 2], "b": [True, False]}
>>> list(FullParameterGrid(params)) == (
...    [{"a": 1, "b": True}, {"a": 1, "b": False},
...     {"a": 2, "b": True}, {"a": 2, "b": False}])
True
>>> FullParameterGrid(params)[1] == {"a": 1, "b": False}
True

timeeval.params.IndependentParameterGrid¶

class timeeval.params.IndependentParameterGrid(param_grid: Mapping[str, Any], default_params: Optional[Mapping[str, Any]] = None)¶

Bases: ParameterGridConfig

Grid of parameters with a discrete number of values for each.

The parameters in the dict are considered independent and explored one after the other (no cartesian product). Uses the sklearn.model_selection.ParameterGrid internally.

Parameters

param_grid (dict of str to sequence, or sequence of such) – The parameter grid to explore, as either - a dictionary mapping parameters to sequences of allowed values, or - a sequence of dicts signifying a sequence of grids to search. An empty dict signifies default parameters.
default_params (dict of str to any values) – Default values for the parameters that are not in the current parameter grid.

Examples

>>> from timeeval.params import IndependentParameterGrid
>>> params = {"a": [1, 2], "b": [True, False]}
>>> default_params = {"a": 1, "b": True, "c": "auto"}
>>> list(IndependentParameterGrid(params)) == ([
...     {"a": 1, "b": True, "c": "auto"},
...     {"a": 2, "b": True, "c": "auto"},
...     {"a": 1, "b": True, "c": "auto"},
...     {"a": 1, "b": False, "c": "auto"}
... ])
True

timeeval.params.BayesianParameterSearch¶

class timeeval.params.BayesianParameterSearch(config: OptunaStudyConfiguration, params: Mapping[str, BaseDistribution], include_default_params: bool = False)¶

Bases: ParameterConfig

Performs Bayesian optimization using Optuna integration.

Note

Please install the Optuna package to use this class. If you use the recommended PostgreSQL storage backend, you also need to install the psycopg2 or psycopg2-binary package:

pip install optuna>=3.1.0 psycopg2

Warning

Parameter search using this class and the Optuna integration is non-deterministic. The results may vary between different runs, even if the same seed is used (e.g., for the Optuna sampler or pruner). This is because TimeEval needs to re-seed the Optuna samplers for every trial in distributed mode. This is necessary to ensure that initial random samples are different over all workers.

Parameters

config (OptunaStudyConfiguration) – Configuration for the Optuna study. Optional parameters are filled in with the default values from the global Optuna configuration.
params (Mapping[str, BaseDistribution]) – Mapping from parameter names to the corresponding Optuna distributions, such as IntDistribution, FloatDistribution, or CategoricalDistribution.

Examples

>>> from timeeval.params import BayesianParameterSearch
>>> from timeeval.metrics import RangePrAUC
>>> from timeeval.integration.optuna import OptunaStudyConfiguration
>>> from optuna.distributions import FloatDistribution, IntDistribution
>>> config = OptunaStudyConfiguration(n_trials=10, metric=RangePrAUC())
>>> distributions = {
...     "max_features": FloatDistribution(low=0.0, high=1.0, step=0.01),
...     "window_size": IntDistribution(low=5, high=1000, step=5),
... }
>>> BayesianParameterSearch(config, distributions)
<timeeval.params.bayesian.BayesianParameterSearch object at 0x7f9cdc8faf50>