API Reference

Core package

.. py:module:: ddmo

DDMO: Data-Driven Models for Optimization.

.. py:class:: LS(*, name=’ls_surrogate’, normalize=True, degree=1, ridge=0.0, lasso=0.0, n_lassos=30, n_folds=5, max_iter=10000, tol=1e-08, random_state=0)

module:

ddmo

canonical:

ddmo.models.ls.LS

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Polynomial least-squares (response surface) surrogate with optional ridge/lasso.

The basis is the full polynomial of total degree degree, including interaction terms (e.g. degree 2 in two variables gives 1, x0, x1, x0^2, x0*x1, x1^2). Non-constant basis columns are scaled to unit standard deviation so the penalties treat all terms equally, and the intercept is never penalized. The objective is::

1/2 ||y - A w||^2 + ridge/2 ||w||^2 + lasso ||w||_1

  • lasso == 0: solved with an orthogonal-factorization least-squares solver (minimum-norm solution when underdetermined and ridge == 0).

  • lasso > 0: solved by coordinate descent. The L1 penalty sets the coefficients of uninformative terms exactly to zero and, among strongly correlated terms, tends to keep one (for exact duplicates the split is arbitrary; remove them first with ddmo.CollinearityFilter). Combined with ridge this is the elastic net.

  • lasso="auto": chooses the lasso strength from a log-spaced path of n_lassos values (from the smallest value that zeroes every coefficient down to 1e-3 times that) by n_folds-fold cross-validation.

Penalties are summed over samples, so the same value is relatively weaker with more data. support_ marks the input features used by at least one nonzero term; selected_features_ lists their indices.

.. py:attribute:: MOE

module:

ddmo

alias of :py:class:~ddmo.models.moe.WeightedEnsemble

.. py:class:: RBF(*, name=’rbf_surrogate’, normalize=True, gamma=1.0, regularization=1e-10, kernel=’gaussian’, degree=None)

module:

ddmo

canonical:

ddmo.models.rbf.RBF

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Radial basis function interpolant with a polynomial tail.

The model is s(x) = sum_i w_i * phi(gamma * ||x - x_i||) + q(x) where q is a polynomial of total degree degree and the weights satisfy the usual moment conditions P^T w = 0. gamma is a shape parameter with the same meaning for every shape kernel (larger = more localized); it has no effect on linear, cubic and thin_plate. Pass gamma="auto" to pick it by Rippa’s leave-one-out cross-validation.

degree=None uses 1 for cubic/thin_plate and 0 otherwise; degree=-1 (no tail) is only allowed for positive definite kernels. regularization is added to the kernel diagonal (a smoothing term).

.. py:class:: BaseSurrogateModel(*, name=’surrogate’, normalize=True)

module:

ddmo

canonical:

ddmo.base.BaseSurrogateModel

Bases: :py:class:~abc.ABC

Abstract base template for all surrogate models in ddmo.

Subclasses implement _fit_impl and _predict_impl, which receive inputs that are already validated and (if normalize) standardized with statistics computed from the training data in fit.

.. py:method:: BaseSurrogateModel.fit(X, y)

module:

ddmo

.. py:method:: BaseSurrogateModel.predict(X)

module:

ddmo

.. py:method:: BaseSurrogateModel.predict_gradient(X)

module:

ddmo

Gradient of the prediction with respect to the inputs, shape (n_samples, n_features).

Returned in the original (unnormalized) input units.

.. py:method:: BaseSurrogateModel.score(X, y)

module:

ddmo

Coefficient of determination R^2 (higher is better).

.. py:class:: CollinearityFilter(method=’vif’, threshold=None)

module:

ddmo

canonical:

ddmo.feature_selection.CollinearityFilter

Bases: :py:class:object

Reusable filter that learns which input columns to keep and applies it to new data.

method is "vif" (default threshold 10) or "correlation" (default threshold 0.95). After fit: support_ is a boolean mask over the input columns, selected_features_ the kept indices and dropped_features_ the rest.

Example::

filt = CollinearityFilter(method="vif").fit(X)
model = Kriging().fit(filt.transform(X), y)
model.predict(filt.transform(X_new))

.. py:method:: CollinearityFilter.fit(X, y=None)

module:

ddmo

.. py:method:: CollinearityFilter.transform(X)

module:

ddmo

.. py:method:: CollinearityFilter.fit_transform(X, y=None)

module:

ddmo

.. py:class:: Kriging(*, name=’kriging_surrogate’, normalize=True, theta=None, p=2.0, nugget=1e-10, theta_bounds=(0.001, 100.0), n_restarts=5, random_state=0)

module:

ddmo

canonical:

ddmo.models.kriging.Kriging

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Ordinary kriging with an anisotropic powered-exponential correlation.

The correlation between two points is::

R(x, x') = exp(-sum_k theta_k * |x_k - x'_k| ** p)

with 0 < p <= 2 (p = 2 is the Gaussian kernel). When theta is None the per-dimension theta_k are estimated by maximizing the concentrated log-likelihood (multi-start L-BFGS-B in log10 space within theta_bounds). A scalar or per-dimension array fixes theta.

predict(X, return_std=True) also returns the kriging standard error (square root of the ordinary-kriging mean squared error), which is zero at the training points and grows away from them.

.. py:method:: Kriging.predict(X, return_std=False)

module:

ddmo

.. py:attribute:: KrigingSurrogate

module:

ddmo

alias of :py:class:~ddmo.models.kriging.Kriging

.. py:attribute:: LinearSurrogate

module:

ddmo

alias of :py:class:~ddmo.models.ls.LS

.. py:attribute:: MixtureOfExperts

module:

ddmo

alias of :py:class:~ddmo.models.moe.WeightedEnsemble

.. py:class:: ModelBundle(model, feature_names=None, target_name=None, metadata=, ddmo_version=None, created_at=None)

module:

ddmo

canonical:

ddmo.persistence.ModelBundle

Bases: :py:class:object

A fitted model together with the information needed to reuse it.

.. py:attribute:: ModelBundle.model

module:

ddmo

type:

~ddmo.base.BaseSurrogateModel

.. py:attribute:: ModelBundle.feature_names

module:

ddmo

type:

list[str] | None

value:

None

.. py:attribute:: ModelBundle.target_name

module:

ddmo

type:

str | None

value:

None

.. py:attribute:: ModelBundle.metadata

module:

ddmo

type:

dict[str, ~typing.Any]

.. py:attribute:: ModelBundle.ddmo_version

module:

ddmo

type:

str | None

value:

None

.. py:attribute:: ModelBundle.created_at

module:

ddmo

type:

str | None

value:

None

.. py:method:: ModelBundle.predict(X)

module:

ddmo

.. py:method:: ModelBundle.predict_gradient(X)

module:

ddmo

.. py:attribute:: RBFSurrogate

module:

ddmo

alias of :py:class:~ddmo.models.rbf.RBF

.. py:class:: WeightedEnsemble(*, experts=None, weights=None, n_folds=5, random_state=0, name=’ensemble’)

module:

ddmo

canonical:

ddmo.models.moe.WeightedEnsemble

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Weighted average of several surrogate models.

The weights are global (not input-dependent), so this is an ensemble rather than a gated mixture of experts. weights may be:

  • None or "uniform": equal weights;

  • "cv": weights proportional to 1 / MSE of each expert under n_folds-fold cross-validation (computed on copies of the experts);

  • a sequence of non-negative numbers, one per expert.

Each expert handles its own input normalization, so the ensemble passes raw inputs through unchanged. The fitted, normalized weights are stored in weights_.

.. py:function:: load_model(file)

module:

ddmo

Read a model written by :func:save_model. Only load files you trust.

.. py:function:: save_model(model, file, *, feature_names=None, target_name=None, metadata=None)

module:

ddmo

Write a fitted model to file (a path or a binary file object).

Base interfaces

.. py:module:: ddmo.base

.. py:class:: BaseSurrogateModel(*, name=’surrogate’, normalize=True)

module:

ddmo.base

Bases: :py:class:~abc.ABC

Abstract base template for all surrogate models in ddmo.

Subclasses implement _fit_impl and _predict_impl, which receive inputs that are already validated and (if normalize) standardized with statistics computed from the training data in fit.

.. py:method:: BaseSurrogateModel.fit(X, y)

module:

ddmo.base

.. py:method:: BaseSurrogateModel.predict(X)

module:

ddmo.base

.. py:method:: BaseSurrogateModel.predict_gradient(X)

module:

ddmo.base

Gradient of the prediction with respect to the inputs, shape (n_samples, n_features).

Returned in the original (unnormalized) input units.

.. py:method:: BaseSurrogateModel.score(X, y)

module:

ddmo.base

Coefficient of determination R^2 (higher is better).

Models

.. py:module:: ddmo.models.ls

.. py:class:: LS(*, name=’ls_surrogate’, normalize=True, degree=1, ridge=0.0, lasso=0.0, n_lassos=30, n_folds=5, max_iter=10000, tol=1e-08, random_state=0)

module:

ddmo.models.ls

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Polynomial least-squares (response surface) surrogate with optional ridge/lasso.

The basis is the full polynomial of total degree degree, including interaction terms (e.g. degree 2 in two variables gives 1, x0, x1, x0^2, x0*x1, x1^2). Non-constant basis columns are scaled to unit standard deviation so the penalties treat all terms equally, and the intercept is never penalized. The objective is::

1/2 ||y - A w||^2 + ridge/2 ||w||^2 + lasso ||w||_1

  • lasso == 0: solved with an orthogonal-factorization least-squares solver (minimum-norm solution when underdetermined and ridge == 0).

  • lasso > 0: solved by coordinate descent. The L1 penalty sets the coefficients of uninformative terms exactly to zero and, among strongly correlated terms, tends to keep one (for exact duplicates the split is arbitrary; remove them first with ddmo.CollinearityFilter). Combined with ridge this is the elastic net.

  • lasso="auto": chooses the lasso strength from a log-spaced path of n_lassos values (from the smallest value that zeroes every coefficient down to 1e-3 times that) by n_folds-fold cross-validation.

Penalties are summed over samples, so the same value is relatively weaker with more data. support_ marks the input features used by at least one nonzero term; selected_features_ lists their indices.

.. py:attribute:: LinearSurrogate

module:

ddmo.models.ls

alias of :py:class:~ddmo.models.ls.LS

.. py:module:: ddmo.models.rbf

.. py:class:: RBF(*, name=’rbf_surrogate’, normalize=True, gamma=1.0, regularization=1e-10, kernel=’gaussian’, degree=None)

module:

ddmo.models.rbf

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Radial basis function interpolant with a polynomial tail.

The model is s(x) = sum_i w_i * phi(gamma * ||x - x_i||) + q(x) where q is a polynomial of total degree degree and the weights satisfy the usual moment conditions P^T w = 0. gamma is a shape parameter with the same meaning for every shape kernel (larger = more localized); it has no effect on linear, cubic and thin_plate. Pass gamma="auto" to pick it by Rippa’s leave-one-out cross-validation.

degree=None uses 1 for cubic/thin_plate and 0 otherwise; degree=-1 (no tail) is only allowed for positive definite kernels. regularization is added to the kernel diagonal (a smoothing term).

.. py:attribute:: RBFSurrogate

module:

ddmo.models.rbf

alias of :py:class:~ddmo.models.rbf.RBF

.. py:module:: ddmo.models.kriging

.. py:class:: Kriging(*, name=’kriging_surrogate’, normalize=True, theta=None, p=2.0, nugget=1e-10, theta_bounds=(0.001, 100.0), n_restarts=5, random_state=0)

module:

ddmo.models.kriging

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Ordinary kriging with an anisotropic powered-exponential correlation.

The correlation between two points is::

R(x, x') = exp(-sum_k theta_k * |x_k - x'_k| ** p)

with 0 < p <= 2 (p = 2 is the Gaussian kernel). When theta is None the per-dimension theta_k are estimated by maximizing the concentrated log-likelihood (multi-start L-BFGS-B in log10 space within theta_bounds). A scalar or per-dimension array fixes theta.

predict(X, return_std=True) also returns the kriging standard error (square root of the ordinary-kriging mean squared error), which is zero at the training points and grows away from them.

.. py:method:: Kriging.predict(X, return_std=False)

module:

ddmo.models.kriging

.. py:attribute:: KrigingSurrogate

module:

ddmo.models.kriging

alias of :py:class:~ddmo.models.kriging.Kriging

.. py:module:: ddmo.models.moe

.. py:class:: WeightedEnsemble(*, experts=None, weights=None, n_folds=5, random_state=0, name=’ensemble’)

module:

ddmo.models.moe

Bases: :py:class:~ddmo.base.BaseSurrogateModel

Weighted average of several surrogate models.

The weights are global (not input-dependent), so this is an ensemble rather than a gated mixture of experts. weights may be:

  • None or "uniform": equal weights;

  • "cv": weights proportional to 1 / MSE of each expert under n_folds-fold cross-validation (computed on copies of the experts);

  • a sequence of non-negative numbers, one per expert.

Each expert handles its own input normalization, so the ensemble passes raw inputs through unchanged. The fitted, normalized weights are stored in weights_.

.. py:attribute:: MixtureOfExperts

module:

ddmo.models.moe

alias of :py:class:~ddmo.models.moe.WeightedEnsemble

.. py:attribute:: MOE

module:

ddmo.models.moe

alias of :py:class:~ddmo.models.moe.WeightedEnsemble

Feature selection

.. py:module:: ddmo.feature_selection

Utilities for detecting and dropping collinear or uninformative input features.

All functions take the raw input matrix X of shape (n_samples, n_features) and return indices of the features to keep, in their original order. Constant columns are always dropped.

.. py:function:: variance_inflation_factors(X)

module:

ddmo.feature_selection

Variance inflation factor of each column: 1 / (1 - R^2_j).

R^2_j is from regressing column j (with intercept) on all other columns. A value of 1 means no collinearity; values above 5-10 are usually considered problematic; exact linear dependence gives inf. Constant columns get nan.

.. py:function:: vif_filter(X, threshold=10.0)

module:

ddmo.feature_selection

Repeatedly drop the feature with the largest VIF until all VIFs are <= threshold.

Ties are broken by dropping the later column, so earlier columns are preferred.

.. py:function:: correlation_filter(X, threshold=0.95)

module:

ddmo.feature_selection

Keep a feature only if its absolute Pearson correlation with every feature already kept is <= threshold. Features are considered in column order.

This catches pairwise collinearity only; use :func:vif_filter to also catch a feature that is a combination of several others.

.. py:function:: lasso_select(X, y, lasso=’auto’, **ls_params)

module:

ddmo.feature_selection

Indices of the features kept by a lasso-penalized :class:~ddmo.models.LS fit.

Other keyword arguments (degree, ridge, n_folds, …) are passed to LS. With degree > 1 a feature is kept if it appears in any nonzero term.

.. py:class:: CollinearityFilter(method=’vif’, threshold=None)

module:

ddmo.feature_selection

Bases: :py:class:object

Reusable filter that learns which input columns to keep and applies it to new data.

method is "vif" (default threshold 10) or "correlation" (default threshold 0.95). After fit: support_ is a boolean mask over the input columns, selected_features_ the kept indices and dropped_features_ the rest.

Example::

filt = CollinearityFilter(method="vif").fit(X)
model = Kriging().fit(filt.transform(X), y)
model.predict(filt.transform(X_new))

.. py:method:: CollinearityFilter.fit(X, y=None)

module:

ddmo.feature_selection

.. py:method:: CollinearityFilter.transform(X)

module:

ddmo.feature_selection

.. py:method:: CollinearityFilter.fit_transform(X, y=None)

module:

ddmo.feature_selection

Metrics

.. py:module:: ddmo.metrics

.. py:function:: mse(y_true, y_pred)

module:

ddmo.metrics

.. py:function:: rmse(y_true, y_pred)

module:

ddmo.metrics

.. py:function:: mae(y_true, y_pred)

module:

ddmo.metrics

.. py:function:: mape(y_true, y_pred)

module:

ddmo.metrics

Mean absolute percentage error in percent.

Values where y_true == 0 are ignored to avoid division-by-zero blowups. Returns 0.0 when all targets are zero.

.. py:function:: r2(y_true, y_pred)

module:

ddmo.metrics

Coefficient of determination. Returns 1.0 for a perfect fit of constant data.

Persistence

.. py:module:: ddmo.persistence

Save fitted surrogate models to disk and load them back without retraining.

Models are stored with :mod:pickle (the approach scikit-learn recommends for persisting estimators), wrapped in a small envelope that records the ddmo version, the input/output column names and any user metadata.

.. warning:: Loading a pickle file can execute arbitrary code. Only load model files that you created yourself or that come from a source you trust.

.. py:class:: ModelBundle(model, feature_names=None, target_name=None, metadata=, ddmo_version=None, created_at=None)

module:

ddmo.persistence

Bases: :py:class:object

A fitted model together with the information needed to reuse it.

.. py:attribute:: ModelBundle.model

module:

ddmo.persistence

type:

~ddmo.base.BaseSurrogateModel

.. py:attribute:: ModelBundle.feature_names

module:

ddmo.persistence

type:

list[str] | None

value:

None

.. py:attribute:: ModelBundle.target_name

module:

ddmo.persistence

type:

str | None

value:

None

.. py:attribute:: ModelBundle.metadata

module:

ddmo.persistence

type:

dict[str, ~typing.Any]

.. py:attribute:: ModelBundle.ddmo_version

module:

ddmo.persistence

type:

str | None

value:

None

.. py:attribute:: ModelBundle.created_at

module:

ddmo.persistence

type:

str | None

value:

None

.. py:method:: ModelBundle.predict(X)

module:

ddmo.persistence

.. py:method:: ModelBundle.predict_gradient(X)

module:

ddmo.persistence

.. py:function:: save_model(model, file, *, feature_names=None, target_name=None, metadata=None)

module:

ddmo.persistence

Write a fitted model to file (a path or a binary file object).

.. py:function:: load_model(file)

module:

ddmo.persistence

Read a model written by :func:save_model. Only load files you trust.

Backend service

.. py:module:: ddmo_backend.service

.. py:class:: ModelOption(key: ‘str’, label: ‘str’)

module:

ddmo_backend.service

Bases: :py:class:object

.. py:attribute:: ModelOption.key

module:

ddmo_backend.service

type:

str

.. py:attribute:: ModelOption.label

module:

ddmo_backend.service

type:

str

.. py:function:: available_models()

module:

ddmo_backend.service

.. py:function:: decode_uploaded_csv(contents)

module:

ddmo_backend.service

.. py:function:: split_train_test(X, y, test_ratio, random_state)

module:

ddmo_backend.service

.. py:function:: build_model(model_key, params)

module:

ddmo_backend.service

.. py:function:: evaluate_model(model, X_train, y_train, X_test, y_test)

module:

ddmo_backend.service

.. py:function:: evaluate_models(model_keys, params, X_train, y_train, X_test, y_test, ranking_mode=’standard’, metric_weights=None)

module:

ddmo_backend.service