skpro.metrics.LinearizedLogLoss

class skpro.metrics.LinearizedLogLoss(range=1, multioutput='uniform_average', multivariate=False)

Lineararized logarithmic loss for distributional predictions.

For a predictive distribution \(d\) with pdf \(p_d\) and a ground truth value \(y\), the linearized logarithmic loss is defined as \(L(y, d) := -\log p_d(y)\) if \(p_d(y) \geq r\), and \(L(y, d) := -\log p_d(r) + 1 - \frac{1}{r} p_d(r)\) otherwise, where \(r\) is the range of linearization parameter, range below.

• evaluate computes the average test sample loss.

• evaluate_by_index produces the loss sample by test data point.

• multivariate controls averaging over variables.

Parameters:

rangepositive float, optional, default=1

range of linearization, i.e., where to linearize the log-loss for values smaller than range, the log-loss is linearized

multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape

(n_outputs,), default=’uniform_average’

Defines whether and how to aggregate metric for across variables.

• If ‘uniform_average’ (default), errors are mean-averaged across variables.

• If array-like, errors are weighted averaged across variables, values as weights.

• If ‘raw_values’, does not average errors across variables, columns are retained.

multivariatebool, optional, default=False

• if True, behaves as multivariate squared loss: the score is computed for entire row, results one score per row

• if False, is univariate squared loss: the score is computed per variable marginal, results in many scores per row

Attributes:

name

Return the name of the object or estimator.

Methods

__call__(y_true, y_pred, **kwargs)	Calculate metric value using underlying metric function.
clone()	Obtain a clone of the object with same hyper-parameters and config.
clone_tags(estimator[, tag_names])	Clone tags from another object as dynamic override.
create_test_instance([parameter_set])	Construct an instance of the class, using first test parameter set.
create_test_instances_and_names([parameter_set])	Create list of all test instances and a list of names for them.
evaluate(y_true, y_pred, **kwargs)	Evaluate the metric on given inputs.
evaluate_by_index(y_true, y_pred, **kwargs)	Evaluate the metric by instance index (row).
get_class_tag(tag_name[, tag_value_default])	Get class tag value from class, with tag level inheritance from parents.
get_class_tags()	Get class tags from class, with tag level inheritance from parent classes.
get_config()	Get config flags for self.
get_param_defaults()	Get object's parameter defaults.
get_param_names([sort])	Get object's parameter names.
get_params([deep])	Get a dict of parameters values for this object.
get_tag(tag_name[, tag_value_default, ...])	Get tag value from instance, with tag level inheritance and overrides.
get_tags()	Get tags from instance, with tag level inheritance and overrides.
get_test_params()	Test parameter settings.
is_composite()	Check if the object is composed of other BaseObjects.
reset()	Reset the object to a clean post-init state.
set_config(**config_dict)	Set config flags to given values.
set_params(**params)	Set the parameters of this object.
set_random_state([random_state, deep, ...])	Set random_state pseudo-random seed parameters for self.
set_tags(**tag_dict)	Set instance level tag overrides to given values.

classmethod get_test_params()

Test parameter settings.

clone()

Obtain a clone of the object with same hyper-parameters and config.

A clone is a different object without shared references, in post-init state. This function is equivalent to returning sklearn.clone of self.

Equivalent to constructing a new instance of type(self), with parameters of self, that is, type(self)(**self.get_params(deep=False)).

If configs were set on self, the clone will also have the same configs as the original, equivalent to calling cloned_self.set_config(**self.get_config()).

Also equivalent in value to a call of self.reset, with the exception that clone returns a new object, instead of mutating self like reset.

Raises:

RuntimeError if the clone is non-conforming, due to faulty __init__.

clone_tags(estimator, tag_names=None)

Clone tags from another object as dynamic override.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

clone_tags sets dynamic tag overrides from another object, estimator.

The clone_tags method should be called only in the __init__ method of an object, during construction, or directly after construction via __init__.

The dynamic tags are set to the values of the tags in estimator, with the names specified in tag_names.

The default of tag_names writes all tags from estimator to self.

Current tag values can be inspected by get_tags or get_tag.

Parameters:

estimatorAn instance of :class:BaseObject or derived class

tag_namesstr or list of str, default = None

Names of tags to clone. The default (None) clones all tags from estimator.

Returns:

self

Reference to self.

classmethod create_test_instance(parameter_set='default')

Construct an instance of the class, using first test parameter set.

Parameters:

parameter_setstr, default=”default”

Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.

Returns:

instanceinstance of the class with default parameters

classmethod create_test_instances_and_names(parameter_set='default')

Create list of all test instances and a list of names for them.

Parameters:

parameter_setstr, default=”default”

Name of the set of test parameters to return, for use in tests. If no special parameters are defined for a value, will return “default” set.

Returns:

objslist of instances of cls

i-th instance is cls(**cls.get_test_params()[i])

nameslist of str, same length as objs

i-th element is name of i-th instance of obj in tests. The naming convention is {cls.__name__}-{i} if more than one instance, otherwise {cls.__name__}

evaluate(y_true, y_pred, **kwargs)

Evaluate the metric on given inputs.

Parameters:

y_truepd.Series, pd.DataFrame, 1D np.array, or 2D np.ndarray

Ground truth (correct) target values.

y_predreturn object of probabilistic predictition method scitype:y_pred

must have same index and columns as y_true Predicted values, i-th row is prediction for i-th row of y_true.

Returns:

lossfloat or 1-column pd.DataFrame with calculated metric value(s)

float if multioutput = “uniform_average” or multivariate = True

1-column df if multioutput = “raw_values” and metric is not multivariate metric is always averaged (arithmetic) over rows

evaluate_by_index(y_true, y_pred, **kwargs)

Evaluate the metric by instance index (row).

Parameters:

y_truepd.Series, pd.DataFrame, 1D np.array, or 2D np.ndarray

Ground truth (correct) target values.

y_predskpro BaseDistribution of same shape as y_true

Predictive distribution. Must have same index and columns as y_true.

Returns:

losspd.Series or pd.DataFrame

Calculated metric, by time point (default=jackknife pseudo-values).

pd.Series if self.multioutput="uniform_average" or array-like

• index is equal to index of y_true

• entry at index i is metric at time i, averaged over variables

pd.DataFrame if self.multioutput="raw_values"

• index and columns equal to those of y_true

• i,j-th entry is metric at time i, at variable j

classmethod get_class_tag(tag_name, tag_value_default=None)

Get class tag value from class, with tag level inheritance from parents.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_class_tag method is a class method, and retrieves the value of a tag taking into account only class-level tag values and overrides.

It returns the value of the tag with name tag_name from the object, taking into account tag overrides, in the following order of descending priority:

1. Tags set in the _tags attribute of the class.

2. Tags set in the _tags attribute of parent classes,

in order of inheritance.

Does not take into account dynamic tag overrides on instances, set via set_tags or clone_tags, that are defined on instances.

To retrieve tag values with potential instance overrides, use the get_tag method instead.

Parameters:

tag_namestr

Name of tag value.

tag_value_defaultany type

Default/fallback value if tag is not found.

Returns:

tag_value

Value of the tag_name tag in self. If not found, returns tag_value_default.

classmethod get_class_tags()

Get class tags from class, with tag level inheritance from parent classes.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_class_tags method is a class method, and retrieves the value of a tag taking into account only class-level tag values and overrides.

It returns a dictionary with keys being keys of any attribute of _tags set in the class or any of its parent classes.

Values are the corresponding tag values, with overrides in the following order of descending priority:

1. Tags set in the _tags attribute of the class.

2. Tags set in the _tags attribute of parent classes,

in order of inheritance.

Instances can override these tags depending on hyper-parameters.

To retrieve tags with potential instance overrides, use the get_tags method instead.

Does not take into account dynamic tag overrides on instances, set via set_tags or clone_tags, that are defined on instances.

For including overrides from dynamic tags, use get_tags.

Returns:

collected_tagsdict

Dictionary of tag name : tag value pairs. Collected from _tags class attribute via nested inheritance. NOT overridden by dynamic tags set by set_tags or clone_tags.

get_config()

Get config flags for self.

Configs are key-value pairs of self, typically used as transient flags for controlling behaviour.

get_config returns dynamic configs, which override the default configs.

Default configs are set in the class attribute _config of the class or its parent classes, and are overridden by dynamic configs set via set_config.

Configs are retained under clone or reset calls.

Returns:

config_dictdict

Dictionary of config name : config value pairs. Collected from _config class attribute via nested inheritance and then any overrides and new tags from _onfig_dynamic object attribute.

classmethod get_param_defaults()

Get object’s parameter defaults.

Returns:

default_dict: dict[str, Any]

Keys are all parameters of cls that have a default defined in __init__. Values are the defaults, as defined in __init__.

classmethod get_param_names(sort=True)

Get object’s parameter names.

Parameters:

sortbool, default=True

Whether to return the parameter names sorted in alphabetical order (True), or in the order they appear in the class __init__ (False).

Returns:

param_names: list[str]

List of parameter names of cls. If sort=False, in same order as they appear in the class __init__. If sort=True, alphabetically ordered.

get_params(deep=True)

Get a dict of parameters values for this object.

Parameters:

deepbool, default=True

Whether to return parameters of components.

• If True, will return a dict of parameter name : value for this object, including parameters of components (= BaseObject-valued parameters).

• If False, will return a dict of parameter name : value for this object, but not include parameters of components.

Returns:

paramsdict with str-valued keys

Dictionary of parameters, paramname : paramvalue keys-value pairs include:

• always: all parameters of this object, as via get_param_names values are parameter value for that key, of this object values are always identical to values passed at construction

• if deep=True, also contains keys/value pairs of component parameters parameters of components are indexed as [componentname]__[paramname] all parameters of componentname appear as paramname with its value

• if deep=True, also contains arbitrary levels of component recursion, e.g., [componentname]__[componentcomponentname]__[paramname], etc

get_tag(tag_name, tag_value_default=None, raise_error=True)

Get tag value from instance, with tag level inheritance and overrides.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_tag method retrieves the value of a single tag with name tag_name from the instance, taking into account tag overrides, in the following order of descending priority:

1. Tags set via set_tags or clone_tags on the instance,

at construction of the instance.

2. Tags set in the _tags attribute of the class.

3. Tags set in the _tags attribute of parent classes,

in order of inheritance.

Parameters:

tag_namestr

Name of tag to be retrieved

tag_value_defaultany type, optional; default=None

Default/fallback value if tag is not found

raise_errorbool

whether a ValueError is raised when the tag is not found

Returns:

tag_valueAny

Value of the tag_name tag in self. If not found, raises an error if raise_error is True, otherwise it returns tag_value_default.

Raises:

ValueError, if raise_error is True.

The ValueError is then raised if tag_name is not in self.get_tags().keys().

get_tags()

Get tags from instance, with tag level inheritance and overrides.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

The get_tags method returns a dictionary of tags, with keys being keys of any attribute of _tags set in the class or any of its parent classes, or tags set via set_tags or clone_tags.

Values are the corresponding tag values, with overrides in the following order of descending priority:

1. Tags set via set_tags or clone_tags on the instance,

at construction of the instance.

2. Tags set in the _tags attribute of the class.

3. Tags set in the _tags attribute of parent classes,

in order of inheritance.

Returns:

collected_tagsdict

Dictionary of tag name : tag value pairs. Collected from _tags class attribute via nested inheritance and then any overrides and new tags from _tags_dynamic object attribute.

is_composite()

Check if the object is composed of other BaseObjects.

A composite object is an object which contains objects, as parameters. Called on an instance, since this may differ by instance.

Returns:

composite: bool

Whether an object has any parameters whose values are BaseObject descendant instances.

property name

Return the name of the object or estimator.

reset()

Reset the object to a clean post-init state.

Results in setting self to the state it had directly after the constructor call, with the same hyper-parameters. Config values set by set_config are also retained.

A reset call deletes any object attributes, except:

• hyper-parameters = arguments of __init__ written to self, e.g., self.paramname where paramname is an argument of __init__

• object attributes containing double-underscores, i.e., the string “__”. For instance, an attribute named “__myattr” is retained.

• config attributes, configs are retained without change. That is, results of get_config before and after reset are equal.

Class and object methods, and class attributes are also unaffected.

Equivalent to clone, with the exception that reset mutates self instead of returning a new object.

After a self.reset() call, self is equal in value and state, to the object obtained after a constructor call``type(self)(**self.get_params(deep=False))``.

Returns:

self

Instance of class reset to a clean post-init state but retaining the current hyper-parameter values.

set_config(**config_dict)

Set config flags to given values.

Configs are key-value pairs of self, typically used as transient flags for controlling behaviour.

set_config sets dynamic configs, which override the default configs.

Default configs are set in the class attribute _config of the class or its parent classes, and are overridden by dynamic configs set via set_config.

Configs are retained under clone or reset calls.

Parameters:

config_dictdict

Dictionary of config name : config value pairs.

Returns:

selfreference to self.

Notes

Changes object state, copies configs in config_dict to self._config_dynamic.

set_params(**params)

Set the parameters of this object.

The method works on simple skbase objects as well as on composite objects. Parameter key strings <component>__<parameter> can be used for composites, i.e., objects that contain other objects, to access <parameter> in the component <component>. The string <parameter>, without <component>__, can also be used if this makes the reference unambiguous, e.g., there are no two parameters of components with the name <parameter>.

Parameters:

**paramsdict

BaseObject parameters, keys must be <component>__<parameter> strings. __ suffixes can alias full strings, if unique among get_params keys.

Returns:

selfreference to self (after parameters have been set)

set_random_state(random_state=None, deep=True, self_policy='copy')

Set random_state pseudo-random seed parameters for self.

Finds random_state named parameters via self.get_params, and sets them to integers derived from random_state via set_params. These integers are sampled from chain hashing via sample_dependent_seed, and guarantee pseudo-random independence of seeded random generators.

Applies to random_state parameters in self, depending on self_policy, and remaining component objects if and only if deep=True.

Note: calls set_params even if self does not have a random_state, or none of the components have a random_state parameter. Therefore, set_random_state will reset any scikit-base object, even those without a random_state parameter.

Parameters:

random_stateint, RandomState instance or None, default=None

Pseudo-random number generator to control the generation of the random integers. Pass int for reproducible output across multiple function calls.

deepbool, default=True

Whether to set the random state in skbase object valued parameters, i.e., component estimators.

• If False, will set only self’s random_state parameter, if exists.

• If True, will set random_state parameters in component objects as well.

self_policystr, one of {“copy”, “keep”, “new”}, default=”copy”

• “copy” : self.random_state is set to input random_state

• “keep” : self.random_state is kept as is

• “new” : self.random_state is set to a new random state,

derived from input random_state, and in general different from it

Returns:

selfreference to self

set_tags(**tag_dict)

Set instance level tag overrides to given values.

Every scikit-base compatible object has a dictionary of tags. Tags may be used to store metadata about the object, or to control behaviour of the object.

Tags are key-value pairs specific to an instance self, they are static flags that are not changed after construction of the object.

set_tags sets dynamic tag overrides to the values as specified in tag_dict, with keys being the tag name, and dict values being the value to set the tag to.

The set_tags method should be called only in the __init__ method of an object, during construction, or directly after construction via __init__.

Current tag values can be inspected by get_tags or get_tag.

Parameters:

**tag_dictdict

Dictionary of tag name: tag value pairs.

Returns:

Self

Reference to self.