Performance indicators

Performance indicators for multi-objective RL algorithms.

We mostly rely on pymoo for the computation of axiomatic indicators (HV and IGD), but some are customly made.

morl_baselines.common.performance_indicators.cardinality(front: List[ndarray]) → float

Cardinality Metric.

Cardinality of the Pareto front approximation.

Parameters:

front – current pareto front to compute the cardinality on

Returns:

float – cardinality metric

morl_baselines.common.performance_indicators.expected_utility(front: ~typing.List[~numpy.ndarray], weights_set: ~typing.List[~numpy.ndarray], utility: ~typing.Callable = <function dot>) → float

Expected Utility Metric.

Expected utility of the policies on the PF for various weights. Similar to R-Metrics in MOO. But only needs one PF approximation. Paper: L. M. Zintgraf, T. V. Kanters, D. M. Roijers, F. A. Oliehoek, and P. Beau, “Quality Assessment of MORL Algorithms: A Utility-Based Approach,” 2015.

Parameters:

  • front – current pareto front to compute the eum on

  • weights_set – weights to use for the utility computation

  • utility – utility function to use (default: dot product)

Returns:

float – eum metric

morl_baselines.common.performance_indicators.hypervolume(ref_point: ndarray, points: List[_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]]) → float

Computes the hypervolume metric for a set of points (value vectors) and a reference point (from Pymoo).

Parameters:

  • ref_point (np.ndarray) – Reference point

  • points (List[np.ndarray]) – List of value vectors

Returns:

float – Hypervolume metric

morl_baselines.common.performance_indicators.igd(known_front: List[ndarray], current_estimate: List[ndarray]) → float

Inverted generational distance metric. Requires to know the optimal front.

Parameters:

  • known_front – known pareto front for the problem

  • current_estimate – current pareto front

Returns:

a float stating the average distance between a point in current_estimate and its nearest point in known_front

morl_baselines.common.performance_indicators.maximum_utility_loss(front: ~typing.List[~numpy.ndarray], reference_set: ~typing.List[~numpy.ndarray], weights_set: ~numpy.ndarray, utility: ~typing.Callable = <function dot>) → float

Maximum Utility Loss Metric.

Maximum utility loss of the policies on the PF for various weights. Paper: L. M. Zintgraf, T. V. Kanters, D. M. Roijers, F. A. Oliehoek, and P. Beau, “Quality Assessment of MORL Algorithms: A Utility-Based Approach,” 2015.

Parameters:

  • front – current pareto front to compute the mul on

  • reference_set – reference set (e.g. true Pareto front) to compute the mul on

  • weights_set – weights to use for the utility computation

  • utility – utility function to use (default: dot product)

Returns:

float – mul metric

morl_baselines.common.performance_indicators.sparsity(front: List[ndarray]) → float

Sparsity metric from PGMORL.

(!) This metric only considers the points from the PF identified by the algorithm, not the full objective space. Therefore, it is misleading (e.g. learning only one point is considered good) and we recommend not using it when comparing algorithms.

Basically, the sparsity is the average distance between each point in the front.

Parameters:

front – current pareto front to compute the sparsity on

Returns:

float – sparsity metric