weka.etplugin.metrics

Class Ema

java.lang.Object

weka.classifiers.evaluation.AbstractEvaluationMetric

weka.etplugin.metrics.Ema

All Implemented Interfaces:

Serializable, StandardEvaluationMetric, TechnicalInformationHandler

public class Ema
extends AbstractEvaluationMetric
implements StandardEvaluationMetric, TechnicalInformationHandler

Evaluation metric for classifiers. The Entropy Modulated Accuracy is a pessimistic estimate of the accuracy with the influence of the input distribution factored out. It is defined mathematically as the inverse of the RemainingPerplexity after the learning process \[ EMA = \frac{1}{PP_{X|Y}} = \frac{1}{2^{H_{X|Y}}} \] Its bounds are \( \frac{1}{PP_X} \leq EMA \leq 1 \), where \( PP_X \) is the ClassPerplexity.

For more information, see

Valverde-Albacete, F. J., & Peláez-Moreno, C. (2014). 100% classification accuracy considered harmful: The normalized information transfer factor explains the accuracy paradox. PLoS ONE 9(1).

NOTE: This class needs to have an associated Evaluation object to compute the metric. The Evaluation object can be set via AbstractEvaluationMetric.setBaseEvaluation(Evaluation eval).
If the package is properly installed in Weka, a reference to the object of this class associated with every Evaluation object can be obtained via the method Evaluation.getPluginMetric(String metricName), or the method Evaluation.getPluginMetrics() to get a list of all the plugin metrics associated with that Evaluation object.

Author:

Antonio Pastor

See Also:

RemainingPerplexity, Evaluation, Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class weka.classifiers.evaluation.AbstractEvaluationMetric

AbstractEvaluationMetric.UnknownStatisticException

Field Summary

Fields

Modifier and Type	Field and Description
static String	METRIC_NAME Constant string with the metric name

Constructor Summary

Constructors

Constructor and Description
Ema()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	appliesToNominalClass() Returns true.
boolean	appliesToNumericClass() Returns false.
String	getMetricDescription() Get a short description of this metric.
String	getMetricName() Get the name of this metric.
double	getStatistic(String statName) Get the value of the named statistic, should be "EMA".
List<String>	getStatisticNames() Get a list with the name of the metric.
TechnicalInformation	getTechnicalInformation() Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
boolean	statisticIsMaximisable(String statName) Returns true.
String	toSummaryString() Returns a formatted string (suitable for displaying in console or GUI output) containing this metric.
void	updateStatsForClassifier(double[] predictedDistribution, Instance instance) Not used.
void	updateStatsForPredictor(double predictedValue, Instance instance) Not used.

Methods inherited from class weka.classifiers.evaluation.AbstractEvaluationMetric

getPluginMetrics, setBaseEvaluation

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

METRIC_NAME

public static String METRIC_NAME

Constant string with the metric name

Constructor Detail

Ema

public Ema()

Method Detail

appliesToNominalClass

public boolean appliesToNominalClass()

Returns true.

Specified by:

appliesToNominalClass in class AbstractEvaluationMetric

appliesToNumericClass

public boolean appliesToNumericClass()

Returns false.

Specified by:

appliesToNumericClass in class AbstractEvaluationMetric

getMetricDescription

public String getMetricDescription()

Get a short description of this metric.

Specified by:

getMetricDescription in class AbstractEvaluationMetric

Returns:

a short description of this metric

getMetricName

public String getMetricName()

Get the name of this metric.

Specified by:

getMetricName in class AbstractEvaluationMetric

Returns:

the name of this metric

getStatistic

public double getStatistic(String statName)

Get the value of the named statistic, should be "EMA".

Specified by:

getStatistic in class AbstractEvaluationMetric

Parameters:

statName - the name of the statistic, should be "EMA"

Returns:

the computed statistic

Throws:

AbstractEvaluationMetric.UnknownStatisticException - if the statistic name is not "EMA"

getStatisticNames

public List<String> getStatisticNames()

Get a list with the name of the metric.

Specified by:

getStatisticNames in class AbstractEvaluationMetric

Returns:

the names of the metric

toSummaryString

public String toSummaryString()

Returns a formatted string (suitable for displaying in console or GUI output) containing this metric.

Specified by:

toSummaryString in interface StandardEvaluationMetric

Returns:

a formatted string containing the metric

statisticIsMaximisable

public boolean statisticIsMaximisable(String statName)

Returns true.

Overrides:

statisticIsMaximisable in class AbstractEvaluationMetric

updateStatsForClassifier

public void updateStatsForClassifier(double[] predictedDistribution,
                            Instance instance)
                              throws Exception

Not used. This method is required to conform to the StandardEvaluationMetric interface, but not implemented.

Specified by:

updateStatsForClassifier in interface StandardEvaluationMetric

Throws:

Exception

updateStatsForPredictor

public void updateStatsForPredictor(double predictedValue,
                           Instance instance)
                             throws Exception

Not used. This method is required to conform to the StandardEvaluationMetric interface, but not implemented.

Specified by:

updateStatsForPredictor in interface StandardEvaluationMetric

Throws:

Exception

getTechnicalInformation

public TechnicalInformation getTechnicalInformation()

Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.

Specified by:

getTechnicalInformation in interface TechnicalInformationHandler

Returns:

the technical information about this class