weka.etplugin.metrics

Class RemainingPerplexity

java.lang.Object

weka.classifiers.evaluation.AbstractEvaluationMetric

weka.etplugin.metrics.RemainingPerplexity

All Implemented Interfaces:

Serializable, InformationTheoreticEvaluationMetric

public class RemainingPerplexity
extends AbstractEvaluationMetric
implements InformationTheoreticEvaluationMetric

Information theoretic evaluation metric for classifiers. The Remaining Perplexity represents the residual difficulty for the classification task after the learning process \[ PP_{X|Y} = 2^{H_{X|Y}} \] Where \( H_{X|Y} \) is the entropy of the input probability distribution given the output, and measures the information that the classifier failed to learn. The minimum \( PP_{X|Y} \) is one, meaning no remaining difficulty.

For more information, see

Valverde-Albacete, F. J., & Peláez-Moreno, C. (2014). 100% classification accuracy considered harmful: The normalized information transfer factor explains the accuracy paradox. PLoS ONE 9(1).

NOTE: This class needs to have an associated Evaluation object to compute the metric. The Evaluation object can be set via AbstractEvaluationMetric.setBaseEvaluation(Evaluation eval).
If the package is properly installed in Weka, a reference to the object of this class associated with every Evaluation object can be obtained via the method Evaluation.getPluginMetric(String metricName), or the method Evaluation.getPluginMetrics() to get a list of all the plugin metrics associated with that Evaluation object.

Author:

Antonio Pastor

See Also:

Ema, Evaluation, Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class weka.classifiers.evaluation.AbstractEvaluationMetric

AbstractEvaluationMetric.UnknownStatisticException

Field Summary

Fields

Modifier and Type	Field and Description
static String	METRIC_NAME Constant string with the metric name

Constructor Summary

Constructors

Constructor and Description
RemainingPerplexity()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	appliesToNominalClass() Returns true.
boolean	appliesToNumericClass() Returns false.
String	getMetricDescription() Get a short description of this metric.
String	getMetricName() Get the name of this metric.
double	getStatistic(String statName) Get the value of the named statistic, should be "Remaining perplexity".
List<String>	getStatisticNames() Get a list with the name of the metric.
boolean	statisticIsMaximisable(String statName) Returns false.
String	toSummaryString() Returns a formatted string (suitable for displaying in console or GUI output) containing this metric.
void	updateStatsForClassifier(double[] predictedDistribution, Instance instance) Not used.
void	updateStatsForConditionalDensityEstimator(ConditionalDensityEstimator classifier, Instance classMissing, double classValue) Not used.
void	updateStatsForPredictor(double predictedValue, Instance instance) Not used.

Methods inherited from class weka.classifiers.evaluation.AbstractEvaluationMetric

getPluginMetrics, setBaseEvaluation

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

METRIC_NAME

public static String METRIC_NAME

Constant string with the metric name

Constructor Detail

RemainingPerplexity

public RemainingPerplexity()

Method Detail

toSummaryString

public String toSummaryString()

Returns a formatted string (suitable for displaying in console or GUI output) containing this metric.

Specified by:

toSummaryString in interface InformationTheoreticEvaluationMetric

Returns:

a formatted string containing the metric

getMetricName

public String getMetricName()

Get the name of this metric.

Specified by:

getMetricName in class AbstractEvaluationMetric

Returns:

the name of this metric

appliesToNominalClass

public boolean appliesToNominalClass()

Returns true.

Specified by:

appliesToNominalClass in class AbstractEvaluationMetric

appliesToNumericClass

public boolean appliesToNumericClass()

Returns false.

Specified by:

appliesToNumericClass in class AbstractEvaluationMetric

statisticIsMaximisable

public boolean statisticIsMaximisable(String statName)

Returns false.

Overrides:

statisticIsMaximisable in class AbstractEvaluationMetric

getMetricDescription

public String getMetricDescription()

Get a short description of this metric.

Specified by:

getMetricDescription in class AbstractEvaluationMetric

Returns:

a short description of this metric

getStatisticNames

public List<String> getStatisticNames()

Get a list with the name of the metric.

Specified by:

getStatisticNames in class AbstractEvaluationMetric

Returns:

the names of the metric

getStatistic

public double getStatistic(String statName)

Get the value of the named statistic, should be "Remaining perplexity".

Specified by:

getStatistic in class AbstractEvaluationMetric

Parameters:

statName - the name of the statistic, should be "Remaining perplexity"

Returns:

the computed statistic

Throws:

AbstractEvaluationMetric.UnknownStatisticException - if the statistic name is not "Remaining perplexity"

updateStatsForClassifier

public void updateStatsForClassifier(double[] predictedDistribution,
                            Instance instance)
                              throws Exception

Not used. This method is required to conform to the InformationTheoreticEvaluationMetric interface, but not implemented.

Specified by:

updateStatsForClassifier in interface InformationTheoreticEvaluationMetric

Throws:

Exception

updateStatsForPredictor

public void updateStatsForPredictor(double predictedValue,
                           Instance instance)
                             throws Exception

Not used. This method is required to conform to the InformationTheoreticEvaluationMetric interface, but not implemented.

Specified by:

updateStatsForPredictor in interface InformationTheoreticEvaluationMetric

Throws:

Exception

updateStatsForConditionalDensityEstimator

public void updateStatsForConditionalDensityEstimator(ConditionalDensityEstimator classifier,
                                             Instance classMissing,
                                             double classValue)
                                               throws Exception

Not used. This method is required to conform to the InformationTheoreticEvaluationMetric interface, but not implemented.

Specified by:

updateStatsForConditionalDensityEstimator in interface InformationTheoreticEvaluationMetric

Throws:

Exception