PUMA
Istituto di Scienza e Tecnologie dell'Informazione     
Sebastiani F. An axiomatically derived measure for the evaluation of classification algorithms. In: ICTIR'15 - 5th ACM International Conference on the Theory of Information Retrieval (Northampton, US, 27-30 September 2015). Proceedings, pp. 11 - 20. ACM, 2015.
 
 
Abstract
(English)
We address the general problem of finding suitable evaluation measures for classification systems. To this end, we adopt an axiomatic approach, i.e., we discuss a number of properties ("axioms") that an evaluation measure for classification should arguably satisfy. We start our analysis by addressing binary classification. We show that F1, nowadays considered a standard measure for the evaluation of binary classification systems, does not comply with a number of them, and should thus be considered unsatisfactory. We go on to discuss an alternative, simple evaluation measure for binary classification, that we call K, and show that it instead satisfies all the previously proposed axioms. We thus argue that researchers and practitioners should replace F1 with K in their everyday binary classification practice. We carry on our analysis by showing that K can be smoothly extended to deal with single-label multi-class classification, cost-sensitive classification, and ordinal classification.
URL: http://dl.acm.org/citation.cfm?doid=2808194.2809449
DOI: 10.1145/2808194.2809449
Subject Evaluation measures
Information retrieval
H.3.3. Information Search and Retrieval


Icona documento 1) Download Document PDF


Icona documento Open access Icona documento Restricted Icona documento Private

 


Per ulteriori informazioni, contattare: Librarian http://puma.isti.cnr.it

Valid HTML 4.0 Transitional