Visualization Software
Visualization Software for Clasifier Evaluation:
Classifier performance evaluation typically gives rise to
vast numbers of results that are difficult to interpret. On the
one hand, a variety of different performance metrics can be
applied; and on the other hand, evaluation must be conducted
on multiple domains to get a clear view of the classifiers'
general behaviour. We present a visualization
technique that allows a user to study the results
from a domain point of view, from a classifier point of
view or from the evaluation metrics point of view.
We argue that classifier evaluation should be done on
an exploratory basis. In particular, we suggest that, rather
than pre-selecting a few metrics and domains to conduct
our evaluation on, we should use as many metrics and domains
as possible and mine the results of this study to draw
valid and relevant knowledge about the behaviour of our algorithms.
The software made available on this page will enable
such a process.
Our software tool based on WEKA and R was developed by:
- Alexandre Kouznetsov , August 2009
- Disclaimer: The tool is currently undergoing tests. If you notice any bug or have any
comments
about ways to improve the tool, please, send an e-mail to nat@site.uottawa.ca.
The tool is based on the following papers:
- "Visualizing Classifier Performance" Alaiz-Rodriguez, R., Japkowicz,
N., and Tischer, P. in the Proceedings of the 20th IEEE International
Conference on Tools for Artificial Intelligence (ICTAI'2008) Paper (in PDF format)
- "A Visualization-Based Exploratory Tool for Classifier Comparison with
respect
to Multiple Metrics and Multiple Domains", Alaiz-Rodriguez, R., Japkowicz, N., and Tischer, P. in the Proceedings of the
ECML PKDD 2008 Conference (Short Demo Paper).
- "A Projection-Based Framework for Classifier Performance Evaluation", Japkowicz, N., Sanghi, P. and Tischer, P. in the Proceedings of the ECML PKDD 2008 Conference . Paper (in PDF format).
The Tool can be downloaded from the following Website:
http://www.site.uottawa.ca/~nat/Visualization_Software/
The source code is also available at:
http://www.site.uottawa.ca/~nat/Visualization_Software/src