Package: polar-diagrams
Polar Diagrams for Model ComparisonManuscriptThis library is created for the following paper:"Interactive Polar Diagrams for Model Comparison"by Aleksandar Anžel, Dominik Heider, and Georges HattabPlease cite the paper as:@article{ANZEL2023107843,
title ={Interactive polar diagrams for model comparison},
journal ={Computer Methods and Programs in Biomedicine},
volume ={242},
pages ={107843},
year ={2023},
issn ={0169-2607},
doi ={https://doi.org/10.1016/j.cmpb.2023.107843},
url ={https://www.sciencedirect.com/science/article/pii/S0169260723005096},
author ={Aleksandar Anžel and Dominik Heider and Georges Hattab},
keywords ={Bioinformatics, Machine-learning, Visualization, Evaluation, Climate, Comparison, Ai, Data-visualization, Information-visualization, Predictive-analysis, Model-comparison, Climate-model-visualization, Ml-model-evaluation, Taylor-diagram, Mutual-information-diagram, Entropy, Mutual-information, Variation-of-information, Correlation, Medical-data},
abstract ={Objective
Evaluating the performance of multiple complex models, such as those found in biology, medicine, climatology, and machine learning, using conventional approaches is often challenging when using various evaluation metrics simultaneously. The traditional approach, which relies on presenting multi-model evaluation scores in the table, presents an obstacle when determining the similarities between the models and the order of performance.
Methods
By combining statistics, information theory, and data visualization, juxtaposed Taylor and Mutual Information Diagrams permit users to track and summarize the performance of one model or a collection of different models. To uncover linear and nonlinear relationships between models, users may visualize one or both charts.
Results
Our library presents the first publicly available implementation of the Mutual Information Diagram and its new interactive capabilities, as well as the first publicly available implementation of an interactive Taylor Diagram. Extensions have been implemented so that both diagrams can display temporality, multimodality, and multivariate data sets, and feature one scalar model property such as uncertainty. Our library, named polar-diagrams, supports both continuous and categorical attributes.
Conclusion
The library can be used to quickly and easily assess the performances of complex models, such as those found in machine learning, climate, or biomedical domains.}}Abstract:Objective:
Evaluating the performance of multiple complex models, such as those found in biology, medicine, climatology, and machine learning, using conventional approaches is often challenging when using various evaluation metrics simultaneously. The traditional approach, which relies on presenting multi-model evaluation scores in the table, presents an obstacle when determining the similarities between the models and the order of performance.Methods:
By combining statistics, information theory, and data visualization, juxtaposed Taylor and Mutual Information Diagrams permit users to track and summarize the performance of one model or a collection of different models. To uncover linear and nonlinear relationships between models, users may visualize one or both charts.Results:
Our library presents the first publicly available implementation of the Mutual Information Diagram and its new interactive capabilities, as well as the first publicly available implementation of an interactive Taylor Diagram.
Extensions have been implemented so that both diagrams can display temporality, multimodality, and multivariate data sets, and feature one scalar model property such as uncertainty.
Our library, named \emph{polar-diagrams}, supports both continuous and categorical attributes.Conclusion:
The library can be used to quickly and easily assess the performances of complex models, such as those found in machine learning, climate, or biomedical domains.DependenciesThe code is written in Python 3.9.15 and tested on Linux with the following libraries installed:LibraryVersionnumpy1.26.3pandas2.1.4scikit-learn1.2.2scipy1.11.4plotly5.18.0kaleido0.2.1The dependencies can also be found inrequirements.txt.DataLocationDescriptionData/contains all datasets used inSource/main.ipynb.Data/Dataset_0/contains the Anscombe's quartet data set and the Datasaurus data set.Data/Dataset_1/contains the official, automatically generated script for downloading the CMIP3 data from thehttps://esgf-node.llnl.gov/projects/cmip3/. To generate the whole data set, the user should first place itself into this directory and then run the following command from the terminalsh generate_dataset_1.sh. [1]Data/Dataset_2/contains the data set from the subsection3.2 Example 2 — Machine Learning Model Evaluationof our paper.Data/Dataset_3/contains the data set from the subsection3.3 Example 3 — Biomedical Similarity Assertionof our paper.[1]The script used for downloading theDataset_1/was generated using the tutorial found herehttps://esgf.github.io/esgf-user-support/faq.html#how-to-preserve-the-directory-structure.
Script can be automatically generated and downloaded again from herehttps://esgf-data.dkrz.de/esg-search/wget?download_structure=model&project=CMIP3&experiment=historical&ensemble=run1&variable=ts.CodeSource CodeDescriptionSource/contains all source scripts.Source/main.ipynbcontains the IPython (jupyter) notebook that demonstrates the library using multiple datasets. This notebook reproduces all of the results we presented in our paper.Source/polar_diagrams/contains the root source code directory of our library.Source/polar_diagrams/src/polar_diagrams/polar_diagrams.pycontains the source code that imports the data, modifies it, calculates statistical and information theory properties, and builds diagrams.Source/polar_diagrams/tests/test.pycontains the source code for all unit tests.InstallationStableWe recommend installing the library using pip:pipinstallpolar-diagramsUnstablegitclonehttps://github.com/AAnzel/Polar-Diagrams-for-Model-Comparison.gitcdPolar-Diagrams-for-Model-Comparison/Source/polar_diagrams/
pipinstall--editable.RunningPlease check the API documentation of our library atSource/polar_diagrams/docs/polar_diagrams.mdor the IPython (jupyter) notebook that demonstrates the library atSource/main.ipynb.LicenseLicensed under the GNU General Public License, Version 3.0 (LICENSEorhttps://www.gnu.org/licenses/gpl-3.0.en.html)ContributionAny contribution intentionally submitted for inclusion in the work by you, shall be licensed under the GNU GPLv3.