Package: dazed
💫 Dazed - A Confusion Matrix PackageDazed is a little confusion matrix package designed to make your life easier.
Its key features are:support for lots of different data formats (sparse integers, sparse strings, one-hot arrays, dataframes)support for multilabel dataability to list most confused labelsability to index sample information by confused label namesprints out nicelyInstallationFor the basic installation:$pipinstalldazedTo include pandas dataframe support:$pipinstalldazed[pandas]Basic UsageTo give you an idea of why you might want to use dazed, here is a toy example
demonstrating the kind of investigation it was designed to help with. Note: I
am using sparse string labels here but dazed’s interfaces can cope with integers,
onehot encoded arrays and dataframes as well (refer to theAPI Referencefor more information).Imagine your building a machine learning model to catalogue a pet store’s
inventory (primarily cats, dogs and fish). The owner has given you an image of
each animal and you’ve trained your model and made some predictions. Your data
looks like:filenames = [
   "img0.jpg", "img1.jpg", "img2.jpg", "img3.jpg", "img4.jpg", "img5.jpg"
]
truth = ["cat", "dog", "cat", "dog", "fish", "dog"]
pred = ["cat", "dog", "dog", "cat", "fish", "cat"]In order to understand how your model is doing, you make a quick confusion
matrix:from dazed import ConfusionMatrix

cm = ConfusionMatrix.from_sparse(truth, pred, info=filenames)
print(cm)| 0 1 2     index | label
---------     -------------
0 | 1 1 0         0 |   cat
1 | 2 1 0         1 |   dog
2 | 0 0 1         2 |  fish
---------     -------------From the confusion matrix it looks like the model might be prone to thinking that
dogs are actually cats. To double check:cm.most_confused()[('dog', 'cat', 2), ('cat', 'dog', 1)]Ah yes, dogs were predicted to be cats twice and cats to be dogs
once. To try and find out what the problem might be you decide that you should
check the images. To get the appropiate images:cm.label_pair_info("dog", "cat")['img3.jpg', 'img5.jpg']Upon investigating the images you notice that both dogs are white. You
decide to go back through and label your images for animal colour.truth = [
   ["cat", "white"],
   ["dog", "brown"],
   ["cat", "brown"],
   ["dog", "white"],
   ["fish", "orange"],
   ["dog", "white"]
]
pred = [
   ["cat", "white"],
   ["dog", "brown"],
   ["dog", "brown"],
   ["cat", "white"],
   ["fish", "orange"],
   ["cat", "white"]
]
cm = ConfusionMatrix.from_sparse(
   truth, pred, info=filenames, multilabel=True
)
print(cm)| 0 1 2 3 4     index |        label
-------------     --------------------
0 | 0 0 1 0 0         0 |   cat, brown
1 | 0 1 0 0 0         1 |   cat, white
2 | 0 0 1 0 0         2 |   dog, brown
3 | 0 2 0 0 0         3 |   dog, white
4 | 0 0 0 0 1         4 | fish, orange
-------------     --------------------Hmm looks like all white dogs were miss classified as white cats.cm.most_confused()[('dog, white', 'cat, white', 2), ('cat, brown', 'dog, brown', 1)]Ah yes looks like your model might be basing much of its prediction on animal
colour, maybe time to go collect some more data.To find out more about dazed take a look at theAPI Reference.