Paper:
ABSTRACT Semmelhack et al. (2014) have achieved high classification accuracy in distinguishing swim bouts of zebrafish using a Support Vector Machine (SVM). Convolutional Neural Networks (CNNs) have reached superior performance in various image recognition tasks over SVMs, but their learnt features are not immediately visible. Reaching better transparency helps to build trust in their classifications and makes learned features interpretable to experts. Using a recently developed technique called Deep Taylor Decomposition, we generated heatmaps to highlight input regions of high relevance for predictions. We find that our CNN makes predictions by analyzing the steadiness of the tail's trunk, which markedly differs from the manually extracted features used by Semmelhack et al. (2014). We further uncovered that the network paid attention to experimental artifacts. Removing these artifacts ensured the validity of predictions. After correction, our best CNN beats the SVM by 6.12%, achieving a classification accuracy of 96.32%. Our work thus demonstrates the utility of AI explainability for CNNs.

Review:
SUMMARY: explore the use of CNN in a binary task on images of zebrafish
It is important to note that researchers in the field of AI and deep learning are themselves aware of the fallacies of deep learning, and are striving everyday to overcome these themselves. The hype over deep learning has caused certain disdain among a section of the research community over the workings of deep neural networks. This is evident in this paper with the authors calling CNNs "black box" and the learnings of a neural network "cheating". Perhaps the authors are not aware that CNNs are hardly black boxes, their inner workings quite transparent in mathematical terms, which the submitted paper itself explores. Perhaps the authors are also not aware that the fallacies that causes CNNs to overfit on some characteristics in the input data are also present in other machine learning tools such as SVMs. Perhaps the intention of the authors is to bring more relevance to the dangers of spurious correlations, especially when the applications are critical. I hope the community can work together in improving the state of the art while improving transparency and explainability.
BACKGROUND:
Hypothesis: Prey movements in zebrafish are characterized by specific motions, that are triggered by a specific pathway involving an area called AF7.
Validation: Semmelhack et. al. (2014) removed the AF7 neuropil and observed that they failed to respond to prey stimuli
AI: the prey stimuli was a characteristic movement that an SVM was trained to detect.
Good to know that the authors will share their code.
It is not explained why the authors chose to pretrain on ImageNet, since ImageNet does not have any image classes that are comparable to the dataset the authors use. They then proceed to do a hyperparameter sweep to fine-tune on their dataset.
It is also not explained why they chose to average outputs of 500 output nodes to get two outputs, instead of simply replacing the last layer with a 2-neuron layer and finetune, as is general practice.
The level of detail in the training procedure is very helpful to reproduce the setting as well as establish a reference for any future work in this direction. This itself is a notable achievement and a good use of significant research time. Furthermore, the authors conduct one good analysis (DTD) to explain the results of their CNN. The conclusion has many repeated points from previous sections, but it is a good summary.
Given their premise about explainability in machine learning, perhaps more significance was to be given to DTD, and other methods also performed to check if the results coincide with those from DTD.