<figure><img src="image_1.png" /></figure>

HAL
open science

# Robustness of image classification on imbalanced datasets using capsules networks

Steve ONANA, Tchuani Diane, Tinku Claude, Fippo Louis, Kouamou George

Edouard

► To cite this version:

Steve ONANA, Tchuani Diane, Tinku Claude, Fippo Louis, Kouamou George Edouard. Robustness of image classification on imbalanced datasets using capsules networks. 2024. hal-04528243

HAL Id: hal-04528243

https://inria.hal.science/hal-04528243

Preprint submitted on 5 Apr 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Robustness of image classification on imbalanced datasets using capsules networks

Steve ONANA¹, Diane TCHUANI², Claude TINKU¹, Louis FIPPO¹, George Edouard Kouamou¹

¹Université de yaoundé 1, Cameroun

²Université d'Ebolowa, Cameroun

*E-mail: onana.stevealain@gmail.com, louis.fippo@univ-yaoundel.com

## Abstract

The purpose of this study is to assess the robustness of the capsule networks for the image classification tasks on imbalanced dataset. In fact, in many real-life situations, the distribution of samples is skewed, with representatives of certain classes appearing much more frequently. This raise a problem for learning algorithms, as they will be biased in favor of the majority group. At the same time, the minority class is usually the most important, because, despite its rarity, it may carry important and useful information. In this work, we firstly design a capsule networks architecture that we apply to a medical dataset extremely imbalanced provided by the Broad Institute, then we randomly subsampled that dataset and we compare the result to CNN architectures. To gauge our model's ability to perform well on diverse datasets, we tested it on a specifically configured version of the CIFAR-10 dataset. This version has a class imbalance ratio of 0.1, meaning some classes have significantly fewer data points than others. This mimics the real-world scenario of imbalanced datasets. We then compared our model's performance on this dataset with the findings of recent research in the field. To evaluate results, we use the following metrics: Accuracy, Precision, Recall, and F1-score. The results of experiments show that capsule networks can give good results on highly imbalanced datasets (BBBC41 and BBBC042) and with small observations and can be in certain instance better than convolutional neural networks. While our Capsule network architecture achieved a recall of 0.7 on the imbalanced CIFAR10 dataset, recent studies utilizing focal loss and complement cross-entropy with convolutional neural networks (CNNs) have demonstrated superior performance. These findings highlight the impact of both model architecture and data characteristics. Our model achieved better results on datasets with more severe imbalances (1:700 and 1:15) compared to CIFAR10's milder imbalance (1:10). This underscores the importance of considering both model and dataset factors when evaluating performance on imbalanced data. In conclusion, Capsule networks show promise for imbalanced learning, but they are not a universal solution. Further research is needed to explore their effectiveness across a wider range of imbalanced data scenario.

## Keywords

Imbalanced datasets, Image classification, Capsule Networks, Convolutional Neural Networks

# I INTRODUCTION

Image classification, a cornerstone of machine learning and deep learning, plays a critical role in various applications. It involves assigning a category label to an image based on its content.
# Robustness of image classification on imbalanced datasets using capsules networks

Steve ONANA¹, Diane TCHUANI², Claude TINKU¹, Louis FIPPO¹, George Edouard Kouamou¹

¹Université de yaoundé 1, Cameroun

²Université d'Ebolowa, Cameroun

*E-mail: onana.stevealain@gmail.com, louis.fippo@univ-yaoundel.com

## Abstract

The purpose of this study is to assess the robustness of the capsule networks for the image classification tasks on imbalanced dataset. In fact, in many real-life situations, the distribution of samples is skewed, with representatives of certain classes appearing much more frequently. This raise a problem for learning algorithms, as they will be biased in favor of the majority group. At the same time, the minority class is usually the most important, because, despite its rarity, it may carry important and useful information. In this work, we firstly design a capsule networks architecture that we apply to a medical dataset extremely imbalanced provided by the Broad Institute, then we randomly subsampled that dataset and we compare the result to CNN architectures. To gauge our model's ability to perform well on diverse datasets, we tested it on a specifically configured version of the CIFAR-10 dataset. This version has a class imbalance ratio of 0.1, meaning some classes have significantly fewer data points than others. This mimics the real-world scenario of imbalanced datasets. We then compared our model's performance on this dataset with the findings of recent research in the field. To evaluate results, we use the following metrics: Accuracy, Precision, Recall, and F1-score. The results of experiments show that capsules networks can give good results on highly imbalanced datasets (BBBC41 and BBBC042) and with small observations and can be in certain instance better than convolutional neural networks. While our Capsule network architecture achieved a recall of 0.7 on the imbalanced CIFAR10 dataset, recent studies utilizing focal loss and complement cross-entropy with convolutional neural networks (CNNs) have demonstrated superior performance. These findings highlight the impact of both model architecture and data characteristics. Our model achieved better results on datasets with more severe imbalances (1:700 and 1:15) compared to CIFAR10's milder imbalance (1:10). This underscores the importance of considering both model and dataset factors when evaluating performance on imbalanced data. In conclusion, Capsule networks show promise for imbalanced learning, but they are not a universal solution. Further research is needed to explore their effectiveness across a wider range of imbalanced data scenario.

## Keywords

Imbalanced datasets, Image classification, Capsule Networks, Convolutional Neural Networks

# I INTRODUCTION

Image classification, a cornerstone of machine learning and deep learning, plays a critical role in various applications. It involves assigning a category label to an image based on its content.
However, traditional classification algorithms assume a balanced dataset, where all classes are represented equally. In reality, real-world data often exhibits class imbalance, meaning some classes appear significantly more frequently than others. This imbalance poses a challenge for learning algorithms, as they become biased towards the majority class, neglecting the potentially crucial information held by the underrepresented (minority) class. This bias is particularly problematic for online learning models trained on live streams, where the majority class is constantly encountered. To address this issue, researchers have developed various techniques for effective classification on imbalanced datasets.[20].

There are methods based on data. These methods apply changes to the dataset. Among these methods we can mention, Data augmentation(applications of matrix transformations to the training data) [5]. Sampling methods, there are several sampling methods, random oversampling, Smote, Adasyn [15, 18], random subsampling, Tomek Link [22]. Some studies also use GAN as oversampling [13] techniques often when the dataset is composed of images. In addition to these methods, which have the disadvantage of performing transformations on the dataset, There are also algorithm methods. These methods can be used by creating robust learning algorithms. These include, Ensemble Learning [6] [4], Transfer Learning [23], Weighted Loss function (focal loss [3], Maximum margin Loss [10]), Complement Cross Entropy Loss [13]. It's also possible to have hybrid methods which are the combination of all these methods. All These methods achieve great results and improve the overall performance when they were tested.

The year 2017 saw the introduction of a novel architecture: capsule networks [2]. This approach demonstrated impressive performance on image classification tasks, particularly with the balanced MNIST dataset. Capsule networks offer several advantages over traditional convolutional neural networks (CNNs). Notably, they achieve high performance with fewer parameters and possess the unique ability to preserve feature relationships through the routing agreement algorithm, replacing the pooling layer. Additionally, the margin loss function employed further strengthens this architecture. These characteristics motivated us to investigate the robustness of capsule networks when applied to imbalanced datasets. This paper examines the effectiveness of capsule networks on imbalanced datasets. While we demonstrate that capsule networks, combined with a simple data augmentation technique, can achieve significant results in certain cases, we emphasize that they are not a universal solution for all imbalanced data problems. We will evaluate the performance of capsule networks on two imbalanced datasets and compare our findings with recent research on imbalanced data management strategies. The remainder of this paper is structured as follows: Section II describes the methods used in this paper and presents the datasets. In Section III, we discuss and present the experimental settings, the environment, the experiments, and the results for this paper. Results are discussed in Section 3.3.

## II METHODS

In this section, we describe convolutional neural networks and capsule networks architectures. We describe the datasets used.

### 2.1 Convolutional neural networks

Convolution neural networks are an architecture created by [14]. That architecture was mainly designed for image classification and tried to mimic how the visual cortex works. Convolutional neural networks since 2012 with the presence of big data and power computing has become the state of the art for image classification tasks. They are made of three layers.
However, traditional classification algorithms assume a balanced dataset, where all classes are represented equally. In reality, real-world data often exhibits class imbalance, meaning some classes appear significantly more frequently than others. This imbalance poses a challenge for learning algorithms, as they become biased towards the majority class, neglecting the potentially crucial information held by the underrepresented (minority) class. This bias is particularly problematic for online learning models trained on live streams, where the majority class is constantly encountered. To address this issue, researchers have developed various techniques for effective classification on imbalanced datasets.[20].

There are methods based on data. These methods apply changes to the dataset. Among these methods we can mention, Data augmentation(applications of matrix transformations to the training data) [5]. Sampling methods, there are several sampling methods, random oversampling, Smote, Adasyn [15, 18], random subsampling, Tomek Link [22]. Some studies also use GAN as oversampling [13] techniques often when the dataset is composed of images. In addition to these methods, which have the disadvantage of performing transformations on the dataset, There are also algorithm methods. These methods can be used by creating robust learning algorithms. These include, Ensemble Learning [6] [4], Transfer Learning [23], Weighted Loss function (focal loss [3], Maximum margin Loss [10]), Complement Cross Entropy Loss [13]. It's also possible to have hybrid methods which are the combination of all these methods. All These methods achieve great results and improve the overall performance when they were tested.

The year 2017 saw the introduction of a novel architecture: capsule networks [2]. This approach demonstrated impressive performance on image classification tasks, particularly with the balanced MNIST dataset. Capsule networks offer several advantages over traditional convolutional neural networks (CNNs). Notably, they achieve high performance with fewer parameters and possess the unique ability to preserve feature relationships through the routing agreement algorithm, replacing the pooling layer. Additionally, the margin loss function employed further strengthens this architecture. These characteristics motivated us to investigate the robustness of capsule networks when applied to imbalanced datasets. This paper examines the effectiveness of capsule networks on imbalanced datasets. While we demonstrate that capsule networks, combined with a simple data augmentation technique, can achieve significant results in certain cases, we emphasize that they are not a universal solution for all imbalanced data problems. We will evaluate the performance of capsule networks on two imbalanced datasets and compare our findings with recent research on imbalanced data management strategies. The remainder of this paper is structured as follows: Section II describes the methods used in this paper and presents the datasets. In Section III, we discuss and present the experimental settings, the environment, the experiments, and the results for this paper. Results are discussed in Section 3.3.

## II METHODS

In this section, we describe convolutional neural networks and capsule networks architectures. We describe the datasets used.

### 2.1 Convolutional neural networks

Convolution neural networks are an architecture created by [14]. That architecture was mainly designed for image classification and tried to mimic how the visual cortex works. Convolutional neural networks since 2012 with the presence of big data and power computing has become the state of the art for image classification tasks. They are made of three layers.
* Convolution Layer. That layer with the pooling layer are responsible for feature extraction. Filters apply cross-correlation product to an image in order to detect important features then apply the activation function and pass the result to the pooling layer.

* Pooling Layer. That Layer is used to downsample the output of the convolution layer. We can use average or max functions to perform pooling. Convolution and pooling can be performed several times if the model needs to have more abstract features.

* Fully Connected Layer. After all the convolution and pooling tasks are performed the final output is sent to the fully connected layers for classification sake.

While Convolutional Neural Networks (CNNs) have dominated image classification tasks, their pooling layers present a limitation. Specifically, pooling can discard spatial relationships between features, hindering the networks' ability to achieve equivariance and invariance to rotations. To address this limitation, Capsule Networks were introduced. In this study, we compare these two approaches. We utilize three CNN architectures: a baseline model, and two pre-trained models, DenseNet121 and InceptionV3. These pre-trained models achieved success in image classification on the ImageNet dataset. By employing these architectures, we aim to comprehensively evaluate Capsule Networks against CNNs on a medical dataset.

## 2.2 Capsule Networks

A capsule networks is a type of artificial neural networks created by [2] to address several limitations of convolutional neural networks(maintain hierarchical relations). A capsule according to [2] is a group of neurons that can be represented as an abstract vector. That vector represents the instantiation parameters(hue, width, height, texture, velocity...) of a specific type of entity such as an object or an object part. The length(normal) of that vector represents the probability that the entity exists and its orientation represents the instantiation parameters. So capsule networks is made of convolution layer to extract features and capsules layers(composed of capsules). there must be at least two capsules layers(two levels). Active capsules at one level make predictions, via transformation matrices1, for the instantiation parameters of higher-level capsules. And the active layer of the higher level capsules are chosen by the routing algorithm.

$$v_j = \text{squash}\left(\sum_i c_{ij} w_{ij} * u_i\right) \qquad (1)$$

The squash function is a mathematical function or a nonlinear activation function that takes as input a vector and outputs the norm of that vector bounded between 0 and 1. This norm is the probability that the entity exists.

$v_j$: is the output of the capsule j

$w_{ij}$: is the weight matrix between the capsule i and the capsule j. The values of the elements of this matrix are obtained by backpropagation.

$u_i$: is the output of the capsule j, $c_{ij}$: are coupling coefficients that are determined by the iterative dynamic routing process. They represent the strength of the coupling between two capsules

## 2.3 Dynamic routing Algorithm

The fact that capsule networks can preserve the hierarchy between part and entity is due to the routing agreement algorithm. It's an iterative algorithm that takes as arguments the number of iterations. The capsule level and the prediction vector, as shown in the following figure.
* Convolution Layer. That layer with the pooling layer are responsible for feature extraction. Filters apply cross-correlation product to an image in order to detect important features then apply the activation function and pass the result to the pooling layer.

* Pooling Layer. That Layer is used to downsample the output of the convolution layer. We can use average or max functions to perform pooling. Convolution and pooling can be performed several times if the model needs to have more abstract features.

* Fully Connected Layer. After all the convolution and pooling tasks are performed the final output is sent to the fully connected layers for classification sake.

While Convolutional Neural Networks (CNNs) have dominated image classification tasks, their pooling layers present a limitation. Specifically, pooling can discard spatial relationships between features, hindering the networks' ability to achieve equivariance and invariance to rotations. To address this limitation, Capsule Networks were introduced. In this study, we compare these two approaches. We utilize three CNN architectures: a baseline model, and two pre-trained models, DenseNet121 and InceptionV3. These pre-trained models achieved success in image classification on the ImageNet dataset. By employing these architectures, we aim to comprehensively evaluate Capsule Networks against CNNs on a medical dataset.

## 2.2 Capsule Networks

A capsule networks is a type of artificial neural networks created by [2] to address several limitations of convolutional neural networks(maintain hierarchical relations). A capsule according to [2] is a group of neurons that can be represented as an abstract vector. That vector represents the instantiation parameters(hue, width, height, texture, velocity...) of a specific type of entity such as an object or an object part. The length(normal) of that vector represents the probability that the entity exists and its orientation represents the instantiation parameters. So capsule networks is made of convolution layer to extract features and capsules layers(composed of capsules). there must be at least two capsules layers(two levels). Active capsules at one level make predictions, via transformation matrices1, for the instantiation parameters of higher-level capsules. And the active layer of the higher level capsules are chosen by the routing algorithm.

$$v_j = \text{squash}\left(\sum_i c_{ij} w_{ij} * u_i\right) \qquad (1)$$

The squash function is a mathematical function or a nonlinear activation function that takes as input a vector and outputs the norm of that vector bounded between 0 and 1. This norm is the probability that the entity exists.

$v_j$: is the output of the capsule j

$w_{ij}$: is the weight matrix between the capsule i and the capsule j. The values of the elements of this matrix are obtained by backpropagation.

$u_i$: is the output of the capsule j, $c_{ij}$: are coupling coefficients that are determined by the iterative dynamic routing process. They represent the strength of the coupling between two capsules

## 2.3 Dynamic routing Algorithm

The fact that capsule networks can preserve the hierarchy between part and entity is due to the routing agreement algorithm. It's an iterative algorithm that takes as arguments the number of iterations. The capsule level and the prediction vector, as shown in the following figure.
<figure><img src="image_1.png" /><figcaption>Figure 1: routing Agreement algorithm
[2]</figcaption></figure>

## 2.4 Architecture of capsule networks and loss function

Capsules networks architecture can be divided into two big parts:

* Encoder. The main role of the Encoder is the classification and is made of:

- Convolution layers that are used for feature extraction. We can use 2 or more convolution layers. In this study we use five convolution layers inspired by the work of [7]. The more we have convolution layers the more the features are diverse.

- Capsules layers. We just use two capsule layers for this study. A primary capsules layers which will take the output after the final convolution layer and transform the neurons in capsules. The second layer of capsules is used for classification. The output of that layer will be obtained after the routing algorithm.

* The decoder is composed of fully connected neural networks layers. The decoder has two roles. The first role is the reconstruction role that role gives the autoencoder function to capsule networks. the second role is the regularization function that gives the architecture the possibility to avoid overfitting. For this study the reconstruction wasn't relevant we get the best result without the decoder part.

<figure><img src="image_2.png" /><figcaption>Figure 2: Encoder architecture</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Figure 1: routing Agreement algorithm
[2]</figcaption></figure>

## 2.4 Architecture of capsule networks and loss function

Capsules networks architecture can be divided into two big parts:

* Encoder. The main role of the Encoder is the classification and is made of:

- Convolution layers that are used for feature extraction. We can use 2 or more convolution layers. In this study we use five convolution layers inspired by the work of [7]. The more we have convolution layers the more the features are diverse.

- Capsules layers. We just use two capsule layers for this study. A primary capsules layers which will take the output after the final convolution layer and transform the neurons in capsules. The second layer of capsules is used for classification. The output of that layer will be obtained after the routing algorithm.

* The decoder is composed of fully connected neural networks layers. The decoder has two roles. The first role is the reconstruction role that role gives the autoencoder function to capsule networks. the second role is the regularization function that gives the architecture the possibility to avoid overfitting. For this study the reconstruction wasn't relevant we get the best result without the decoder part.

<figure><img src="image_4.png" /><figcaption>Figure 2: Encoder architecture</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Figure 3: Decoder Architecture
[2]</figcaption></figure>

The loss function of the capsule layers depends on the facts that we use the decoder or not in our model.

The general formula of the total loss

$$L = \sum_{k} L_{k} = T_{k}max(0, m^{+} - \|V_{c}\|)^{2} + \lambda(1 - T_{k})max(0, \|V_{c}\| - m^{-})^{2} + \alpha.mse(T_{k}, T_{rk}) \quad (2)$$

$V_c$: output of the predicted capsule

$T_k$: label

$m^+$: upper bound

$m^-$: lower bound

$T_{rk}$: label's reconstruction

$\alpha$: reconstruction coefficient

This study on the assessment of capsules networks on imbalanced datasets is motivated by the fact that capsules networks have the following properties.

* Robustness to Noise and occlusions: Medical images often contain noise and occlusions, which can make it difficult for traditional convolutional neural networks to accurately identify objects and lesions.

* Ability to capture Spatial relationships: This can be helpful to solve the Picasso problem which is a difficult issue for traditional convolutional neural networks. That Ability is also important to learn hierarchical representations

* Invariance to transformation: Capsules networks are invariant to rotation and translation and have the equivariance property. Traditional convolutional neural networks are not robust against rotation transformation.

* Interpretability: Capsule networks are more interpretable than traditional convolutional neural networks because they are based on vectors that encode the presence, orientation, and position of features.

## 2.5 Datasets

The first dataset is a medical dataset that was created for a study by [1]. The dataset consists of two classes of uninfected cells (RBCs and leukocytes) and four classes of infected cells (gametocytes, rings, trophozoites, and schizonts). Annotators were permitted to mark some cells as difficult if not clearly in one of the cell classes. The data had a heavy imbalance towards
<figure><img src="image_5.png" /><figcaption>Figure 3: Decoder Architecture
[2]</figcaption></figure>

The loss function of the capsule layers depends on the facts that we use the decoder or not in our model.

The general formula of the total loss

$$L = \sum_{k} L_{k} = T_{k}max(0, m^{+} - \|V_{c}\|)^{2} + \lambda(1 - T_{k})max(0, \|V_{c}\| - m^{-})^{2} + \alpha.mse(T_{k}, T_{rk}) \quad (2)$$

$V_c$: output of the predicted capsule

$T_k$: label

$m^+$: upper bound

$m^-$: lower bound

$T_{rk}$: label's reconstruction

$\alpha$: reconstruction coefficient

This study on the assessment of capsules networks on imbalanced datasets is motivated by the fact that capsules networks have the following properties.

* Robustness to Noise and occlusions: Medical images often contain noise and occlusions, which can make it difficult for traditional convolutional neural networks to accurately identify objects and lesions.

* Ability to capture Spatial relationships: This can be helpful to solve the Picasso problem which is a difficult issue for traditional convolutional neural networks. That Ability is also important to learn hierarchical representations

* Invariance to transformation: Capsules networks are invariant to rotation and translation and have the equivariance property. Traditional convolutional neural networks are not robust against rotation transformation.

* Interpretability: Capsule networks are more interpretable than traditional convolutional neural networks because they are based on vectors that encode the presence, orientation, and position of features.

## 2.5 Datasets

The first dataset is a medical dataset that was created for a study by [1]. The dataset consists of two classes of uninfected cells (RBCs and leukocytes) and four classes of infected cells (gametocytes, rings, trophozoites, and schizonts). Annotators were permitted to mark some cells as difficult if not clearly in one of the cell classes. The data had a heavy imbalance towards
uninfected RBCs versus uninfected leukocytes and infected cells, making up over 95% of all cells. This dataset has approximately 73000 elements.

A class label and set of bounding box coordinates were given for each cell. For all data sets, infected cells were given a class label by Stefanie Lopes, a malaria researcher at the Dr. Heitor Vieira Dourado Tropical Medicine Foundation hospital, indicating the stage of development or marked as difficult.

The segmentation process is not relevant to this study. For this study we will call the initial dataset BBBC041, that dataset has an imbalance ratio of 1:700. We will derive a new dataset from the initial dataset by applying the random subsampling(only on the majority class) in order to reduce the imbalance ratio. We will call the subsampled dataset BBBC042. The dataset BBBC042 will have 3800 elements with an imbalanced ratio of 1:15. We obtain the BBBC042 dataset by removing randomly images of the majority class until we get our desired number of observations.

In order to see if our model can work on some type of balanced medical networks and validate our intuition that capsules networks performs well on medical either the dataset is balanced or imbalanced we use a balanced dataset for malaria that dataset has 2 classes.

The distribution of the two datasets is shown in the following figures:

<figure><img src="image_4.png" /><figcaption>Figure 4: box plot of the dataset BBBC042 with an imbalance ratio 1:15</figcaption></figure>
uninfected RBCs versus uninfected leukocytes and infected cells, making up over 95% of all cells. This dataset has approximately 73000 elements.

A class label and set of bounding box coordinates were given for each cell. For all data sets, infected cells were given a class label by Stefanie Lopes, a malaria researcher at the Dr. Heitor Vieira Dourado Tropical Medicine Foundation hospital, indicating the stage of development or marked as difficult.

The segmentation process is not relevant to this study. For this study we will call the initial dataset BBBC041, that dataset has an imbalance ratio of 1:700. We will derive a new dataset from the initial dataset by applying the random subsampling(only on the majority class) in order to reduce the imbalance ratio. We will call the subsampled dataset BBBC042. The dataset BBBC042 will have 3800 elements with an imbalanced ratio of 1:15. We obtain the BBBC042 dataset by removing randomly images of the majority class until we get our desired number of observations.

In order to see if our model can work on some type of balanced medical networks and validate our intuition that capsules networks performs well on medical either the dataset is balanced or imbalanced we use a balanced dataset for malaria that dataset has 2 classes.

The distribution of the two datasets is shown in the following figures:

<figure><img src="image_6.png" /><figcaption>Figure 4: box plot of the dataset BBBC042 with an imbalance ratio 1:15</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Figure 5: box plot of the dataset BBBC041 with an imbalance ratio 1:700</figcaption></figure>

To benchmark our architecture against recent approaches, we will employ a second dataset: the CIFAR-10 long-tail dataset with a 1:10 class imbalance ratio[21]. This dataset consists of ten classes, with each observation being a 32x32x3 image. Additionally, we will leverage the findings from the study [11] on balanced CIFAR-10 utilized with capsule networks.

<figure><img src="image_6.png" /><figcaption>Figure 6: Bar plot of CIFAR10 long tail imbalance ratio 1:10</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Figure 5: box plot of the dataset BBBC041 with an imbalance ratio 1:700</figcaption></figure>

To benchmark our architecture against recent approaches, we will employ a second dataset: the CIFAR-10 long-tail dataset with a 1:10 class imbalance ratio[21]. This dataset consists of ten classes, with each observation being a 32x32x3 image. Additionally, we will leverage the findings from the study [11] on balanced CIFAR-10 utilized with capsule networks.

<figure><img src="image_8.png" /><figcaption>Figure 6: Bar plot of CIFAR10 long tail imbalance ratio 1:10</figcaption></figure>
We evaluated our capsule network’s performance on a balanced medical dataset from NIH [8] to validate our hypothesis that it generalizes well to both balanced and imbalanced medical data. Specifically, we employed a balanced malaria dataset with two classes for this initial assessment.

<figure><img src="image_7.png" /><figcaption>Figure 7: Bar plot of NIH dataset</figcaption></figure>

# III EXPERIMENTS & RESULTS

This section first briefly overviews the experimental setup and implementation details and then presents experimental results per each imbalanced image dataset.

## 3.1 Training process

* We load the datasets from the disk.

* We divide the initial and the subsampled datasets into three parts training/validation/test. To avoid recollection(overfitting) of the model thus we will have a good evaluation of the model.

* We train our models without with data augmentation. The data augmentation methods used are random flip and random rotation

* We use Capsule networks with and without the decoder on the first dataset and on the subsampled dataset.

* We use pre-trained convolutional neural networks(transfer learning) on the initial dataset and on the subsampled dataset.

Because we use an imbalanced dataset Accuracy used alone as a metric is not relevant because it's a grouping metric. The majority class will dictate the value of the Accuracy. So we will have to use recall, precision, and F1-score to have a good evaluation of our model. In order to compare with other works for the second dataset the only metric used is the recall.

## 3.2 Experiments

For the experiments, we use the framework TensorFlow 2.11 as a backend of Keras and the programming language Python3. We take inspiration from the following GitHub repositories [17] and [16] to write our codes. We also use the examples available on the TensorFlow website. For hardware, we use Google Colab and a personal computer.

Our first model composed of capsule networks applied to the datasets BBBC041 and BBBC042 has the following hyperparameters:
We evaluated our capsule network’s performance on a balanced medical dataset from NIH [8] to validate our hypothesis that it generalizes well to both balanced and imbalanced medical data. Specifically, we employed a balanced malaria dataset with two classes for this initial assessment.

<figure><img src="image_9.png" /><figcaption>Figure 7: Bar plot of NIH dataset</figcaption></figure>

# III EXPERIMENTS & RESULTS

This section first briefly overviews the experimental setup and implementation details and then presents experimental results per each imbalanced image dataset.

## 3.1 Training process

* We load the datasets from the disk.
* We divide the initial and the subsampled datasets into three parts training/validation/test. To avoid recollection(overfitting) of the model thus we will have a good evaluation of the model.
* We train our models without data augmentation. The data augmentation methods used are random flip and random rotation
* We use Capsule networks with and without the decoder on the first dataset and on the subsampled dataset.
* We use pre-trained convolutional neural networks(transfer learning) on the initial dataset and on the subsampled dataset.

Because we use an imbalanced dataset Accuracy used alone as a metric is not relevant because it's a grouping metric. The majority class will dictate the value of the Accuracy. So we will have to use recall, precision, and F1-score to have a good evaluation of our model. In order to compare with other works for the second dataset the only metric used is the recall.

## 3.2 Experiments

For the experiments, we use the framework TensorFlow 2.11 as a backend of Keras and the programming language Python3. We take inspiration from the following GitHub repositories [17] and [16] to write our codes. We also use the examples available on the TensorFlow website. For hardware, we use Google Colab and a personal computer.

Our first model composed of capsule networks applied to the datasets BBBC041 and BBBC042 has the following hyperparameters:
* Image size: (96, 96, 3).

* Convolution layers kernel size: 5

* Convolution layers number of kernel: 3 for the first convolution layer and 64 for the remainder

* padding is valid

* stride: 1 for the first convolution layer 2 for the remainder

* dropout parameter is 0.4

* batch normalization layer

* The optimizer used is ADAM

* The Learning rate 0.0001

* The batch size is 64

* The number of epochs is 50.

* The number of iterations of the dynamic routing algorithm is 4.

Our second model composed of capsules networks applied to the CIFAR10 long trail dataset has the following hyperparameters:

* Image size: (96, 96, 3).

* Convolution layers kernel size: 5

* Convolution layers number of kernel: 3 for the first convolution layer and 64 for the remainder

* padding is valid

* stride: 1 for the first convolution layer 2 for the remainder

* dropout parameter is 0.4

* batch normalization layer

* The optimizer used is ADAM

* The Learning rate 0.001

* The batch size is 256

* The number of epochs is 300.

* The number of iterations of the dynamic routing algorithm is 4.

These hyperparameters have been selected based on the trade-off between the performances, the computing power, and the learning time.

In order to see how our model perform we generate confusion matrix from which we can derive our metrics. Confusion matrix results come from the test set.
* Image size: (96, 96, 3).

* Convolution layers kernel size: 5

* Convolution layers number of kernel: 3 for the first convolution layer and 64 for the remainder

* padding is valid

* stride: 1 for the first convolution layer 2 for the remainder

* dropout parameter is 0.4

* batch normalization layer

* The optimizer used is ADAM

* The Learning rate 0.0001

* The batch size is 64

* The number of epochs is 50.

* The number of iterations of the dynamic routing algorithm is 4.

Our second model composed of capsules networks applied to the CIFAR10 long trail dataset has the following hyperparameters:

* Image size: (96, 96, 3).

* Convolution layers kernel size: 5

* Convolution layers number of kernel: 3 for the first convolution layer and 64 for the remainder

* padding is valid

* stride: 1 for the first convolution layer 2 for the remainder

* dropout parameter is 0.4

* batch normalization layer

* The optimizer used is ADAM

* The Learning rate 0.001

* The batch size is 256

* The number of epochs is 300.

* The number of iterations of the dynamic routing algorithm is 4.

These hyperparameters have been selected based on the trade-off between the performances, the computing power, and the learning time.

In order to see how our model perform we generate confusion matrix from which we can derive our metrics. Confusion matrix results come from the test set.
<figure><img src="image_8.png" /><figcaption>Figure 8: Confusion Matrix of the dataset BBBC041</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 9: Confusion matrix of subsampled dataset BBBC042</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Figure 8: Confusion Matrix of the dataset BBBC041</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 9: Confusion matrix of subsampled dataset BBBC042</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Figure 10: Confusion Matrix of NIH balanced dataset</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 11: Confusion Matrix of CIFAR10 long trail dataset</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>Figure 10: Confusion Matrix of NIH balanced dataset</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure 11: Confusion Matrix of CIFAR10 long trail dataset</figcaption></figure>
<table><caption>Table 1: Initial dataset: BBBC041. The naming convention for capsXX. The first number represents the usage of the decoder if 1 we use that augmentation if 0 we don't use the decoder and the second number represents the usage of the data augmentation.</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Accuracy</th><th>Precision</th><th>Recall</th><th>F1score</th></tr></thead><tbody><tr><td>BBBC041</td><td>DENSENET</td><td>-</td><td>yes</td><td>0.98</td><td>0.36</td><td>0.34</td><td>0.24</td></tr><tr><td>BBBC041</td><td>INCEPTION</td><td>-</td><td>yes</td><td>0.99</td><td>0.41</td><td>0.39</td><td>0.39</td></tr><tr><td><strong>BBBC041</strong></td><td><strong>Caps01</strong></td><td><strong>no</strong></td><td><strong>yes</strong></td><td><strong>0.99</strong></td><td><strong>0.82</strong></td><td><strong>0.75</strong></td><td><strong>0.77</strong></td></tr><tr><td>BBBC041</td><td>Caps11</td><td>yes</td><td>yes</td><td>0.99</td><td>0.78</td><td>0.77</td><td>0.77</td></tr><tr><td>BBBC041</td><td>Caps00</td><td>no</td><td>no</td><td>0.99</td><td>0.71</td><td>0.60</td><td>0.63</td></tr><tr><td>BBBC041</td><td>CNN</td><td>-</td><td>yes</td><td>0.99</td><td>0.72</td><td>0.66</td><td>0.68</td></tr></tbody></table>

<table><caption>Table 2: Subsampled dataset: BBBC042</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Accuracy</th><th>Precision</th><th>Recall</th><th>F1score</th></tr></thead><tbody><tr><td>BBBC042</td><td>INCEPTION</td><td>-</td><td>yes</td><td>0.86</td><td>0.66</td><td>0.47</td><td>0.48</td></tr><tr><td>BBBC042</td><td>Caps11</td><td>yes</td><td>yes</td><td>0.83</td><td>0.69</td><td>0.64</td><td>0.65</td></tr><tr><td><strong>BBBC042</strong></td><td><strong>Caps01</strong></td><td><strong>no</strong></td><td><strong>yes</strong></td><td><strong>0.94</strong></td><td><strong>0.87</strong></td><td><strong>0.85</strong></td><td><strong>0.85</strong></td></tr></tbody></table>

<table><caption>Table 3: this table compare the result obtained by our architecture to the results obtained with complement cross entropy loss and focal loss. We can see that our architecture doesn't perform better than those architectures with that dataset</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Recall</th></tr></thead><tbody><tr><td><strong>CIFAR10-LT</strong></td><td><strong>Caps01</strong></td><td><strong>no</strong></td><td><strong>yes</strong></td><td><strong>0.7</strong></td></tr><tr><td>CIFAR10-LT</td><td>Resnet34+CCE [19]</td><td>-</td><td>-</td><td>0.8837</td></tr><tr><td>CIFAR10-LT</td><td>Focal Loss [19]</td><td>-</td><td>-</td><td>0.8616</td></tr></tbody></table>

<table><caption>Table 4: this table shows how capsule networks behave on balanced dataset. The medical dataset NIH achieves great results meanwhile balanced CIFAR10 is under 0.7.</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Recall</th></tr></thead><tbody><tr><td>NIH</td><td>Caps01</td><td>no</td><td>yes</td><td>0.95</td></tr><tr><td>CIFAR10</td><td>CapsNet [19]</td><td>yes</td><td>yes</td><td>0.68</td></tr></tbody></table>

## 3.3 Discussions

Our experiments demonstrated the effectiveness of capsule networks for handling highly imbalanced datasets (Table 1, line 3). Capsule networks consistently outperformed convolutional networks, regardless of decoder usage. This was true for both the original BBBC041 dataset and the subsampled BBBC042 dataset (Tables 1 & 2, lines 3 & 4 and lines 2 & 3). Interestingly, capsule networks achieved better performance on the smaller dataset, sacrificing some accuracy but improving other metrics. This suggests their suitability for limited data scenarios.

Data augmentation's importance is evident when comparing lines 4 and 5 of Table 1. Without augmentation, performance suffered. However, our capsule architecture applied to the CIFAR-10 long tail dataset yielded results surpassed by other architectures (Table 3). While the CIFAR-10 results are not insignificant from a broader perspective, they fall short of recent benchmarks for this dataset. Our findings, as shown in Table 4, suggest that Capsule Networks may be more suited for specific datasets. While performance on the balanced CIFAR-10 dataset was modest, they achieved excellent results on the NIH medical dataset. This indicates that Capsule Networks might not generalize well to all balanced datasets but have the potential to excel.
<table><caption>Table 1: Initial dataset: BBBC041. The naming convention for capsXX. The first number represents the usage of the decoder if 1 we use that augmentation if 0 we don't use the decoder and the second number represents the usage of the data augmentation.</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Accuracy</th><th>Precision</th><th>Recall</th><th>F1score</th></tr></thead><tbody><tr><td>BBBC041</td><td>DENSENET</td><td>-</td><td>yes</td><td>0.98</td><td>0.36</td><td>0.34</td><td>0.24</td></tr><tr><td>BBBC041</td><td>INCEPTION</td><td>-</td><td>yes</td><td>0.99</td><td>0.41</td><td>0.39</td><td>0.39</td></tr><tr><td><strong>BBBC041</strong></td><td><strong>Caps01</strong></td><td><strong>no</strong></td><td><strong>yes</strong></td><td><strong>0.99</strong></td><td><strong>0.82</strong></td><td><strong>0.75</strong></td><td><strong>0.77</strong></td></tr><tr><td>BBBC041</td><td>Caps11</td><td>yes</td><td>yes</td><td>0.99</td><td>0.78</td><td>0.77</td><td>0.77</td></tr><tr><td>BBBC041</td><td>Caps00</td><td>no</td><td>no</td><td>0.99</td><td>0.71</td><td>0.60</td><td>0.63</td></tr><tr><td>BBBC041</td><td>CNN</td><td>-</td><td>yes</td><td>0.99</td><td>0.72</td><td>0.66</td><td>0.68</td></tr></tbody></table>

<table><caption>Table 2: Subsampled dataset: BBBC042</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Accuracy</th><th>Precision</th><th>Recall</th><th>F1score</th></tr></thead><tbody><tr><td>BBBC042</td><td>INCEPTION</td><td>-</td><td>yes</td><td>0.86</td><td>0.66</td><td>0.47</td><td>0.48</td></tr><tr><td>BBBC042</td><td>Caps11</td><td>yes</td><td>yes</td><td>0.83</td><td>0.69</td><td>0.64</td><td>0.65</td></tr><tr><td><strong>BBBC042</strong></td><td><strong>Caps01</strong></td><td><strong>no</strong></td><td><strong>yes</strong></td><td><strong>0.94</strong></td><td><strong>0.87</strong></td><td><strong>0.85</strong></td><td><strong>0.85</strong></td></tr></tbody></table>

<table><caption>Table 3: this table compare the result obtained by our architecture to the results obtained with complement cross entropy loss and focal loss. We can see that our architecture doesn't perform better than those architectures with that dataset</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Recall</th></tr></thead><tbody><tr><td><strong>CIFAR10-LT</strong></td><td><strong>Caps01</strong></td><td><strong>no</strong></td><td><strong>yes</strong></td><td><strong>0.7</strong></td></tr><tr><td>CIFAR10-LT</td><td>Resnet34+CCE [19]</td><td>-</td><td>-</td><td>0.8837</td></tr><tr><td>CIFAR10-LT</td><td>Focal Loss [19]</td><td>-</td><td>-</td><td>0.8616</td></tr></tbody></table>

<table><caption>Table 4: this table shows how capsule networks behave on balanced dataset. The medical dataset NIH achieves great results meanwhile balanced CIFAR10 is under 0.7.</caption><thead><tr><th>Dataset</th><th>Model</th><th>Decoder</th><th>Augmentation</th><th>Recall</th></tr></thead><tbody><tr><td>NIH</td><td>Caps01</td><td>no</td><td>yes</td><td>0.95</td></tr><tr><td>CIFAR10</td><td>CapsNet [19]</td><td>yes</td><td>yes</td><td>0.68</td></tr></tbody></table>

## 3.3 Discussions

Our experiments demonstrated the effectiveness of capsule networks for handling highly imbalanced datasets (Table 1, line 3). Capsule networks consistently outperformed convolutional networks, regardless of decoder usage. This was true for both the original BBBC041 dataset and the subsampled BBBC042 dataset (Tables 1 & 2, lines 3 & 4 and lines 2 & 3). Interestingly, capsule networks achieved better performance on the smaller dataset, sacrificing some accuracy but improving other metrics. This suggests their suitability for limited data scenarios.

Data augmentation's importance is evident when comparing lines 4 and 5 of Table 1. Without augmentation, performance suffered. However, our capsule architecture applied to the CIFAR-10 long tail dataset yielded results surpassed by other architectures (Table 3). While the CIFAR-10 results are not insignificant from a broader perspective, they fall short of recent benchmarks for this dataset. Our findings, as shown in Table 4, suggest that Capsule Networks may be more suited for specific datasets. While performance on the balanced CIFAR-10 dataset was modest, they achieved excellent results on the NIH medical dataset. This indicates that Capsule Networks might not generalize well to all balanced datasets but have the potential to excel.
in medical image analysis. In summary, our study demonstrates that Capsule Networks can be effective for both balanced datasets (in the case of the NIH data) and imbalanced medical datasets(BBBC041 and BBBC042).

## IV CONCLUSION

This study investigated the robustness of capsules networks for image classification on imbalanced datasets. We compared capsules networks and convolutional neural network (CNN) architectures on two datasets, applying established techniques to address imbalanced data bias.

The results were promising. capsules networks outperformed convolutional neural networks on highly imbalanced medical datasets. However, our capsules network architecture did not surpass recent results on the CIFAR-10 long-tail dataset. This suggests that Capsules networks are not a universal solution, and their performance depends on the dataset characteristics. We achieved better results with lower imbalance ratios on datasets BBBBC041 and BBBC042, indicating potential in this area. We also found that even with balanced datasets capsules networks doesn't always get great results and that all medical datasets used in this study give promising results whether they were balanced or not. Encouraged by these findings, we propose further research directions:

* Dataset suitability and explainability: Investigate if specific data features or descriptors favor Capsules networks.

* Loss functions: Implement the recently proposed complement cross-entropy loss function for capsules networks or design a new loss function for capsules networks

* Network architecture: Experiment with deeper Capsules Networks architectures (more than two capsules layers) and alternative data augmentation techniques beyond random flipping and rotation.

## REFERENCES

### Publications

[1] V. Ljosa, K. Sokolnicki, and A. Carpenter. "Annotated high-throughput microscopy image sets for validation". In: *Nature methods* (2012).

[2] S. Sabour, N. Frosst, and G. E. Hinton. "Dynamic Routing Between Capsules". In: *Advances in Neural Information Processing Systems*. 2017.

[3] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. "Focal Loss for Dense Object Detection". In: *IEEE Conference on Computer Vision and Pattern Recognition*. 2018.

[4] Y. Gao, M. Wang, G. Zhang, L. Zhou, J. Luo, and L. Liu. "Cluster-Based Ensemble Learning Model for Aortic Dissection Screening". In: *International Journal of Environmental Research and Public Health* (2019).

[5] C. Shorten and T. M. Khoshgoftaar. "A survey on Image Data Augmentation for Deep Learning". In: *Journal of Big Data* (2019).

[6] Z. Liu, W. Cao, Z. Gao, J. Bian, H. Chen, Y. Chang, and T.-Y. Liu. "Self-paced Ensemble for Highly Imbalanced Massive Data Classification". In: *2020 IEEE 36th International Conference on Data Engineering (ICDE)*. 2020.

[7] G. Madhu, A. Govardhan, B. Srinivas, S. A. Patel, B. Rohit, and B. Bharadwaj. "Capsule Networks for Malaria Parasite Classification: An Application Oriented Model". In: *2020 IEEE International Conference for Innovation in Technology (INOCON)*. 2020.
in medical image analysis. In summary, our study demonstrates that Capsule Networks can be effective for both balanced datasets (in the case of the NIH data) and imbalanced medical datasets(BBBC041 and BBBC042).

## IV CONCLUSION

This study investigated the robustness of capsules networks for image classification on imbalanced datasets. We compared capsules networks and convolutional neural network (CNN) architectures on two datasets, applying established techniques to address imbalanced data bias.

The results were promising. capsules networks outperformed convolutional neural networks on highly imbalanced medical datasets. However, our capsules network architecture did not surpass recent results on the CIFAR-10 long-tail dataset. This suggests that Capsules networks are not a universal solution, and their performance depends on the dataset characteristics. We achieved better results with lower imbalance ratios on datasets BBBBC041 and BBBC042, indicating potential in this area. We also found that even with balanced datasets capsules networks doesn't always get great results and that all medical datasets used in this study give promising results whether they were balanced or not. Encouraged by these findings, we propose further research directions:

* Dataset suitability and explainability: Investigate if specific data features or descriptors favor Capsules networks.

* Loss functions: Implement the recently proposed complement cross-entropy loss function for capsules networks or design a new loss function for capsules networks

* Network architecture: Experiment with deeper Capsules Networks architectures (more than two capsules layers) and alternative data augmentation techniques beyond random flipping and rotation.

## REFERENCES

### Publications

[1] V. Ljosa, K. Sokolnicki, and A. Carpenter. "Annotated high-throughput microscopy image sets for validation". In: *Nature methods* (2012).

[2] S. Sabour, N. Frosst, and G. E. Hinton. "Dynamic Routing Between Capsules". In: *Advances in Neural Information Processing Systems*. 2017.

[3] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. "Focal Loss for Dense Object Detection". In: *IEEE Conference on Computer Vision and Pattern Recognition*. 2018.

[4] Y. Gao, M. Wang, G. Zhang, L. Zhou, J. Luo, and L. Liu. "Cluster-Based Ensemble Learning Model for Aortic Dissection Screening". In: *International Journal of Environmental Research and Public Health* (2019).

[5] C. Shorten and T. M. Khoshgoftaar. "A survey on Image Data Augmentation for Deep Learning". In: *Journal of Big Data* (2019).

[6] Z. Liu, W. Cao, Z. Gao, J. Bian, H. Chen, Y. Chang, and T.-Y. Liu. "Self-paced Ensemble for Highly Imbalanced Massive Data Classification". In: *2020 IEEE 36th International Conference on Data Engineering (ICDE)*. 2020.

[7] G. Madhu, A. Govardhan, B. Srinivas, S. A. Patel, B. Rohit, and B. Bharadwaj. "Capsule Networks for Malaria Parasite Classification: An Application Oriented Model". In: *2020 IEEE International Conference for Innovation in Technology (INOCON)*. 2020.
[8] H. K. Ragb, I. T. Dover, and R. Ali. "Deep Convolutional Neural Network Ensemble for Improved Malaria Parasite Detection". In: *2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)*. 2020, pages 1-10.

[9] F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He. *A Comprehensive Survey on Transfer Learning*. 2020.

[10] H. Kang, T. Vu, and C. D. Yoo. "Learning Imbalanced Datasets With Maximum Margin Loss". In: *2021 IEEE International Conference on Image Processing (ICIP)*. 2021.

[11] P. Nair, R. Doshi, and S. Keselj. *Pushing the Limits of Capsule Networks*. 2021. arXiv: 2103.08074 [cs.CV].

[12] Y. Liu, G. Yang, S. Qiao, M. Liu, L. Qu, N. Han, T. Wu, G. Yuan, T. Wu, and Y. Peng. "Imbalanced data classification: Using transfer learning and active sampling". In: *Engineering Applications of Artificial Intelligence* (2023).

[13] I. Belderbos, T. de Jong, and M. Popa. "GANs Based Conditional Aerial Images Generation for Imbalanced Learning". In: *Pattern Recognition and Artificial Intelligence: Third International Conference, ICPRAI 2022, Paris, France, June 1-3, 2022, Proceedings, Part II*.

[14] Y. Bengio and Y. Lecun. "Convolutional Networks for Images, Speech, and Time-Series". In.

[15] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. "SMOTE: Synthetic Minority Over-sampling Technique". In.

[16] D. Do. *Diseases Detection from NIH Chest X-ray data*.

[17] X. Guo. "A Keras implementation of CapsNet in NIPS2017 paper "Dynamic Routing Between Capsules"". In: *Github repository* ().

[18] H. He, Y. Bai, E. Garcia, and S. Li. "ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning". In: *Proceedings of the International Joint Conference on Neural Networks*.

[19] Y. Kim, Y. Lee, and M. Jeon. "Imbalanced image classification with complement cross entropy". In: *Pattern Recognition Letters* ().

[20] B. Krawczyk. "Learning from imbalanced data: Open challenges and future directions". In: *Progress in Artificial Intelligence*.

[21] A. Krizhevsky, V. Nair, and G. Hinton. "CIFAR-10 (Canadian Institute for Advanced Research)". In.

[22] A. Kulkarni, D. Chong, and F. A. Batarseh. "Foundations of data imbalance and solutions for a data democracy". In.

[23] Y. Liu, G. Yang, S. Qiao, M. Liu, L. Qu, N. Han, T. Wu, G. Yuan, T. Wu, and Y. Peng. "Imbalanced data classification: Using transfer learning and active sampling". In: *Engineering Applications of Artificial Intelligence* ().
[8] H. K. Ragb, I. T. Dover, and R. Ali. "Deep Convolutional Neural Network Ensemble for Improved Malaria Parasite Detection". In: 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). 2020, pages 1-10.

[9] F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He. *A Comprehensive Survey on Transfer Learning*. 2020.

[10] H. Kang, T. Vu, and C. D. Yoo. "Learning Imbalanced Datasets With Maximum Margin Loss". In: *2021 IEEE International Conference on Image Processing (ICIP)*. 2021.

[11] P. Nair, R. Doshi, and S. Keselj. *Pushing the Limits of Capsule Networks*. 2021. arXiv: 2103.08074 [cs.CV].

[12] Y. Liu, G. Yang, S. Qiao, M. Liu, L. Qu, N. Han, T. Wu, G. Yuan, T. Wu, and Y. Peng. "Imbalanced data classification: Using transfer learning and active sampling". In: *Engineering Applications of Artificial Intelligence* (2023).

[13] I. Belderbos, T. de Jong, and M. Popa. "GANs Based Conditional Aerial Images Generation for Imbalanced Learning". In: *Pattern Recognition and Artificial Intelligence: Third International Conference, ICPRAI 2022, Paris, France, June 1-3, 2022, Proceedings, Part II*.

[14] Y. Bengio and Y. Lecun. "Convolutional Networks for Images, Speech, and Time-Series". In.

[15] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. "SMOTE: Synthetic Minority Over-sampling Technique". In.

[16] D. Do. *Diseases Detection from NIH Chest X-ray data*.

[17] X. Guo. "A Keras implementation of CapsNet in NIPS2017 paper "Dynamic Routing Between Capsules"". In: Github repository ().

[18] H. He, Y. Bai, E. Garcia, and S. Li. "ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning". In: *Proceedings of the International Joint Conference on Neural Networks*.

[19] Y. Kim, Y. Lee, and M. Jeon. "Imbalanced image classification with complement cross entropy". In: *Pattern Recognition Letters* ().

[20] B. Krawczyk. "Learning from imbalanced data: Open challenges and future directions". In: *Progress in Artificial Intelligence*.

[21] A. Krizhevsky, V. Nair, and G. Hinton. "CIFAR-10 (Canadian Institute for Advanced Research)". In.

[22] A. Kulkarni, D. Chong, and F. A. Batarseh. "Foundations of data imbalance and solutions for a data democracy". In.

[23] Y. Liu, G. Yang, S. Qiao, M. Liu, L. Qu, N. Han, T. Wu, G. Yuan, T. Wu, and Y. Peng. "Imbalanced data classification: Using transfer learning and active sampling". In: *Engineering Applications of Artificial Intelligence* ().