1 Classification of Microplanktons in an
2 Imbalanced Digital Holographic Image Dataset
3 with Deep Network using Channel Attention
SHRIHARI A,1,* PRITHWIJIT GUHA,
1,2
4 AND RISHIKESH DILIP
KULKARNI1,2
5
1
6 Centre for Intelligent Cyber Physical Systems, Indian Institute of Technology Guwahati
2
7 Department of Electronics & Electrical Engg., Indian Institute of Technology Guwahati
*
8 <PRESIDIO_ANONYMIZED_EMAIL_ADDRESS>
9 Abstract: The intricate nature and vast variety of microplanktons make the task of their
10 classification in digital holographic image quite demanding. This difficulty arises due to a myriad
11 of factors such as shifts in viewpoint, alterations in illumination, geometric anomalies, and
12 the presence of noise within a digital holographic microscope, the instrument responsible for
13 generating images of microplanktons. These elements often change the local representation of
14 image features. Moreover, obtaining instances of various microplankton classes is substantially
15 skewed due to the difficulties involved in data collection. This situation gives rise to a challenging
16 classification issue with a prejudiced dataset. Dealing with class-imbalanced datasets poses
17 a significant hurdle in machine learning, where categorical representations are prone to favor
18 majority classes. Accordingly, this research provides two primary contributions. Firstly, it
19 introduces a novel debiasing method using channel attention layers at multiple feature scales.
20 Secondly, it employs hard-sample mining to distinguish between bias-aligned (easy) and bias21 conflicting (hard) samples of microplankton classes. These bias-conflicting (hard) samples
22 are used as benchmarks for testing the influence of bias-aligned attributes in the networks.
23 The proposed method achieved an average F1-score of 94.2% with 6.68M parameters and 6.4
24 GFLOPs. It was found to provide competitive performance in comparison to four state-of-the-art
25 debiasing techniques.
26 © 2024 Chinese Laser Press
27 1. Introduction
28 Microplanktons (MPs) are integral to the function and food chains within aquatic ecosystems [1].
29 The diversity and volume of MPs can offer insightful data on the current environmental conditions.
30 Therefore, examining the variety of submarine MP communities is beneficial in preserving
31 the health of our water bodies [2]. Novel advancements in visual technologies have enabled
32 the direct observation of MPs from large volumes at a highly detailed level, without causing
33 damage to their cellular structures or alignments [3]. One such method is the digital holographic
34 microscopy (DHM). Its extensive depth of field, 3D imaging and in-situ applications [4] have
35 made DHM a popular choice for MP detection. Microplankton (MP) related datasets have gained
36 considerable attention in the realms of limnology and oceanography studies [5]. Nonetheless,
37 the concentration of the MPs in a specific region of water bodies lead to the difficulty in the
38 acquisition of certain rare classes of MPs. This lead to the problem of data imbalance.
39 The concern of training deep learning models using imbalanced image data has been a
40 significant challenge due to the unequal data distribution [6]. This skewed distribution hampers
41 the extraction of relevant features as the model’s representation tends to favor the dominant class.
42 This leads to less accurate performance for the underrepresented class [7]. The data bias results
43 in the model effectively classifying data samples possessing biased traits (bias-aligned samples),
44 but it struggles to accurately predict samples lacking these biased features (bias-conflicting
1
45 samples). Consequently, we can divide the dataset into two distinct parts. Bias-aligned samples –
46 These comprise data with such spurious correlations [8], making up the bulk of the training set.
47 Bias-conflicting samples - This subset contains data that is difficult for the classifier to predict
48 and constitutes a smaller segment of the training set [8].
49 In this study, a comprehensive database of MPs created by Nayak et al. [2] with a submersible
50 digital holographic imaging system has been utilized. This dataset called MP−36 has 36
51 categories of traditionally reconstructed in-focus images for classification purposes1 (refer to
52 Fig. 1). To our comprehension, this is the first study to perform the classification of 36 categories
53 of MPs using the aforementioned MP-36 dataset. The training data at hand is notably imbalanced
54 and skewed, hence, attention blocks have been implemented in this proposed study to give more
55 focus to classes that are less represented. These attention components are designed to comprehend
56 the cost of misclassification for every sample based on the feature pyramid. Subsequently, the
57 learnt misclassification cost is factored into the cost matrix, thereby giving direct focus to
58 samples that are in minority. The implementation of attention blocks in this study is crucial
59 for addressing the issue of imbalanced and skewed training data. The attention components
60 prioritize the minority classes by considering misclassification cost for each sample. It ensures a
61 more balanced classification process and an enhanced classification accuracy for the seven biased
62 models described in [9], namely., VGG16, ShuffleNetV2, ResNet50V2, ResNet152V2, Inception,
63 Xception and ShincNet. This approach allows for a more comprehensive analysis of the MP-36
64 dataset . The major contributions of this paper include:
65 • Proposal of a method for Debiasing using Channel Attention Blocks (DCAB), compared
66 against four state-of-art debiasing methods. With ∽ 6 million parameters and 6.4 GFLOPs,
67 the proposed method achieved an accuracy of 94% for ShincNet as compared to the six
68 other biased models.
69 • Separation of bias-aligned and bias-conflicting samples by hard-sample mining using one
70 of the seven biased networks. These are used as benchmarks for testing the influence of
71 bias-aligned attributes in the networks.
72 2. Related Work
73 A research conducted by Liam et al. [3] employed a digital holographic microscope, HoloSea
74 (both portable and in-line) to collect underwater MPs images. They later deployed four distinct
75 pre-trained deep neural networks using transfer learning methodologies for classification of
76 the collected MPs. Conversely, Nayak et al. [2] executed in-situ tests during various mission
77 deployments from 2014 to 2017. They utilized a submersible digital holographic imaging
78 system, HOLOCAM, to compile an exhaustive MP database. For each category, they acquired
79 in-focus MP images and background-subtracted holograms. This subtraction involved eliminating
80 the background (time-averaged) from the raw holograms, effectively eradicating any noise or
81 abnormal patterns. Subsequently, a digital reconstruction of the holograms was carried out using
82 the Fresnel diffraction formulation [10]. They employed a grayscale equalization process to
83 generate in-focus imagery. This dataset (MP-36) has been widely used in most existing MP
84 classification studies.
85 The MP-36 dataset under discussion consists of 36 categories, subjected to a majority to
86 minority ratio of 121.68 : 1. However, to prevent complications arising from class imbalance,
87 Guo et al. [1] limited the classification to just 10 classes of the MP dataset (MP-10), maintaining
88 a majority to minority ratio of 9.45 : 1 through the utilization of the ShuffleNetV2 architecture.
89 Shrihari et al. [9] introduced a unique network structure known as ShincNet (see Fig. 2(a) &
1The following abbreviations and color codes for each of the 36 classes as in 1 will be used as it is throughout the
paper unless specified.
2
Fig. 1. Images of the 36 classes in MP-36 dataset [2]. Right: Conventionally reconstructed in-focus images
of MPs. Left: Background-subtracted Holograms of MPs. From top to bottom in clockwise direction: AG -
Aggregate , CD - C-Shaped Diatom , CFd - Ceratium fusus (dividing) , CMs - Ceratium muelleri (singular)
, CP - Copepod , D1 - Diatom 1 (c. debilis) , D4 - Diatom 4 (c. concavicornus) , FD - Fuzzy Diatom ,
PS - Phaeocystis sp , SU - Sea Urchin Larvae , TN - Thalassionema nitzschioides , TE - Thin Elongated
Diatom , AP - Appendicularian , CFs - Ceratium furca (dividing) , CFu - Ceratium fusus (singular) , CS
- Chaetoceros socialis , CN - Copepod Nauplii , D2 - Diatom 2 , EC - Elongated Ciliate , KB - Karenia
brevis , PP - Protoperidinium , SC - Square Coscinodiscus , TS - Thalassiosira sp , UD - Unknown Diatom
, BC - Box Chain Diatom , CFc - Ceratium furca (singular) , CDm - Ceratium muelleri (dividing) , CI -
Ciliate , DC - Decoy , D3 - Diatom 3 (ditylum sp.) , FP - Fecal Pellets , MS - Marine Snow , RC - Round
Coscinodiscus , TL - Tadpole-Stage Larvae , TEd - Thick Elongated Diatom , UE - Unknown-Eastsound
90 (b)) for the classification of this MP-10 dataset. It demonstrated an optimal performance with an
91 average F1-score of 86.1% using only 5.29 GFLOPs and 6.08 million parameters. Consequently,
92 outperforming InceptionV3, VGG16 and ShuffleNetV2. It demonstrated comparable performance
93 with respect to the heavier networks such as ResNet50V2, ResNet152V2 and Xception, achieving
94 an accuracy rate of 84%. Despite its preference for majority classes, the ShincNet outperformed
95 previous methodologies employed on the MP-10 dataset. A comprehensive overview of the
96 ShincNet design and a detailed comparison of its results with other methods are available in the
97 supplementary materials.
98 Complex background of holograms and diverse sizes of MPs make their classification
99 difficult [9]. Various shapes and orientations of different MPs across the depths of water bodies
100 cause the neural network to misinterpret certain features of certain categories for other classes.
101 Also, uneven distribution of MP densities in natural water bodies across various geographical
102 locations makes it difficult to obtain adequate samples for certain categories resulting in huge
103 data imbalance [1]. All the existing methods for classification of MPs so far inherently posses
104 this bias in their results. The proposed method DCAB utilizes attention blocks for debiasing
105 the classifier output. Therefore, the related works primarily focus on existing algorithm-level
106 debiasing networks that are compared with DCAB, as well as the specific attention modules that
107 have inspired DCAB.
108 Imbalance in a dataset can result in the lack of accuracy in an image classifier model during
109 prediction due to the bias factor [11]. Recent research has been dedicated to minimizing this bias
110 in datasets, a process known as debiasing [11]. To enhance the debiasing efficiency of a debiased
111 classifier, Lee et al. introduced the BiasEnsemble method [12]. In a similar vein, Bahng et al.
112 put forth ReBias, a method for training a debiased classifier that operates independently from
113 the biased classifier [13]. Kim et al. developed LNL, a technique aimed at reducing reciprocal
3
(a) (b)
Fig. 2. (a) Overview of ShincNet [9] consisting of four processing stages, each composed of processing
blocks. (b) Cross-sectional view of a processing block made of two parallel layers of convolution blocks with
ReLU activation. The top layer has 3 × 3 depth-wise separable convolution. The bottom layer has 4-group
channel shuffled grouped convolution followed by spatially separable convolutions. 1×1 convolution blocks
were added to improve generalization among channels.
114 information between biased attributes [14]. Numerous convolutional architectures combined
115 with attention mechanisms have been proposed for image classification, with a focus on both
116 channel and spatial information [7]. Hu et al. introduced Squeeze-and-Excitation Networks
117 (SENet), which improve channel interdependencies at a low computational cost [15]. Li et al.
118 presented Pyramid Attention Network (PAN), which incorporates a Feature Pyramid Attention
119 module to integrate spatial pyramid attention and global pooling, resulting in improved feature
120 descriptors [16].
121 3. Proposed Methodology
122 It could be noted that (from Table 1) the ShincNet and the other baseline methods presented in [9]
123 has an improved performance over accuracy for the imbalanced MP-36 dataset. But F1-scores
124 are significantly skewed towards the majority classes as shown in Fig. 3, denoting the presence of
bias in the models. The reason for such bias towards majority classes is due to class imbalance
Table 1. Performance comparison of the biased ShincNet and six other baseline methods discussed in [9],
for 36−class MP-36 dataset. Here, A-Accuracy, R-Recall, P-Precision and F1-scores are compared along
with computations in GFLOPS and number of parameters in millions.
Metrics VGG16 ShuffleNetV2 InceptionV3 ResNet50V2 ResNet152V2 Xception ShincNet
A ↑ 0.833 0.864 0.889 0.905 0.912 0.922 0.890
R ↑ 0.818 0.854 0.878 0.897 0.903 0.915 0.878
P ↑ 0.878 0.907 0.928 0.941 0.951 0.958 0.929
F1 ↑ 0.841 0.871 0.897 0.913 0.921 0.931 0.893
DoB ↓ 0.227 0.180 0.236 0.229 0.217 0.169 0.163
KL ↓ 1.58 1.50 1.67 1.64 1.56 1.36 1.20
Comp (GFLOPs) ↓ 30.7 2.09 5.7 6.99 21.9 9.14 5.29
Params (M) ↓ 15.28 8.25 23.94 24.63 61.81 21.93 6.08
125
126 (Fig. 3) in the MP-36 image dataset and also due to complex background and biodiversity of
127 MPs [17]. This forms the motivation for the proposed work.
128 3.1. DCAB: The Proposed Debiasing Method
129 The proposed method Debiasing using Channel Attention Blocks (DCAB) is implemented on
130 the ShincNet and the six state-of-art architectures mentioned in [9] that were heavily biased
4
Fig. 3. Number of samples collected per class in the MP dataset [2] and ShincNet model’s F1-scores vs
sample set distribution of microplankton classes. Here the sizes are normalized by dividing each class total
samples by the class with maximum samples. (Best viewed in color).
131 due to imbalanced MP-36 dataset. The DCAB utilizes channel attention blocks to selectively
132 emphasize informative features and suppress biased information in the holographic image. By
133 incorporating these attention mechanisms, DCAB effectively reduces bias and enhances the
134 overall performance of the biased models, making it a powerful tool for debiasing in holographic
135 image analysis. The weighted pooling layer in DCAB plays a crucial role in aggregating the
136 extracted features from the convolution layers of the biased models and channel attention blocks.
137 This pooling layer effectively combines the information and produces a final representation that is
138 robust to bias. The integration of the biased models and the DCABs showcases the potential for
139 addressing bias in holographic image analysis and advancing the field of debiasing techniques.
140 The choices of the biased models include: ShincNet, VGG16, InceptionV3, ResNet50V2,
141 ResNet152V2, ShuffleNetV2 and Xception. VGG16 is the shallowest in terms of model depth
142 with 3 × 3 convolution and max pooling layers. ResNets and InceptionV3 are of near equal
143 depth [3] with identity connections between the blocks and parallel convolution filters, respectively.
144 Xception is the deepest with depthwise seperable convolutions. ShuffleNetV2 is a lightweight
145 architecture [18] with group convolutions and channel shuffling operations. ShincNet incorporates
146 variable filter sized parallel convolutions followed by 1×1 convolution and group convolutions
147 followed by channel shuffling operation. Advantages included improved accuracy, reduced model
148 complexity, allowing different groups of channels to communicate and recognizing features of
149 different sizes in one layer [19].
150 Figure. 4(a) shows the overview of a debiasing network where DCAB is added to a biased
151 model. The architecture can be divided into two parts, Main Biased Model (MBM) and the
152 attached DCAB module. The input image is fed to MBM where the features are extracted at
153 various scales in into DCAB. Figure. 4(b) shows the architecture of DCAB consisting a channel
154 attention block followed by a weighted pooling layer. Inspired by the squeeze-and-excitation
155 module [15], the DCAB’s channel attention block consists of a global average pooling layer
156 followed by two convolution layers. The output of the final convolution layer of the MBM
U ∈ R
ℎ
′×𝑤′×𝑐
157 is given to the DCAB. It hosts a Global Average Pooling (GAP) layer where the
5
(a) (b)
Fig. 4. (a) Debiasing network formed by the addition of channel attention block to the biased models. It
consists of biased architectures forming a Main Biased Model (MBM) and auxillary Debiasing Channel
Attention Block (DCAB). (b) Cross-sectional view of debiasing channel attention block. It consists of
Global Average Pooling followed by conv layers with sigmoid activation.
feature map U is pooled, along each channel, into a vector u ∈ R
𝑐×1
, where ℎ
′
158 is the height,
𝑤
′
159 is the width and 𝑐 is the number of channels. Afterwards, these channel-wise dependencies
are exploited by two convolution layers. First, compressing the vector u into uˆ ∈ R
𝑐/𝑟×1
160 and
the other expanding u
′
to u
a ∈ R
𝑐×1 where, 𝑟 is compression ratio. Then, these 𝑐 values of u
a
161
are multiplied as weights with the original tensor U to get U˜ ∈ R
ℎ
′×𝑤′×𝑐
162 , scaling each channel
163 based on its importance. This is then passed to convolution weighted pooling layer with stride 2
164 for the weighted average feature embedding of the small region. Similar to Bias Product [20],
let F ∈ R
ℎ×𝑤×𝑘
165 be the high-level feature maps from the output of the last convolution stage of
166 MBM, where ℎ is height, 𝑤 is width and 𝑘 is number of channels. It is upsampled with the kernel
filters S = [s1,s2, . . . ,s𝑐] to produce F˜ ∈ R
ℎ
′×𝑤′×𝑐
167 :
F˜ =
1
ℎ
′ × 𝑤′
∑︁
ℎ,𝑤
∑︁
𝑘
𝑖=1
s𝑖 ⊗ f𝑖 (1)
Where, f𝑖 ∈ R
ℎ×𝑤 is one feature map of F2. This F˜ is multiplied with the low-level features 168 U˜ of
each of the DCAMs to select precise resolution details: w𝑖 = log (u˜𝑖) + log
˜f𝑖

169 . Equivalently,
W ∝ U˜ ⊙ F˜ 170 3, where W, is the misclassification cost matrix in DCAB proceeding to the GAP
171 stage followed by fully connected layers. This feature embedding is transformed into output
172 of 36 nodes hosting SoftMax activation functions. These activation functions produce a class
probability distribution specific to the thirty-six MP categories [21], denoted by 𝑦
′
𝑘
173 and 𝑦ˆ𝑖𝑘 for
174 MBM’s and DCAB’s outputs, respectively, with 𝑦˜𝑘 being the ground truth and 𝑘 being one of
175 the MP categories. The cross-entropy loss is minimized on DCAB and finally added with the
176 MBM’s cross-entropy loss, during training.
JTOT = JMBM + JDCAB (2)
177 Hard Sample Mining (HSM) strategy was employed to create the training dataset for assessing
178 the bias in the model’s predictions. A detailed view of HSM is provided in the supplementary.
2⊗ denotes the convolution.
3⊙ is element-wise multiplication.

179 4. Experiments and Results
180 This section provides quantitative performance analysis of the proposed model along with the
181 baselines, implementation details, and the related ablation studies.
182 Baseline Methods – The proposed DCAB is compared against the following baseline methods [11].
183 First, BiasEnsemble [12] highlights the significance of enhancing bias for a biased classifier, with
184 the aim of enhancing the debiasing efficiency of a debiased classifier. This concept can also be
185 applied to current debiasing algorithms based on reweighting. Second, (ReBias) [13] utilizes the
186 Hilbert-Schmidt Independence Criterion to train a debiased classifier that is independent from
187 the biased classifier. Third, Learning Not to Learn (LNL) [14] adds an extra classifier branch to
188 reduce reciprocal information between the embedded attributes and the biased attribute. The
189 Dataset Augmented ShincNet [9] (DA_ShincNet) is used as the fourth baseline. Here, standard
190 data augmentation techniques [22] were employed including random rotation (10°) with random
191 shift, random vertical flip followed by horizontal flip, and vertical flip and random rotation (10°)
192 in order to balance the thirty-six class MP data.
193 Implementation Details – Minimization of the cross-entropy loss was followed while training
194 both MBM and DCAB with the Adam optimizer. The batch size was set to 16, with learning rate
195 at 0.001. The compression ratio, denoted as 𝑟, was fixed at 8, and a dropout probability of 0.4
196 was applied. To train the model, a stratified 2-fold approach was employed [3], with each fold
197 undergoing 50 epochs. The final prediction score is reported as the average of the performance
198 scores. The networks were implemented on the NVIDIA RTX 3060 GPU with 32GB RAM,
199 utilizing NVIDIA CUDA Toolkit.
200 Accuracy and class-wise F1-scores were used for evaluating the classifier performance [3].
201 Classifier complexities were compared using number of model parameters (in millions) and
202 computational costs (in Giga-FLOPs or GFLOPs). Also, two other metrics Degree of Bias
203 (DoB) [23] and Divergence between score distributions (KL) [24] were used to measure bias in
204 the model prediction. DoB measures the standard deviation of classification accuracy across
205 target attributes of easy samples, a lower value of DoB indicating low bias in model prediction.
206 But, DoB fluctuates depending upon the classification threshold chosen. KL Divergence is
a threshold-invariant measurement and is defined as: 𝐾 𝐿[ 𝑝𝑏=0,𝑡 ∥ 𝑝𝑏=1,𝑡]
+ 𝐾 𝐿[ 𝑝𝑏=1,𝑡 ∥ 𝑝𝑏=0,𝑡]
207 for
208 each target attribute label 𝑡. Wherein, the divergence of 𝑏 = 0 (hard) and 𝑏 = 1 (easy) score
209 distributions was measured, where 𝑝𝑏,𝑡 represents a normalized Gaussian approximated N-bin
210 histogram of an easy/hard sample’s classifier scores 𝑏 and a target label 𝑡, as a probability
211 distribution of the scores. A lower value of KL indicates low bias in model prediction.
212 Quantitative Results – In Table 2, the performance comparisons of biased ShincNet and six
213 other baseline methods discussed in [9] for MP-36 dataset with the addition of DCAB is shown.
214 The model performances were found to have higher performance than their counterparts without
DCAB technique. Decrease in DoB and KL metrics can be noted as compared to Table 1. A plot
Table 2. Overall metrics comparison of ShincNet and six other baseline methods ( [9]) with the addition of
DCAB. Here, A-Accuracy, R-Recall, P-Precision and F1-scores are compared along with GFLOPS and
number of parameters in millions.
Metrics VGG16 ShuffleNetV2 InceptionV3 ResNet50V2 ResNet152V2 Xception ShincNet
A ↑ 0.862 0.893 0.911 0.925 0.934 0.951 0.940
R ↑ 0.854 0.881 0.899 0.906 0.911 0.928 0.920
P ↑ 0.887 0.916 0.950 0.951 0.955 0.967 0.959
F1 ↑ 0.865 0.895 0.914 0.928 0.935 0.953 0.942
DoB ↓ 0.189 0.142 0.198 0.191 0.179 0.131 0.125
KL ↓ 1.20 1.12 1.29 1.26 1.18 0.98 0.82
Comp (GFLOPs) ↓ 31.9 3.19 6.7 9.01 23.2 10.15 6.48
Params (M) ↓ 15.89 8.86 24.53 25.23 62.48 22.54 6.69
7
215 of F1-scores for each class in MP-36 dataset predicted by the ShincNet and other biased models
216 with the addition of DCAB technique is shown in Fig. 5. ShincNet with DCAB achieved the
217 highest F1-score of 96.4% for diatom 4 (c. concaviconous) and the lowest F1-score of 90.1% for
Thalassiosira sp. The F1-scores showed consistency across all classes with DCAB addition.
Fig. 5. Class-wise F1-scores for biased models with the addition of debiasing channel attention block. (Best
viewed in color).
218
219 Ablation Analysis – Exhaustive experiments were conducted in terms of variations in the bias
220 product and variations in number of debiasing channel attention blocks added. Here, ShincNet
221 was used as a biased model to showcase the importance of adding DCAB block to it. This
222 architecture is hereby termed Channel Attention Debiasing Shinc Network (ShincNet_DCAB). It
223 consists of the following two components – first, the Main Biased Model (MBM) containing the
224 components of ShincNet model, and second, the Debiasing Channel Attention Blocks (DCAB).
225 Figure. 6 shows the architecture of ShincNet_DCAB consisting of MBM and DCABs. These two
226 feature extraction components form the backbone module. The primary goal of this module is to
determine the underlying features of the holographic image while reducing the biasing effect4.
Fig. 6. Overview of the proposed ShincNet_DCAB. It consists of ShincNet architecture forming a Main
Biased Model (MBM) and auxillary Debiasing Channel Attention Modules (DCAM).
227
228 In Table 3, the performance comparison of different debiasing methods with respect to the
229 overall accuracy, average precision, DoB and KL divergence are shown. Though the Accuracy
230 of the proposed technique (94.04%) and Average Precision (AP) (95.98%) are slightly lesser
231 than BiasEnsemble, it was found to have a better performance in terms of debiasing with DoB
4Detailed explanations on the architecture and more experimental results are provided in the supplementary material.
8
of just 0.12% and KL Divergence of 0.82. Fig. 7 shows the graph of KL divergence vs average
Table 3. Performance comparision of ShincNet_DCAB
with other state-of-art debiasing techniques. Uparrow ↑in
Accuracy and Average Precision (AP) indicates higher
the better and downarrow ↓in Degree of Bias (DoB) and
Divergence (KL) indicates lower the better.
Metrics LNL ReBias BiasEnsemble DA_ShincNet ShincNet_DCAB
Accuracy ↑ 0.862 0.893 0.951 0.934 0.940
AP ↑ 0.887 0.916 0.967 0.950 0.959
DoB ↓ 0.189 0.142 0.131 0.201 0.125
KL ↓ 1.20 1.12 0.98 1.24 0.82
Comp (GFLOPs) ↓ 3.65 8.01 6.94 5.29 6.48
Params (M) ↓ 14.28 18.95 15.53 6.08 6.69
Fig. 7. KL divergence vs average F1-scores
for different model complexities of ShincNet_DCAB. Larger and darker the bubble, higher
is the model complexity and computation (Best
viewed in color).
232
233 F1-score for different scales of ShincNet_DCAB in terms of number of stages, and hence the
234 number of pyramidal channel attention blocks with their respective loss. The following notation
235 was adopted: DCAB_dc_mp_gf, where DCAB is the proposed ShincNet_DCAB model with dc
236 DCABs, mp million parameters (integer approximated) and gf GFLOPs (integer approximated).
237 It was observed that the proposed ShincNet_DCAB with 3 DCABs, 6 million parameters and 6
238 GFLOPs performed the best in terms of KL divergence score and average F1-score, as compared
239 to other versions of ShincNet_DCAB.
240 Fig. 8 shows the existence of bias attributes in the classwise F1-scores of the biased ShincNet
241 as compared with the proposed technique. ShincNet_DCAB achieved a minimum F1-score
242 of 90.1% for Thalassiosira sp and a maximum of 96.4% for diatom 4 (c. concaviconous),
respectively. Compared to the biased ShincNet, F1-scores were consistent among the classes.
Fig. 8. Class-wise F1-scores for the biased ShincNet and the ShincNet_DCAB. F1-scores of ShincNet_DCAB
are shown as red bars and F1-scores of ShincNet are shown as blue bars (Best viewed in color).
243
244 5. Conclusion
245 This study presents a unique approach to address bias by employing a channel attention mechanism
246 for debiasing. The network, which consists of the MBM and DCAB modules, is trained with a
247 focus on attributes that conflict with bias, aiming to diminish the impact of attributes aligned
248 with bias and gradually achieve debiasing. During training, the proposed technique places
249 greater emphasis on hard samples from minority classes. To illustrate this, ShincNet_DCAB
250 achieved an impressive overall accuracy of 94.04%, with a DoB of 0.12% and KL of 0.82 ± 0.58,
251 utilizing the attention information provided by DCAB to represent the misclassification cost for
252 the main network, MBM. This attention mechanism serves as a cue for identifying and evaluating
253 bias-conflicting attributes during the training process. Extensive experiments conducted on
9
254 an imbalanced microplankton dataset for classification tasks confirm the generalizability and
255 effectiveness of the proposed method.
256 Acknowledgments. The authors acknowledge the financial support provided by TIH IIT Guwahati DST
257 [TIH/TD/0406]
258 Disclosures. The authors declare no conflicts of interest.
259 Data Availability Statement. Data underlying the results presented in this paper are are not publicly
260 available at this time but may be obtained from the authors in [1] upon reasonable request.
261 Supplemental document. See Supplement 1 for supporting content.
262 References
263 1. B. Guo, L. Nyman, A. R. Nayak, D. Milmore, M. McFarland, M. S. Twardowski, J. M. Sullivan, J. Yu, and J. Hong,
264 “Automated plankton classification from holographic imagery with deep convolutional neural networks,” Limnol.
265 Oceanogr. Methods 19, 21–36 (2021).
266 2. A. R. Nayak, M. N. McFarland, M. S. Twardowski, and J. M. Sullivan, “On plankton distributions and biophysical
267 interactions in diverse coastal and limnological environments,” in Ocean Sensing and Monitoring X, vol. 10631
268 (SPIE, 2018), pp. 204–215.
269 3. L. MacNeil, S. Missan, J. Luo, T. Trappenberg, and J. LaRoche, “Plankton classification with high-throughput
270 submersible holographic microscopy and transfer learning,” BMC ecology evolution 21, 1–11 (2021).
271 4. N. Briggs, G. Dall’Olmo, and H. Claustre, “Major role of particle fragmentation in regulating biological sequestration
272 of co2 by the oceans,” Science 367, 791–793 (2020). American Association for the Advancement of Science.
273 5. R. R. Kirby and G. Beaugrand, “Trophic amplification of climate warming,” Proc. Royal Soc. B: Biol. Sci. 276,
274 4095–4103 (2009).
275 6. Q. Dong, S. Gong, and X. Zhu, “Imbalanced deep learning by minority class incremental rectification,” IEEE
276 transactions on pattern analysis machine intelligence 41, 1367–1381 (2018).
277 7. L. Wang, L. Zhang, X. Qi, and Z. Yi, “Deep attention-based imbalanced image classification,” IEEE Trans Neural
278 Netw Learn. Syst 33, 3320–3330 (2022).
279 8. N. Kim, S. Hwang, S. Ahn, J. Park, and S. Kwak, “Learning debiased classifier with biased committee,” Adv. Neural
280 Inf. Process. Syst. 35, 18403–18415 (2022).
281 9. A. Shrihari, P. Guha, and R. D. Kulkarni, “Deep neural network for underwater microplankton classification using
282 holograms,” in Optica Imaging Congress (3D, COSI, DH, FLatOptics, IS, pcAOP), (Optica Publishing Group, 2023),
283 p. HTu2C.6.
284 10. J. Katz and J. Sheng, “Applications of holography in fluid mechanics and particle dynamics,” Annu. Rev. Fluid Mech.
285 42, 531–555 (2010). Annual Reviews, Inc., 4139 El Camino Way Box 10139 Palo Alto CA 94303-0139 USA.
286 11. J. Lee, J. Lee, S. Jung, and J. Choo, “Improving evaluation of debiasing in image classification,” arXiv preprint
287 arXiv:2206.03680 (2022).
288 12. J. Lee, J. Park, D. Kim, J. Lee, E. Choi, and J. Choo, “Revisiting the importance of amplifying bias for debiasing,” in
289 Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37pp. 14974–14981.
290 13. H. Bahng, S. Chun, S. Yun, J. Choo, and S. J. Oh, “Learning de-biased representations with biased representations,”
291 in International Conference on Machine Learning, (PMLR, 2020), pp. 528–539.
292 14. B. Kim, H. Kim, K. Kim, S. Kim, and J. Kim, “Learning not to learn: Training deep neural networks with biased
293 data,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9012–9020.
294 15. J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer
295 vision and pattern recognition, (IEEE), pp. 7132–7141.
296 16. H. Li, P. Xiong, J. An, and L. Wang, “Pyramid attention network for semantic segmentation,” arXiv preprint
297 arXiv:1805.10180 (2018).
298 17. P. Qu, T. Li, L. Zhou, S. Jin, Z. Liang, W. Zhao, and W. Zhang, “Damnet: dual attention mechanism deep neural
299 network for underwater biological image classification,” IEEE Access 11, 6000–6009 (2022). IEEE.
300 18. N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,”
301 pp. 116–131. Proceedings of the European conference on computer vision (ECCV).
302 19. X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile
303 devices,” pp. 6848–6856. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
304 20. C. Clark, M. Yatskar, and L. Zettlemoyer, “Don’t take the easy way out: Ensemble based methods for avoiding known
305 dataset biases,” arXiv preprint arXiv:1909.03683 (2019).
306 21. K. Janocha and W. M. Czarnecki, “On loss functions for deep neural networks in classification,” arXiv preprint
307 arXiv:1702.05659 (2017).
308 22. C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” J. Big Data 6 (2019).
309 23. P. Majumdar, R. Singh, and M. Vatsa, “Attention aware debiasing for unbiased model prediction,” in Proceedings of
310 the IEEE/CVF International Conference on Computer Vision, pp. 4133–4141.
311 24. V. V. Ramaswamy, S. S. Kim, and O. Russakovsky, “Fair attribute classification through latent space de-biasing,” in
312 Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9301–9310.
10

This is a paper. Kindly review the paper like a journal reviewer and give back a new document with corresponding changes in the required places.