Learning Discriminative Representations for Chromosome Classification with Small Datasets
Authors: Minghui Li
Haoxi Zhang
Renhao Zhou
Linfeng Yu
Venue: Submitted to ICLR 2023 (2023)
Keywords: Chromosome classification
data representation learning
deep neural networks
discriminative representation
maximal coding rate reduction

Abstract: Chromosome classification is crucial for karyotype analysis in cytogenetics. Karyotype analysis is a fundamental approach for clinical cytogeneticists to identify numerical and structural chromosomal abnormalities. However, classifying chromosomes accurately and robustly in clinical application is still challenging due to: 1) rich deformations of chromosome shape, 2) similarity of chromosomes, and 3) imbalanced and insufficient labelled dataset. This paper proposes a novel pipeline for the automatic classification of chromosomes. Unlike existing methods, our approach is primarily based on learning meaningful data representations rather than only finding classification features in given samples. The proposed pipeline comprises three stages: The first stage extracts meaningful visual features of chromosomes by utilizing ResNet with triplet loss. The second stage optimizes features from stage one to obtain a linear discriminative representation via maximal coding rate reduction. It ensures the clusters representing different chromosome types are far away from each other while embeddings of the same type are close to each other in the cluster. The third stage is to identify chromosomes. Based on the meaningful feature representation learned in the previous stage, traditional machine learning algorithms such as SVM are adequate for the classification task. Evaluation results on a publicly available dataset show that our method achieves 97.22% accuracy and is better than state-of-the-art methods.

Under review as a conference paper at ICLR 2023
LEARNING DISCRIMINATIVE REPRESENTATIONS FOR CHROMOSOME CLASSIFICATION WITH SMALL DATASETS
Anonymous authors Paper under double-blind review
ABSTRACT
Chromosome classification is crucial for karyotype analysis in cytogenetics. Kary- otype analysis is fundamental approach for clinical cytogeneticists to identify numerical and structural chromosomal abnormalities. However, classifying chro- mosomes accurately and robustly in clinical application is still challenging due to: rich deformations of chromosome shape, 2) similarity of chromosomes, and 3) imbalanced and insufficient labelled dataset. This paper proposes a novel pipeline for the automatic classification of chromosomes. Unlike existing methods, our approach is primarily based on learning meaningful data representations rather than only finding classification features in given samples. The proposed pipeline comprises three stages: The first stage extracts meaningful visual features of chro- mosomes by utilizing ResNet with triplet loss. The second stage optimizes fea- tures from stage to obtain linear discriminative representation via maximal coding rate reduction. It ensures the clusters representing different chromosome types are far away from each other while embeddings of the same type close to each other the cluster. The third stage is to identify chromosomes. Based on the meaningful feature representation learned in the previous stage, traditional machine learning algorithms such as SVM are adequate for the classification task. Evaluation results on a publicly available dataset show that our method achieves 97.22% accuracy and is better than state-of- tifte--f-th-attt the-art methods.
INTRODUCTION
Human chromosome classification crucial for karyotype analysis in cytogenetics. Karyotype anal- ysis is fundamental approach for clinical cytogeneticists to identify numerical and structural chro- mosomal abnormalities, such as Turner syndrome, Chronic myelogenous leukaemia, Edwards syn- drome, and Down syndrome (Stebbins & Ledyard. 1950; Sharma al.; 2017). In clinical practice, karyotyping requires the preparation of a complete set of micro-photographed metaphase chromo- somes the cells, more precisely, a karyogram (Figure 1). To do so, the cytogeneticists need to classify and sort these chromosomes into 23 pairs of chromosomes, including 22 pairs of autosomes and pair of sex chromosomes (X and Y chromosomes in male cells and double in female cells)
JJindll et al. 2017).
K
37
SC
11
7)
12
HH
18
#
1
Figure 1: (a) A G-stained microscopic image of male chromosomes for case. (b) The karyogram of (a).

---

Under review as a conference paper at ICLR 2023
Batch Layer
Backhone
Nermalization Layer
ReduNet
Diccrminatiiv
Classifiers
A-YN
Tagle
LightGRM
:
Stage Fexture Learning
Stage Representation Enhancement
Stage
Figure 2: The flow diagram of our proposed pipeline: Chromosome images C are processed using the Backhone network (ResNet) with Triplet Loss for initial feature Learning. Then the learned feature embeddings are fed the ReduNet for discriminative representation learning. Finally, the enhanced feature representations Z are used for chromosome classification by the classifier.
Chromosomes are highly coiled and condensed metaphase, making karyotyping challenging. complicated, and laborious. Even for experienced cytogeneticists, considerable time and manual effort are indispensable classifying and sorting various types of chromosomes to produce karyo- gram. In order reduce the burden of karyotyping, many researchers (Piper & Granum. 1989 Errington et al., 1993 Mashadi et al., 2007), Sharma et al., 2018 Lin et al.) (020) have been dedi- cated karo-kaayotyiing using computer powers for decades. However, classifying chromosomes accurately and robustly in clinical application is still challenging. These challenges mainly stem
from three aspects( Jindal al. 2017 Lin al. 2020 1) rich deformations of chromosome shape, 2) similarity of chromosomes, and 3) imbalanced and insufficient labelled dataset.
To tackle the above challenges, in this paper, we propose novel pipeline for the automatic classi- fication of chromosomes (Figure 2). However, unlike existing methods, our approach is primarily based on learning meaningful data representations rather than only finding classification features in given samples. Specifically, our proposed pipeline comprises three stages: The first stage extracts meaningful visual features chromosomes by utilizing ResNet (He et al., 2016) with triplet loss (Weinberger et al. 2009 Schroof et al. 2015). The second stage optimizes features from stage one to obtain a linear discriminative representation via maximal coding rate reduction (Chan et al. (021). It ensures the clusters representing different chromosome types are far away from each other while vectors of the same type are close to each other in the cluster. The third stage is to identify chro- mosomes. Based the meaningful feature representation learned in the previous stages, traditional machine learning algorithms such as the Support Vector Machine (SVM) (Boser et al. 1992) can casily be plugged into our pipeline as classifiers.
One public dataset is used to validate the performance and generalizability of our approach. Exten- sive experiments the datasets corroborate that our proposed method achieved better performance than state-of oftie--f-theearr methods. Our contributions can be summarised as follows:
We propose a representation learning-oriented approach for chromosome classification. Based the enhanced feature representation, simple and efficient machine learning al- gorithms are adequate for the classification task.
Inspired by FaceNet's (Schroff et al.) 2015) capability of distinguishing similar faces and ReduNet's competence in representation structure optimization, we propose combining them for more meaningful and effective feature learning with small dataset.
We evaluate the proposed approach on a public dataset. It demonstrates its superior perfor- mance compared with the state-of-the-art method.
RELATED WORK
In the early years, hand-crafted and geometrical features (e.g.. a chromosome's 's axis, centromere position, banding pattern features, and length) and complex pre-processings like straightening the chromosomes are primarily required with traditional machine learning algorithms for automatic chromosome classification (Ledley & Lubs. 1980: Egmont- Petersen et al. 2002 Javan-Roshtkhari
2

---

Under review as a conference paper at ICLR 2023
2007 Jahani al. 2012). Recently, with the significant achievement of deep learning (LeCun et al.
et al.
2013). J. researchers tended to employ deep learning- based methods in chromosome classifica- In rescarch (Jindal al. 2017), Jindal et al. proposed to use the Siamese Networks (Koch 2015), a deep learning- based approach, for chromosome classification. First, they straighten et al.
tion.
all chromosomes and then the Siamese Networks to push the embeddings of similar samples closer. Finally, the classification is carried out via an MLP-based module on top of the embeddings given by the trained Siamese Networks. The experimental results yield 84.6% classification accuracy on their private dataset. Sharma et al. (Sharma et al. 2017) proposed convolutional neural network (CNN) based method for classification. Curved chromosomes were straightened by cropping and stitching and then normalized by the length calculated according to centromere position. Based on such pre-procsssing, their method achieved 86.7% classification accuracy. Monika et al. (Sharma et al. 2018) proposed an attention baaeed (Vaswani et al. (017)) pattern-sequence learning method for chromosome classification. By combining the residual convolutional neural network with the recur- rent attention neural network, their method exploits the local features of chromosome bands and the property of chromosome band sequences. The evaluation results showed that their method achieved 91.94% classification accuracy. Yulei et al. (Qin et al. 2019) proposed global-local combined approach for chromosome classification composed of global and local-scale networks. The global network is used to obtain features and find local regions. The found regions are fed into the local network using varifocal mechanism for local feature extraction. They further promote high-level feature extraction through residual and multi-task learning strategies. The authors claimed that this method yielded 99.2% classification accuracy on their large dataset (i.c. 1909 karyotyping cases, 87814 G-band chromosome images). Lin et al. (Lin et al. 2020) introduced an automatic chro- mosome classification approach based on an optimized version of Inception (Szegedy et al.] (017) and a simple but effective augmentation method called CDA for performance improvement. Their data augmentation algorithm not only can enlarge the training dataset and improve the accuracy and robustness of the classifier but also can eliminate the directional features of chromosomes, which allow the model to classify chromosomes without special pre-process (e.g.. rotating, straightening). The experimental results showed that their method achieved 95. 55.98% classification accuracy on the clinical G-band chromosome dataset.
Although the above methods seem to solve the chromosome classification problem to some extent, there are still some limitations to these methods. First, to tackle the deformation problem, the pre- processing operation, such as straightening, is commonly required in many previous approaches, which brings extra step the clinical application and increases the complexity of the classifica- tion task. Second, most existing methods are classification task oriented, which do not learn patterns of chromosomes but focus on finding features that can distinguish limited samples. Such classifi- cation task-oriented design leads to poor generalizability when tackling chromosome data rich in shape deformations and are very similar. Last, for most of these methods, an extensive dataset de- sired. However, sample labelling is labour-intensive and expensive, which restricts the performance of these methods to a relative level according to their dataset size.
3 METHOD
The proposed method for chromosome classification consists of three stages (Figur 2): the first stage the initial extraction and embedding of features using a deep convolutional neural network with triplet loss. The second stage is to enhance the effectiveness and structure of the representations of feature embeddings via the ReduNet. The third stage classifying the chromosome types based on enhanced data representations.
3.1 STAGE 1: INITIAL FEATURE LEARNING
is common for chromosomes to look similar (Figure 1). Additionally, some types of chromosomes are very hard to distinguish in length and banding. Together with the effect of highly coiled and condensed chromosomes and low low-resolution micro- photooppot,g,pph it becomes even tougher to identify chromosomes. Inspired by FaceNet's (Schroff al. (2015) capability of distinguishing similar faces via triplet loss, we propose combining ResNet with triplet loss to extract more meaningful and effective features of chromosomes.
3

---

Under review as a conference paper at ICLR 2023
The architecture of the initial feature learning comprises three main elements: one batch input layer, one backbone network (i.e. ResNet), and one normalization layer producing the feature embeddings. The batch layer prepares the target sample along with positive and negative samples. The backbone network learns features from these triplet samples through the triplet loss function at the network's end. Moreover, the learned features embedded vectors at the normalization layer the output of this stage.
To learn the ideal features, we aim embed an image e into a feature space Ra through the back- bone network f with parameters 0, such that the distance between all chromosomes of the same class is short, whereas the distance between pair of chromosome images from different classes is long. The learned feature embedding is represented as k fo(c) E R it embeds an image e into d-dimensional Euclidean space. Additionally, we constrain this embedding to live on d-dimensional hypersphere, i.e. || k l12= 1. Here we want to ensure that an image (target chromo- some) particular chromosome closer to all other images of (positive) of the same class than is to any image cº (negative) of any other classes. Formally, want:
|| k/ - +
l12
A( - fo(c)) fo(c), folc)))
(2)
Where a is a margin that enforces the distance between positive and negative pairs, T is the set of all possible triplets in the training set of size U. Then, we can have the loss function as:
U
L Σ III f(c)) - f(2) 112 11 f( f(4) 112 +
(3)
In order to ensure fast convergence and learn a meaningful embedding, we introduce the batch input layer to ensure that minimal number of exemplars of any specific class are present in each mini- batch. To this end, we can train the ResNet with the triplet loss via Stochastic Gradient Descent (SGD) (Wilson et 2003).
3.2 STAGE 2: REPRESENTATION ENHANCEMENT
It would be enough to learn good features through stage one if the data were not rich in shape deformation and there were sufficient samples. However, chromosomes have non-rigid intrinsic nature and are highly coiled and condensed in metaphase. Hence, it is critical to represent features low-dimensional structures invariant to such deformations, which are known to have sophisticated geometric and topological structures and can be challenging to learn precisely, even with rigorously designed CNNs (Cohen et al., 2019, Chan et al.) (021). Therefore, we employ ReduNet for discriminative linear representation after the initial feature learning to overcome this issue.
3.2.
DISCRIMINATIVE REPRESENTATION LEARNING
The objective of the representation enhancement to obtain intrinsic low-dimensional structures from the high-dimensional data, and the leamed stucturresneerrssnnaaiio should meet the require- ments (Chan et al. (222I) I) Compressible for same class: Features of samples from the same class/cluster should be relatively correlated the sense that they belong to particular low- dimensional linear subspace and are close to each other, 2) Discriminative for deferent classes: Features of samples from different classes/clusters should be highly uncorrelated and belong dif- ferent low- www-dimensional linear subspaces, and 3) Diverse Representation: The dimension of features for each class/cluster should be as large as possible long as they stay uncorrelated from the other classes.
Formally, we want learn a representation
ger (k) that maps each of the submanifolds MJ IR D to a linear subspace 8 C Rdd. Given initially learned chromosome feature sam- ..., km), we can have a finite number of samples as learned representations 2j 90 (k) R 1, ..... To satisfy the above requirements, measuring the distribution compactness of a random representation & or its finite samples Z = [21 22,..., 2m] fundamental. First, we estimate the number of binary bits needed encode the learned representation Z up precision e, which can be given by:
ples [kr,
L (Z,
m +
log det (I+ me? 7.7""
(4)
2
4

---

Under review as a conference paper at ICLR 2023
Where the function det log(.) is very effective in clustering classification of mixed data with guaranteed convergence (Fazel et al. 2003 Ma et al. 2007: Kang et al., 2013). By computing the number of bits needed to quantize the SVD of subject the precision by packing e-balls into the space spanned by Z as a Gaussian source, we can derive (4). Please refer to (Chan et al. 2021 Fazel al. 2003) for more technical details and proofs.
Then, the compactness of the learned features can be measured terms of the average coding length per sample, a.k.a. the coding rate subject to the distortion
R (Z, = log det (I+
1
ZZ" me²
d
(5)
3.2.2 OPTIMIZATION
To make the learned representations discriminative, features of different classes/clusters should be maximally incoherent to each other. Hence we can stretch to allow the coding rate the whole set to be large possible. Mcanwhile, for each class/cluster, the learned representations should be highly correlated and coherent. Therefore, we need to compress each class/cluster's space (or subspace) minimal volume make the subspace coding rate small possible.
In general, the representation Z is a mixed data comprising multiple low-dimensional subspaces of different classes. To evaluate the rate-distortion of such mixed data more accurately, partition Z into multiple subsets: = Z U Z U ZN with each Z containing samples in one low- N dimensional subspace. Let A {Aj E 1mxm) Jj=1 be set of diagonal matrices with diagonal entries encoding the associates of the samples in the N classes. Precisely, the diagonal entry (i, i) of A² reveals the probability of sample i belonging to the subset j. Thus, we can have the simplex: & {AI^ 2 0, ^3 +... Ae = = With respect to this partition, the average number of bits per sample (the coding rate) can be obtained from (Ma et al. 2007):
tr(12)
d tr(Ni) 2 NNZ z*).
Ra(Z,e/A)= = Σ
log del
2m
Therefore, given partition A Z to make a significant difference between the coding rate of the whole and all the subsets can be achieved by maximizing the coding rate reduction:
max AR A, e) = R(Z, e) R. (2, e(A), 8'
s.t. (0) 1 my $
Where the Frobenius norm normalizes the learned features to scale with the number of features in Z E Rd x my || (0') my- As a result, the R can be maximized as a function of C gd-1 by using gradient ascent in a deep network (i.e. ReduNet) with a purely forward propagation fashion:
DAR
Ze+1 x Ze +1-
s.t. Ze+1 Sdd-1
JAZ
Where l indicates the layer, Ze [=] .... 2 17 is the step size larger than and the iteration starts with the given data Z, = K, The gradient 8AR entails two derivatives of expanding and compressing in A), respectively:
8a%
Es al1 + aZ
(9)
C = aI (T+ + 0j 24 M z
(10)
0AR
d
Es
C
ZaN
(11)
0Z
Exponsion
J=1
Compression
Eg Expansion of the features sapce at the gth layer
C/ Compression of the jth class features the gth layer.
5

---

Under review as a conference paper at ICLR 2023
Finally, to find the optimal go (K), we create small increment transform at the t layer's feature 2g to carry out the above gradient ascent optimization, which can be approximated by nonlinear activation function o-)) in the neural networks
Ze+1 & Ze + Egzt of
"..... CV CF2))) z4)).
(12)
In ReduNet, the rectified linear unit operation, ReLu, used in this approximation as:
(Zz) α Ze
- ReLu((P/
(13)
==1
Where P/- -C+)+ the projection onto the 8 class so that the optimization can performed through its layers, shown in Figure 3.
To this end, the initial features are transformed into an optimized distribution by ReduNet through this gradient ascent operation, where the representation space for the whole set is expanded as large as possible, and representations of the same classes are compressed as small as possible.
Figure 3: A layer of ReduNet derived from one iteration of gradient ascent for the rate reduction optimization.
3.3 STAGE 3: CLASSIFICATION
Through the previous two stages, map the chromosome images into linear discriminative rep- resentation. As the representation is ready, we can classify chromosomes with conventional pattern recognition algorithms.
In our proposed method, an SVM is used as the classifier. SVM uses an optimum linear separating hyperplane to separate two data sets in feature space (Mashadi et al. 2007). This optimum hyper- plane isproduced by maximizing the minimum margin between the two sets. Therefore the resulting hyperplane will only be depended border training patterns called support vectors.
Specifically, the kernel function has an essential effect on the performance of SVM. this paper, use one of the popular kernel functions, the Gaussian kernel.
4 EXPERIMENTS AND RESULTS
4.1 DATASET
To evaluate the performance of our proposed method, we use the publicly available dataset reported in the study (Lin al., 2020). There are only 65 cases total (32 male and 33 female karyotypes). Usually, one case comprises 46 individual chromosomes, which means 2990 total chromosome samples for 24 classes. Specifically, for 22 autosomes (labelled as to 21), there are 130 samples per class, while for sex chromosomes, there 98 (33 2 32) X samples labelled as 22 and only 32 Y samples labelled as 23. We split the dataset into training and testing sets using the ratio of 90:10.

---

Under review as a conference paper at ICLR 2023
4.2 IMPLEMENTATION DETAILS
We first uniform the input image size as 224x224 pixels and then normalize all chromosome images C' as follows: of cj cj/255, small, we replicated it four times in our experiment. The images are randomly cropped and flipped for data augmentation during the training process. Random cropping performed simulate the effects of crossing and covering among chromosomes, and random flipping augments the position of chromosomes. In stage one, the ResNet-50 used as the backbone network. All modules this paper are trained using the SGD optimizer. The initial learning rate is set to 0.001, which reduces by 50 per cent every ten epochs, a total of 50 epochs. All proposed modules are implemented in Python with the Py Torch framework. All experiments are conducted under Ubuntu OS workstation with one single NVIDIA RTX 3090 GPU.
1, N.. Because the amount of the Y chromosome is too
4.3 EVALUATION METRICS
Four metrics are used evaluate the performance of our method: the accuracy of all the testing chromosome images (Ace), the average precision over classes of all the testing images (P), the average recall over classes of all the testing images (Recall), and the average Fl score over classes of all the testing images (F1) as:
N
TP, + +TN, TP; + TN, + FP, + FN,
Acc
(14)
N
1=1
N N
TP₁
Recall =
(15)
TP; + FN,
1=
N
TP,
Precision
(16)
N
TP3 FPJ
F 2 Precision, Recall,
(17)
Precisiony + Recall,
where TP is the number of true positive instances, TN is the number of true negative instances, FP is the number of false positive instances, and FN is the number of false negative instances, N represents the 24 classes of chromosomes.
4.4 RESULTS
This subsection presents the experimental results in two parts. First, we provide the classification results of our proposed method compared with state-of-the-art methods. Then, we provide the per- formance analysis details.
4.4.1 COMPARISON WITH THE STATE-OF-THE.
The general experimental results of our proposed method and several other methods are shown in Table 1. We use three basic CNN networks, AlexNet (Krizhevsky et al., (2012). V6G-16 (Simonyan Zisserman| 2014), and ResNet- 50 (He et al. 2016). as the baseline methods and three advanced methods reported in (Lin et al. 2020) compare with. All experiments were conducted using the same dataset.
According to the experimental results, proposed approach achieves 97.22% classification
curacy, which is better than other methods. It further improves the classification performance by 13.01 percent of the Acc. metric compared to the ResNet-50 baseline, 9.59 percent to SiameseNet (Lin et al. 2020), and 1.24 percent to the CIR-Net. To the best of our knowledge, CIR-Net the Mlateoo-thhe-a method this public dataset.
Moreover, the receiver operating characteristic (ROC) curves and the corresponding area under the curve (AUC) are popular measurements used for performance analysis. Although they are general

---

Under review as a conference paper at ICLR 2023
Table 1: Performances of our method compared with state-of-the-art methods
Method
Aco(%)
F1(%)
P(%)
Recall(%)
AlcxNet(Krrizhvvkk et al.
2012)
83.86
83.52
84.83
83.56
VGG-166Simonyan & Zisserman
2014
91.22
91.22
92.21
91.20
ResNet-50 et al.
2016)
91.92
91.72
91.78
92.44
Vanilla-CNN(LLnn et al.
2020
86.44
87.00
88.00
86.00
SiameseNet Jin et al.
2020)
87.63
87.00
88.00
97.00
CIR-Net(LLin et al.
2020
95.98
96.00
96.00
96.00
ResNet+RRduNett (ours)
92.63
92.39
97.54
92.94
ResNet+TrippLet-Losss (ours)
96.68
96.43
96.80
96.80
ResNet+Tiipeet-LossRRclleee (ours)
97.22
97.18
97.43
97.22
evaluation metrics for binary classification problems, we converted them in order to support multi- classification problems: A one-to-all scheme is first used for the ROC analysis of each class, and then we averaged the ROC curves for all categories and calculated the AUC for each method. The higher AUC values, the better performance (Figure 4). It can be observed that our proposed method outperforms other methods with the least number of false positive predictions and the highest true positive rate. Our proposed method achieves the highest AUC for the classification task.
Average ROC Type
1.00
0.98
True Postive Rate
0.96
0.94
AUC (Alex/Net=
0.92
AUC (ResNet)=0 9711
AUC [VGG)=0.9654
(OU95)-0.9986
0.90
0.0
0.2
0.4
0.6
1.0
Faise Positivv Rata
Figure 4: Comparsions on ROC curves and AUC values of our proposed method and other CNN models.
4.4.2 PERFORMANCE ANALYSIS
The experimental results (Table 1) show that the classification performance of conventional CNNs are all suboptimal, which suggests that the high similarity of chromosomes a big issue for normal CNNs. Even though the ResNet-50 is 8.06 percent more accurate than AlexNet and achieves the best classification accuracy among conventional CNNs, it is still 4.76 percent less accurate than ResNet with triplet loss and 5.3 percent less accurate than ResNet with triplet loss + ReduNet. When we only apply the ReduNet to ResNet-50, the classification accuracy increases by 0.71 percent, while the accuracy rises from 91.92% to 96.68% when only adding triplet loss to ResNet-50. We argue that this is because the ResNet-50 cannot learn effective chromosome features in the first place, leading the ReduNet can hardly do more with these features. Therefore, we first apply the triplet meaningful features, then ReduNet to enhance the learned
loss ResNet-50 to help learn features. As the results show, the classification accuracy of our proposed method further improves by 0.54 percent when applying the ReduNet ResNet-50 with triplet loss.
8

---

Under review as a conference paper at ICLR 2023
10
019 www.eeoooooooo0000000000
Predicted Iobe!
Prodicted label
Figure 5: Confusion matrix of testing of the proposed approach. The left and right represent the ResNet-50 with triplet loss, ResNet-50 with triplet loss and ReduNet, respectively.
To evaluate the impact of triplet loss and ReduNet on classification performance, we created the confusion matrix (Figure 5) for analysis. As we can see from the figure, triplet loss and ReduNet can enhance the classifier's ' learning, even though the samples are imbalanced. For instance, the pure ResNet cannot model the X chromosome (labelled as 22 in Figure 5) well due to the imbalanced sample amount, and it confuses 22 with 3, 7, and 10 classes. However, with triplet loss and ReduNet, through the same ResNet network, the classifier can easily distinguish them with no error.
At last, to investigate the effect of ReduNet representation enhancement, applied the t-SNE (v. d. Maaten & Hinton 2008) method the learned representations to reduce the dimensionality for 2-D visualization. We probed the representations of testing samples learned without and with ResNet + triplet loss in Figure In the figure, the left and right are the t-SNE visualizations of the learned representations of pure ResNet-50 and ResNet-50 with triplet loss ReduNet, respectively. illustrates that the ReduNet optimizes the structure of the representation embeddings and alleviates the entangling features as well the presentation errors.
10
Figure 6: Representation embeddings for chromosomes using t-SNE method in test dataset. Left represents the embeddings learned by ResNet-50, the right is the learned embeddings of Resnet-50 with triplet loss + ReduNet.
CONCLUSION
This paper presents novel three-stage pipeline for automatic chromosome classification. Unlike existing methods, the proposed approach is primarily based on learning meaningful data represen- tations rather than only finding classification features in given samples. integrates triplet loss and ReduNet with convolution neural networks for better feature extraction and representation. Based on the enhanced feature representation, simple and efficient machine learning algorithms are ade- quate for the classification task. Moreover, the proposed approach does not require manual feature engineering and pre-proeee processicsgg like chromosome strengthening, making it efficient and practical for real-world applications. The experimental results show that the proposed approach achieve very high recognition rate of 7,22%% as measured by testing accuracy and 97. 97.18% by F-measure on small dataset.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Boser, Bernhard E.,, Isabelle M. Guyon, and Vladimir N. Vapnik. training algorithm for optimal margin classifiers. Computational learning theory. 1992.
K. H. R. Chan, C. You D. Y. Yu, Wright H. Qi. and Y. Ma. Redunet: A white-box deep network
from the principle of maximizing rate reduction. arXiv preprint arXiv:2105. 2021.
Cohen, Taco S.and Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns on homogeneous spaces. neural information processing systems, 2019.
M. Egmont-Petersen, D. de Ridder, and H. Handels. Image processing with neural networks review. Pattern Recognition, 2002.
Errington, Phil A., and Jim Graham. Application of artificial neural networks to chromosome clas- sification. Cytometry: The Journal of the International Society for Analytical Cytology, 1993.
Fazel, Maryam, Haitham Hindi, and Stephen Boyd. Log-det heuristic for matrix rank minimiza- tion with applications to hankel and euclidean distance matrices. American Control Conference, 2003.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. IEEE conference on computer vision and pattern recognition, 2016.
Jahani, Sahar, and S. Kamaledin Setarehdan. An automatic algorithm for identification and straight- ening images of curved human chromosomes. Biomedical Engineering: Applications, Basis and Communications, 2012.
Lavan-Roshtkie Mehrsan, and S. Kamaledin Setarehdan. A new approach to automatic classifica- tion of the curved chromosomes. International Symposium On Image And Signal Processing And Analysis, 2007.
Jindal, Swati, Gaurav Gupta, Mohit Yadav, Monika Sharma, and Lovekesh Vig. Siamese networks for chromosome classification. IEEE computer vision and pattern recognition workshops, pp- 71, 2017.
Kang, Zhao, Chong Peng, Jie Cheng, and Qiang Cheng. Logdet rank minimization with application to subspace clustering. Computational intelligence and neuroscience. 2015.
Koch, Gregory, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. ICML deep learning workshop, 2015.
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. neural information processing systems., 2012.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 2015.
Robert S.and Paul S. Ing Ledley and Herbert A. Lubs. Human chromosome classification using discriminant analysis and bayesian probability. Computers in Biology and Medicine, 1980.
Lin, Chengchuang, Gansen Zhao, Zhirong Yang, Aihua Yin, Xinming Wang, Li Guo, and Han- biao Chen al. Cir-net: Automatic classification of human chromosome based on inception- resnet architecture. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2020.
Yi Ma, Harm Derksen, Wei Hong, and John Wright. Segmentation of multivariate mixed data via lossy data coding and compression. IEEE transactions on pattern analysis and machine intelligence, 2007.
Mashadi, Narges Tabatabacy, and Seyed Alireza Seyedin. Direct classification human g banddd chromosome images using support vector machines. International Symposium on Signal Process- ing and Its Applications, 2007.
Piper and E. Granum. On fully automatic feature measurement for banded chromosome classifi- cation. Cytometry: The Journal of the International Society for Analytical Cytology, 1989.
10

---

Under review as a conference paper at ICLR 2023
Qin, Yulei, Juan Wen, Hao Zheng, Ning Song Yue-Min Zhu Linggian Wu Xiaolin Huang, Jie Yang, and Guang-Zhom Zhong Yang. Varifocal-net: chromosome classification approach using deep con- volutional networks. IEEE transactions medical imaging, 2019.
Schroff, Florian, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. IEEE conference on computer vision and paltern recognition, 2015.
Sharma, Monika, Oindrila Saha, Anand Sriraman, Ramya Hebbalaguppe, Lovekesh Vig. and Shirish Karande. Crowdsourcing for chromosome segmentation and deep classification. IEEE computer vision and pattern recognition workshops, pp. 34-41, 2017.
Sharma, Monika, and Lovekesh Vig. Application of artificial neural networks chromosome classification. International joint conference on neural networks (IJCNN), 2018.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arXiv:1409.1s.. 1556, 2014.
Stebbins and G. Ledyard. Variation and Evolution Plants. Columbia University Press, 1950.
Szegedy, Christian, Sergey loffe, Vincent Vanhoucke, and Alexander A. Alemi. Inception-v4, inception-resnet and the impact of residual connections learning. Thirty-first AAAI confer- ence on artificial intelligence, 2017.
L. V. d. Maaten and G. Hinton. Visualizing data using (-sne. machine learning research. 2008.
Vaswani, Ashish. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Lukasz Kaiser Aidan N. Gomez, and Illia Polosukhin. Attention all you need, neural information processing systems, 2017.
Weinberger, Kilian Q.and, and Lawrence K. Saul. Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research, 2009.
Wilson, D. Randall, and Tony R. Martinez. The general inefficiency of batch training for gradient descent learning. Neural networks, 2003.


Reviews:
[{"review_id": "cUCxA-Aus-", "reviewer": "Reviewer_qFAU", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a fully supervised classification pipeline consisting of contrastive learning with a triplet loss using ResNet, followed by ReduNet to modify the learnt features such that a nonlinear support vector machine can be used for classification of the features. The method is evaluated on one publicly available dataset of chromosome classification (24 classes).\n", "summary_of_the_review": "I have a few major issues with this submission: (1) the background section is basically taken over from a publication in 2020, so it is hard to know where this particular work is located in the current state of the art of chromosome classification; (2) the one dataset used for evaluation is very small, hence accuracy gains of 2% correspond to about 5 images, so it is hard to draw conclusions from the experiment and it cannot be expected that these results generalize to any other dataset, even more so, if they are not performed on multiple runs; (3) there are flaws in the presentation of the numerical results, the basis of the study.\n", "strength_and_weaknesses": "Exploiting small datasets is a very relevant task in biomedical applications, where annotations and labels are scarce and expensive. However this paper does not address how to use the little available data to the best extent, leaving a very small test set for a fully-supervised classification model, from which not many conclusions can be drawn with the expectation to general well to other datasets. An outdated background section makes it hard to place the proposed method within literature in this particular application and unclearities with the results make it even harder to draw conclusions from the performed experiments. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "R-80Xs_MhiX", "reviewer": "Reviewer_4brb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This manuscript wants to address chromosome classification with small datasets. Unlike some automatic chromosome classification methods, this approach is primarily based on learning discriminative future representations rather than only finding classification features in given data samples. The proposed method could be summarized as divided into three steps: the first step is to extract discriminative and meaningful features of chromosomes by utilizing ResNet with triplet loss. The second step optimizes features from the first step to obtain a linear discriminative representation via maximal coding rate reduction, which ensures the clusters representing different chromosome\ntypes are far away from each other while the same is close to each other in the cluster. The final step is to employ a classification method (such as SVM) to identify chromosomes. Then, some comparison experiments are conducted to evaluate the effectiveness of the proposed method. \nI think its contributions could be summarized as follows:\n1. The method proposes a representation learning-oriented approach, which forces the network to learn discriminative features rather than classification features in the small dataset. \n2. This paper combines some existing methods into an integrated chromosome classification method.", "summary_of_the_review": "The paper proposes a pipeline for the automatic classification of chromosomes, which integrates three components to learn discriminative representations. However, these components exist and are common in other tasks. These incremental improvements do not convince me to accept this paper. ", "strength_and_weaknesses": "Strengths:\n1. The proposed approach combines three components into an integrated method, and further comparison experiments demonstrate its superior performance compared with some other methods in chromosome classification tasks. \n2. The organization is well, and it is clear to readers. \n\nWeaknesses:\n1. The biggest weakness behind this paper is the lack of novelty. The main components of the proposed method are common and existing. \n2. The comparison experiments are not solid. The baseline methods are limited. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Bt7u50Mtbfy", "reviewer": "Reviewer_cfhg", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method to classify chromosome images consisting of three steps. First, a convolutional network (ResNet50) is trained with a triplet loss, second, the output chromosome representation is optimized via a ReduNet [Chan et al, 2021], and finally fed into an SVM with a Gaussian kernel. Both the first and second steps contribute to a representation that provides well-separated classes. The method does not require any image preprocessing. The experiments evaluate the method on a publicly available dataset.", "summary_of_the_review": "I would not recommend accepting the paper in its current form.\n\nThe proposed method seems to me too complicated given the small improvement of the results. I would find it useful if the experimental section went deeper into checking each part of the method against better and/or stronger baselines (PCA instead of ReduNet, comparison to ResNet50 trained with triplet loss with nearest neighbor using cosine distance, use of a more recent image feature extractor, such as EfficientNet or Swin). Also I would enjoy if the writing was less repetitive and clearer regarding what is new.\n\nMinor improvements:\n- \"for deferent classes\" -> \"for different classes\"\n- \"isproduced\" -> \"is produced\"\n- \"we uniform the input image as 224x224\" -> \"we resize the input image to 224x224\"", "strength_and_weaknesses": "Pros:\n- application to an important read-world challenge\n- the method does not require the chromosomes to be straightened (less preprocessing)\n\nCons:\n- the text is not clear enough (see below) and repetitive with the setting and method exposed multiple times\n- the paper explains prior art that I would expect the readers to be familiar with (triplet loss, accuracy/recall/precision/f-score), but also explains ReduNet, which I did not know and even wrongly assumed it was proposed in the paper\n- weak baselines (AlexNet, VGG-16), missing baselines (PCA instead of ReduNet)\n- the performance improvement does not seem very convincing to me since it is based on just several samples (13 for each autosome, 9.6 for each sexual chromosome given the 90:10 split) and the difference is small (~1.2pp over Lin et al and ~0.6pp over using a triplet loss + svm)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject