# Classification of Breast Cancer Histopathological Images Using DenseNet and Transfer Learning

Authors: Musa Adamu Wakili; Harisu Abdullahi Shehu; Md. Haidar Sharif; Md. Haris Uddin Sharif; Abubakar Umar; Huseyin Kusetogullari; Ibrahim Furkan Ince; Sahin Uyaver
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

Breast cancer is one of the most common invading cancers in women. Analyzing breast cancer is nontrivial and may lead to disagreements among experts. Although deep learning methods achieved an excellent performance in classification tasks including breast cancer histopathological images, the existing state-of-the-art methods are computationally expensive and may overfit due to extracting features from in-distribution images. In this paper, our contribution is mainly twofold. First, we perform a short survey on deep-learning-based models for classifying histopathological images to investigate the most popular and optimized training-testing ratios. Our findings reveal that the most popular training-testing ratio for histopathological image classification is 70%: 30%, whereas the best performance (e.g., accuracy) is achieved by using the training-testing ratio of 80%: 20% on an identical dataset. Second, we propose a method named DenTnet to classify breast cancer histopathological images chiefly. DenTnet utilizes the principle of transfer learning to solve the problem of extracting features from the same distribution using DenseNet as a backbone model. The proposed DenTnet method is shown to be superior in comparison to a number of leading deep learning methods in terms of detection accuracy (up to 99.28% on BreaKHis dataset deeming training-testing ratio of 80%: 20%) with good generalization ability and computational speed. The limitation of existing methods including the requirement of high computation and utilization of the same feature distribution is mitigated by dint of the DenTnet.


---

## Body

## 1. Introduction

Breast cancer is one of the most familiar invasive cancers in women worldwide. Nowadays, it is overtaking lung cancer as the world’s chiefly regularly diagnosed cancer [1]. The diagnosis of breast cancer in the early stages significantly decreases the mortality rate by allowing the choice of adequate treatment. With the onset of pattern recognition and machine learning, a good deal of handcrafted or engineered features-based studies have been proposed for classifying breast cancer histology images. In image classification, feature extraction is a cardinal process used to maximize the classification accuracy by minimizing the number of selected features [2–5]. Deep learning models have the power to automatically extract features, retrieve information, and take in the latest intellectual depictions of data. Thus, they can solve the problems of common feature extraction methods. The automated classification of breast cancer histopathological images is one of the important tasks in CAD (Computer-Aided Detection/Diagnosis) systems, and deep learning models play a remarkable role by detecting, classifying, and segmenting prime breast cancer histopathological images. Many researchers worldwide have invested appreciable efforts in developing robust computer-aided tools for the classification of breast cancer histopathological images using deep learning. At present, in this research arena, the most popular deep learning models proposed in the literature are based on CNNs [6–66].A pretrained CNN model, for example, DenseNet [67], utilizes dense connection between layers, reduces the number of parameters, strengthens propagation, and encourages feature reutilization. This improved parameter efficiency makes the network faster and easier to train. Nevertheless, a DenseNet [67] has an excessive connection, as all its layers have a direct connection to each other. Those lavish connections have been shown to decrease the computational and parameter efficiency of the network. In addition, features extracted by a neural network model stay in the same distribution. Therefore, the model might overfit as the features cannot be guaranteed to be sufficient enough. Besides, a CNN-training task demands a large number of training samples; otherwise, it leads to overfitting and reduces generalization ability. However, it is arduous to secure labeled breast cancer histopathological images, which severely limits the classification ability of CNN [27].On the other hand, the use of transfer learning can expand prior knowledge about data by including information from a different domain to target future data [68]. Consequently, it is a good idea to extract data from a related domain and then transfer those extracted data to the target domain. This way, resources can be saved and the efficiency of the model can be improved during training. A great number of breast cancer diagnosis methods based on transfer learning have been proposed and implemented by distinct researchers (e.g., [57–66]) to achieve state-of-the-art performance (e.g., ACC, AUC, PRS, RES, and F1S) on different datasets. Yet, the limitations of such performance indices, algorithmic assumptions, and computational complexities are indicating a further development of smart algorithms.In this paper, we aim to propose a novel neural-network-based approach called DenTnet (see Figure1) for classifying breast cancer histopathological images by taking the benefits of both DenseNet [67] and transfer learning [68]. To address the cross-domain learning problems, we employ the principle of transfer learning for transferring information from a related domain to the target domain. Our proposed DenTnet is anticipated to increase the accuracy of breast cancer histopathological images classification and accelerate the learning process. The DenTnet demonstrates better performance over its alternative CNN and/or transfer-learning-based methods (e.g., see Table 1) on the same dataset as well as training-testing ratio.Figure 1
Architecture of the proposed DenTnet.Table 1
Comparison of results of various methods using training-testing ratio of 80%: 20% on BreaKHis [33]. The best result is shown in bold.
YearMethodPRSRESF1SAUCACC (%)2020Togacar et al. [26]————97.56Parvin et al. [31]————91.25Man et al. [36]————91.442021Boumaraf et al. [63]————92.15Soumik et al. [60]————98.972022Liu et al. [172]————96.97Zerouaoui and Idri [56]————93.85Chattopadhyay et al. [174]————96.10DenTnet [ours]0.97000.98960.99480.9999.28To find the best performance scores of deep learning models for classifying histopathological images, contrasting training-testing ratios were applied for divergent models on the same dataset. What would be the most popular and/or optimized training-testing ratios to classify histopathological images considering existing state-of-the-art deep learning models? There exist many surveys enriched to sufficient contemporary methods and materials with systematic deep discussion of automatic classification of breast cancer histopathological images [68–72]. Nevertheless, to the best of our knowledge, the direct or indirect indication of this question was not reported in any of the previous studies. Henceforth, we perform a succinct survey to investigate this question. Our findings include that the most popular training-testing ratio for histopathological image classification is 70%: 30%, whereas the best performance (accuracy) is achieved by using the training-testing ratio of 80%: 20% on the identical dataset.In summary, the main contributions of this context are as follows:(i)
Determine the most popular and/or optimized training-testing ratios for classifying histopathological images using the existing state-of-the-art deep learning models.(ii)
Propose a novel approach named DenTnet that amalgamates both DenseNet [67] and transfer learning technique to classify breast cancer histopathological images. DenTnet is anticipated to achieve high accuracy and fasten the learning process due to its utilization of dense connections from its backbone architecture (i.e., DenseNet [67]).(iii)
Determine the generalization ability of DenTnet and the superiority measure considering nonparametric statistical tests.The rest of the paper is organized as follows: Section2 hints some preliminaries; Section 3 surveys briefly the existing deep models for histopathological image classification and reports our findings; Section 4 depicts the architecture of our proposed DenTnet and its implementation details; Section 5 demonstrates the experimental results and comparison on BreaKHis dataset [33]; Section 6 evaluates the generalization ability of DenTnet; Section 7 discusses nonparametric statistical tests, their reported results, and reasons for superiority along with few hints of further study; and Section 8 concludes the paper.


## 2. Preliminaries

Breast cancer is one of the oldest known kinds of cancer first found in Egypt [73]. It is caused by the uncontrolled growth and division of cells in the breast, whereby a mass of tissue called a tumor is created. Nowadays, it is one of the most terrifying cancers in women worldwide. For example, in 2020, there were 2.3 million women diagnosed with breast cancer and 685000 deaths globally [74]. Early detection of breast cancer can save many lives. Breast cancer can be diagnosed in view of histology and radiology images. The radiology images analysis can help to identify the areas, where the abnormality is located. However, they cannot be used to determine whether the area is cancerous [75]. On the other hand, a biopsy is an examination of tissue removed from a living body to discover the presence, cause, or extent of a disease (e.g., cancer). Biopsy is the only reliable way to make sure if an area is cancerous [76]. Upon completion of the biopsy, the diagnosis will be based on the qualification of the histopathologists who determine cancerous regions and malignancy degree [7, 75]. If the histopathologists are not well trained, the histopathology or biopsy report can lead to an incorrect diagnosis. Besides, there might be a lack of specialists, which may cause keeping the tissue samples for up to a few months. In addition, diagnoses made by unspecialized histopathologists are sometimes difficult to replicate. As if that were not enough of a problem, at times, even expert histopathologists tend to disagree with each other. Despite notable progress being reached by diagnostic imaging technologies, the final breast cancer grading and staging are still done by pathologists using visual inspection of histological samples under microscopes.As analyzing breast cancer is nontrivial and would get down to disagreements among experts, computerized and interdisciplinary systems can improve the accuracy of diagnostic results by reducing the processing time. The CAD can help to assist doctors in reading and interpreting medical images by locating and identifying possible abnormalities in the image [69]. It is proclaimed that the utilization of CAD to automatically classify histopathological images does not only improve the diagnostic efficiency with low cost but also provide doctors with more objective and accurate diagnosis results [77]. Consequently, there is an adamant demand for the CAD [78]. There exist several comprehensive surveys for CAD based methods in the literature. For example, Zebari et al. [71] provided a common description and analysis of existing CAD systems that are utilized in both machine learning and deep learning methods as well as their current state based on mammogram image modalities and classification methods. However, the existing breast cancer diagnosis models take issue with complexity, cost, human-dependency, and inaccuracy [73]. Furthermore, the limitation of datasets is another practical problem in this arena of research. In addition, every deep learning model demands a metric to judge its performance. Explicitly, performance evaluation metrics are the part and parcel of every deep learning model as they indicate progress indices.In the two following subsections, we discuss the commonly used datasets for classifying histopathological images and the performance evaluation metrics of various deep learning models.


### 2.1. Brief Description of Datasets

Accessing relevant images and datasets is one of the key challenges for image analysis researchers. Datasets and benchmarks enable validating and comparing methods for developing smarter algorithms. Recently, several datasets of breast cancer histopathology images have been released for this purpose. Figure2 shows a sample breast cancer histopathological image from BreaKHis [33] dataset of a patient who suffered from papillary carcinoma (malignant) with four magnification levels: (a) 40x, (b) 100x (c) 200x, and (d) 400x [79]. The following list of datasets has been used in the literature as incorporated in Table 2:(i)
BreaKHis [33] ⇒ It is considered as the most popular and clinical valued public breast cancer histopathological dataset. It consists of 7909 breast cancer histopathology images, 2480 benign and 5429 malignant samples, from 82 patients with different magnification factors (e.g., 40x, 100x, 200x, and 400x) [33].(ii)
Bioimaging2015 [122] ⇒ The Bioimaging2015 [122] dataset contained 249 microscopy training images and 36 microscopy testing images in total, equally distributed among the four classes.(iii)
ICIAR2018 [78] ⇒ This dataset, available as part of the BACH grand challenge [78], was an extended version of the Bioimaging2015 dataset [8, 122]. It contained 100 images in each of four categories (i.e., normal, benign, in situ carcinoma, and invasive carcinoma) [8].(iv)
BACH [78] ⇒ The database of BACH holds images obtained from ICIAR2018 Grand Challenge [78]. It consists of 400 images with equal distribution of normal (100), benign (100), in situ carcinoma (100), and invasive carcinoma (100). The high-resolution images are digitized with the same conditions and magnification factor of 200x. In this dataset, images have a fixed size of 2048×1536 pixels [175].(v)
TMA [99] ⇒ The TMA (Tissue MicroArray) database from Stanford University is a public resource with an access to 205161 images. All the whole-slide images have been scanned by a 20x magnification factor for the tissue and 40x for the cells [176].(vi)
Camelyon [97] ⇒ The Camelyon (cancer metastases in lymph nodes) was established based on a research challenge dataset competition in 2016. The Camelyon organizers trained CNNs on smaller datasets for classifying breast cancer in lymph nodes and prostate cancer biopsies. The training dataset consists of 270 whole-slide images; among them 160 are normal slides and 110 slides contain metastases [97].(vii)
PCam [121] ⇒ It is a modified version of the Patch Camelyon (PCam) dataset, which consists of 327680 microscopy images with 96×96-pixel sized patches extracted from the whole-slide images with a binary label hinting the presence of metastatic tissue [8].(viii)
HASHI [129] ⇒ Each image in the dataset of HASHI (high-throughput adaptive sampling for whole-slide histopathology image analysis) [129] has the size of 3002×2384 [161].(ix)
MIAS [85] ⇒ The Mammographic Image Analysis Society (MIAS) database of digital mammograms [85] contains 322 mammogram images, each of which has a size of 1024×1024 pixels with PGM format [59].(x)
INbreast [92] ⇒ The INbreast database has a total of 410 images collected from 115 cases (i.e., patients) indicating benign, malignant, and normal cases having sizes of 2560×3328 or 3328×4084 pixels. It contains 36 benign and 76 malignant masses [92].(xi)
DDSM [84] ⇒ The DDSM [84] dataset was collected by the expert team at the University of South Florida [84]. It contains 2620 scanned film mammography studies. Explicitly, it involves 2620 breast cases (i.e., patients) categorized in 43 different volumes with average size of 3000×4800 pixels [48].(xii)
CBIS-DDSM [128] ⇒ The CBIS-DDSM [128] is an updated version of the DDSM providing easily accessible data and improved region-of-interest segmentation [128, 146]. The CBIS-DDSM dataset comprises 2781 mammograms in the PNG format [49].(xiii)
CMTHis [37] ⇒ The CMTHis (Canine Mammary Tumor Histopathological Image) [37] dataset comprises 352 images acquired from 44 clinical cases of canine mammary tumors.(xiv)
FABCD [133] ⇒ The FABCD (Fully Annotated Breast Cancer Database) [133] consists of 21 annotated images of carcinomas and 19 images of benign tissue taken from 21 patients [130].(xv)
IICBU2008 [87] ⇒ The IICBU2008 (Image Informatics and Computational Biology Unit) malignant lymphoma dataset contains 374 H&E stained microscopy images captured using bright field microscopy [21].(xvi)
VLAD [136] ⇒ The VLAD (Vector of Locally Aggregated Descriptors) dataset [136] consists of 300 annotated images with resolution of 1280×960 [29].(xvii)
LSC [137] ⇒ The LSC (Lymphoma Subtype Classification) [137] dataset has been prepared by pathologists from different laboratories to create a real-world type cohort which contains a larger degree of stain and scanning variances [137]. It consists of 374 images with resolution of 1388×1040 [29].(xviii)
KimiaPath24 [126] ⇒ The official KimiaPath24 [126] dataset consists of a total of 23916 images for training and 1325 images for testing. It is publicly available. It shows various body parts with texture patterns [41].Figure 2
A sample breast cancer histopathological image [79] with four magnification levels of (a) 40x, (b) 100x, (c) 200x, and (d) 400x.Table 2
A succinct survey of deep-learning-based histopathological image classification methods. NA indicates either “not available” or “no answer” from the associated authors.
YearRefAimTechniqueDatasetSampleTraining (%)Testing (%)ResultPerformanceAUCACC2016Chan and Tuszynski [80]To predict tumor malignancy in breast cancerEmployed binarization, fractal dimension, SVMBreaKHis [33]79095050ACC of 97.90%, 16.50%, 16.50%, and 25.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA39.05%Spanhol et al. [33]To classify histopathological imagesEmployed CNN based on AlexNet [81]BreaKHis [33]79097030ACC of 90.0%, 88.4%, 84.6%, and 86.1% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA87.28%Bayram-oglu et al. [38]To classify breast cancer histopathology imagesEmployed single-task CNN and multitask CNNBreaKHis [33]79097030For single-task CNN, ACC of 83.08%, 83.17%, 84.63%, and 82.10%, obtained for 40x, 100x, 200x, and 400x magnification factors, respectively; accordingly, for multitask CNN, ACC of 81.87%, 83.39%, 82.56%, and 80.69%NA82.69%Abbas [77]To diagnose breast massesApplied SURF [82], LBPV [83]DDSM [84], MIAS [85]6004060Overall 92%, 84.20%, 91.50%, and 0.91 obtained for sensitivity, specificity, ACC, and AUC, respectively0.9191.50%2017Song et al. [21]To classify histopathology imagesEmployed a model of CNN, Fisher vector [86], SVMBreaKHis [33], IICBU2008 [87]82837030ACC of 94.42%, 89.49%, 87.25%, and 85.62% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA89.19%Wei et al. [22]To analyze tissue imagesEmployed a modification of GoogLeNet [88]BreaKHis [33]79097525ACC of 97.46%, 97.43%, 97.73%, and 97.74% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.59%Das et al. [23]To classify histopathology imagesEmployed GoogLeNet [88]BreaKHis [33]79098020ACC of 94.82%, 94.38%, 94.67%, and 93.49% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.34%Kahya et al. [89]To identify features of breast cancerEmployed dimensionality reduction, adaptive sparse SVMBreaKHis [33]79097030ACC of 94.97%, 93.62%, 94.54%, and 94.42% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.38%Song et al. [24]To classify histopathology images easilyEmployed CNN-based Fisher vector [86], SVMBreaKHis [33]79097030ACC of 90.02%, 88.90%, 86.90%, and 86.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.03%Gupta and Bhavsar [90]To classify histopathology images.Employed an integrated modelBreaKHis [33]79097030Average ACC of 88.09% and 88.40% obtained for image and patient levels, respectivelyNA88.25%Dhungel et al. [91]To analyze masses in mammogramsApplied multiscale deep belief netsINbreast [92]4106020The best results on the testing set with an ACC got 95% on manual and 91% on the minimal user intervention setup0.7691.03%Spanhol et al. [34]To classify breast cancer imagesUsing deep CNNBreaKHis [33]79007030ACC of 84.30%, 84.35%, 85.25% and 82.10% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA83.96%Han et al. [35]To study breast cancer multiclassificationEmployed class structure based CNNBreaKHis [33]79095050ACC of 95.80%, 96.90%, 96.70%, and 94.9% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA96.08%Sun and Binder [39]To assess performance of H&E stain dat.A comparative study among ResNet-50 [75], CaffeNet [93], and GoogLeNet [88]BreaKHis [33]79097030ACC of 85.75%, 87.03%, and 84.18% obtained for GoogLeNet [88], ResNet-50 [75], and CaffeNet [93], respectivelyNA85.65%Kaymak et al. [94]To organize breast cancer imagesBack-Propagation [95] and Radial Basis Neural Networks [96]176 images from a hospital1766535Overall ACC of 59.0% and 70.4% got from Back-Propagation [95] and Radial Basis [96], respectivelyNA64.70%Liu et al. [47]To detect cancer metastases in imagesEmployed a CNN architectureCamelyon16 [97]1106832An AUC of 97.60 (93.60, 100) obtained on par with Camelyon16 [97] test set performance0.9795.00%Zhi et al. [57]To diagnose breast cancer imagesEmployed a variation of VGGNet [98]BreaKHis [33]79098020ACC of 91.28%, 91.45%, 88.57%, and 84.58% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.97%Chang et al. [58]To solve the limited amount of training dataEmployed CNN model from Inception [88] family (e.g., Inception V3)BreaKHis [33]40177030ACC of 83.00% for benign class and 89.00% for malignant class. AUC of malignant was 93.00% and AUC of benign was also 93.00%0.9386.00%2018Jannesari et al. [6]To classify breast cancer imagesEmployed variations of Inception [88], ResNet [75]BreaKHis [33], 6402 images from TMA [99]143118515With ResNets ACC of 99.80%, 98.70%, 94.80%, and 96.40% obtained for four cancer types. Inception V2 with fine-tuning all layers got ACC of 94.10%0.9996.34%Bardou et al. [7]To classify breast cancer based on histology imagesEmployed CNN topology, data augmentationBreaKHis [33]79097030ACC of 98.33%, 97.12%, 97.85%, and 96.15% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.36%Kumar and Rao [9]To train CNN for using image classificationEmployed CNN topologyBreaKHis [33]79097030ACC of 85.52%, 83.60%, 84.84%, and 82.67% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.16%Das et al. [11]To classify breast histopathology imagesEmployed variation of CNN modelBreaKHis [33]79098020ACC of 89.52%, 89.06%, 88.84%, and 87.67% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.77%Nahid et al. [100]To classify biomedical breast cancer imagesEmployed Boltzmann machine [101], Tamura et al. features [102]BreaKHis [33]79097030ACC of 88.70%, 85.30%, 88.60%, and 88.40% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA87.75%Badejo et al. [103]To classify medical imagesEmployed local phase quantization, SVMBreaKHis [33]79097030ACC of 91.10%, 90.70%, 86.20%, and 84.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.08%Alireza-zadeh et al. [104]To arrange breast cancer imagesThreshold adjacency [105], quadratic analysis [106]BreaKHis [33]79097030ACC of 89.16%, 87.38%, 88.46%, and 86.68% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA87.92%Du et al. [13]To distribute breast cancer imagesEmployed AlexNet [81]BreaKHis [33]79097030ACC of 90.69%, 90.46%, 90.64%, and 90.96% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA90.69%Gandom-kar et al. [14]To model CNN for breast cancer image diagnosisEmployed a variation of ResNet [75] (e.g., ResNet152)BreaKHis [33]77867030ACC of 98.60%, 97.90%, 98.30%, and 97.60% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA98.10%Gupta and Bhavsar [15]To model CNN for breast cancer image diagnosisEmployed DenseNet [67], XGBoost classifier [107]BreaKHis [33]79097030ACC of 94.71%, 95.92%, 96.76%, and 89.11% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.12%Ben-hammou et al. [17]To study CNN for breast cancer imagesEmployed Inception V3 [88] moduleBreaKHis [33]79097030ACC of 87.05%, 82.80%, 85.75%, and 82.70% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.58%Morillo et al. [108]To label breast cancer imagesEmployed KAZE features [109]BreaKHis [33]79097030ACC of 86.15%, 80.70%, 77.95%, and 72.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.20%Chattoraj and Vishwakarma [110]To study breast carcinoma imagesZernike moments [111], entropies of Renyi [112], Yager [113]BreaKHis [33]79097030ACC of 87.7%, 85.8%, 88.0%, and 84.6% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA96.53%Sharma and Mehra [19]To analyze behavior of magnification independent breast cancerEmployed models of VGGNet [98] and ResNet [75] (e.g., VGG16, VGG19, and ResNet50)BreaKHis [33]79099010Pretrained VGG16 with logistic regression classifier showed the best performance with 92.60% ACC, 95.65% AUC, and 95.95% ACC precision score for 90%–10% training-testing data splitting0.9594.28%Zheng et al. [114]To study content-based image retrievalEmployed binarization encoding, Hamming distance [115]BreaKHis [33] and others163097030ACC of 47.00%, 40.00%, 40.00%, and 37.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA41.00%Cascianelli et al. [20]To study features extraction from imagesEmployed dimensionality reduction using CNNBreaKHis [33]79097525ACC of 84.00%, 88.20%, 87.00%, and 80.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.88%Mukkamala et al. [116]To study deep model for feature extractionEmployed PCANet [117]BreaKHis [33]79098020ACC of 96.12%, 97.41%, 90.99%, and 85.85% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.59%Mahraban Nejad et al. [51]To retrieve breast cancer imagesEmployed a variation of VGGNet [98], SVMBreaKHis [33]79099802An average ACC of 80.00% was demonstrated from BreaKHis [33]NA80.00%Rakhlin et al. [118]To analyze breast cancer imagesSeveral deep neural networks and gradient boosted trees classifierBACH [78]4007525For 4-class classification task ACC was 87.2% but for 2-class classification ACC was reported to be 93.8%0.9790.50%Almasni et al. [119]To detect breast massesApplied regional deep learning techniqueDDSM [84]6008020Distinguished between benign and malignant lesions with an overall ACC of 97%0.9697.00%2019Kassani et al. [8]To use deep learning for binary classification of breast histology imagesUsage of VGG19 [98], MobileNet [120], and DenseNet [67]BreaKHis [33], ICIAR2018 [78], PCam [121], Bioimaging2015 [122]85948713Multimodel method got better predictions than single classifiers and other algorithms with ACC of 98.13%, 95.00%, 94.64% and 83.10% obtained for BreaKHis [33], ICIAR2018 [78], PCam [121], and Bioimaging2015 [122], respectivelyNA92.72%Alom et al. [10]To classify breast cancer from histopathological imagesInception recurrent residual CNNBreaKHis [33], Bioimaging2015 [122]81587030From BreaKHis [33], ACC of 97.90%, 97.50%, 97.30%, and 97.40%, obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.9897.53%Nahid and Kong [12]To classify histopathological breast imagesEmployed RGB histogram [123] with CNNBreaKHis [33]79098515ACC of 95.00%, 96.60%, 93.500%, and 94.20% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.68%Jiang et al. [16]To use CNN for breast cancer histopathological imagesEmployed CNN, Squeeze-and-Excitation [124] based ResNet [75]BreaKHis [33]79097030The achieved accuracy between 98.87% and 99.34% for the binary classification as well as between 90.66% and 93.81% for the multiclass classification0.9995.67%Sudharshan et al. [18]To use instance learning for image sortingEmployed CNN-based multiple instance learning algorithmBreaKHis [33]79097030ACC of 86.59%, 84.98%, 83.47%, and 82.79% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.46%Gupta and Bhavsar [25]To segment breast cancer imagesEmployed ResNet [75] for multilayer feature extractionBreaKHis [33]79097030ACC of 88.37%, 90.29%, 90.54%, and 86.11% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.82%Vo et al. [125]To extract visual features from training imagesCombined weak classifiers into a stronger classifierBreaKHis [33], Bioimaging2015 [122]81948713ACC of 95.10%, 96.30%, 96.90%, and 93.80% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA95.56%Qi et al. [32]To organize breast cancer imagesEmployed a CNN network to complete the classification taskBreaKHis [33]79097030ACC of 91.48%, 92.20%, 93.01%, and 92.58% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.32%Talo [41]To detect and classify diseases in imagesDenseNet [67], ResNet [75] (e.g., DenseNet161, ResNet50)KimiaPath24 [126]252418020DenseNet161 pretrained and ResNet50 achieved ACC of 97.89% and 98.87% on grayscale and color images, respectivelyNA98.38%Li et al. [127]To detect invading component in cancer imagesConvolutional autoencoder-based contrast pattern mining framework361 samples of the breast cancer3619010ACC was taken into account. The overall ACC achieved was 76.00%, whereas 77.70% was presented for F1SNA76.00%Ragab et al. [44]To detect breast cancer from imagesAlexNet [81] and SVMDDSM [84], CBIS-DDSM [128]27817030The deep CNN presented an ACC of 73.6%, whereas the SVM demonstrated an ACC of 87.2%0.8873.60%Romero et al. [45]To study cancer imagesA modification of Inception module [88]HASHI [129]1514656337From deep learning networks, an overall ACC of 89.00% was demonstrated along with F1S of 90.00%0.9689.00%Minh et al. [46]To diagnose breast cancer imagesA modification of ResNet [75] and InceptionV3 [88]BACH [78]4007020ACC with 95% for 4 types of cancer classes and ACC with 97.5% for two combined groups of cancer0.9796.25%2020Stanitsas et al. [130]To visualize a health system for cliniciansEmployed region covariance [131], SVM, multiple instance learning [132]FABCD [133], BreaKHis [33]79497015ACC of 91.27% and 92.00% at the patient and image level, respectively0.9891.64%Togacar et al. [26]To analyze breast cancer images rapidlyEmployed a ResNet [75] architecture with attention modulesBreaKHis [33]79098020ACC of 97.99%, 97.84%, 98.51%, and 95.88% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.56%Asare et al. [134]To study breast cancer imagesEmployed self-training and self-paced learningBreaKHis [33]79097030ACC of 93.58%, 91.04%, 93.38%, and 91.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.25%Gour et al. [28]To diagnose breast cancer tumors imagesEmployed a modification of ResNet [75]BreaKHis [33]79097030ACC of 90.69%, 91.12%, 95.36%, and 90.24% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.9192.52%Li et al. [29]To grade pathological imagesEmployed a modification of Xception network [135]BreaKHis [33], VLAD [136], LSC [137]85836040ACC of 95.13%, 95.21%, 94.09%, and 91.42% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA93.96%Feng et al. [138]To allocate breast cancer imagesDeep neural-network-based manifold preserving autoencoder [139]BreaKHis [33]79097030ACC of 90.12%, 88.89%, 91.57%, and 90.25% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA90.53%Parvin and Mehedi Hasan [31]To study CNN models for cancer imagesLeNet [140], AlexNet [81], VGGNet [98], ResNet [75], Inception V3 [88]BreaKHis [33]79098020ACC of 89.00%, 92.00%, 94.00% and 90.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.8591.25%Carvalho et al. [141]To classify histological breast imagesEntropies of Shannon [142], Renyi [112], Tsallis [143]BreaKHis [33]49607030ACC of 95.40%, 94.70%, 97.60%, and 95.50% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.9995.80%Li et al. [144]To analyze breast cancer imagesEmployed global covariance pooling module [145]BreaKHis [33]79097030ACC of 96.00%, 96.16%, 98.01%, and 95.97% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.93%Man et al. [36]To classify cancer imagesUsage of generative adversarial networks, DenseNet [67]BreaKHis [33]79098020ACC of 97.72%, 96.19%, 86.66%, and 85.18% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA91.44%Kumar et al. [37]To classify human breast cancer and canine mammary tumorsEmployed a framework based on a variant of VGGNet [98] (e.g., VGGNet16) and SVMBreaKHis [33] and CMTHis [37]82617030For BreaKHis [33], ACC of 95.94%, 96.22%, 98.15%, and 94.41% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively; the same for CMTHis [37], ACC of 94.54%, 97.22%, 92.07%, and 82.84% obtained0.9596.93%Kaushal and Singla [40]To detect cancerous cells in images.Employed a CNN model of self-training and self-paced learningTotal 50 images of various patients509010ACC was taken into account. Estimation of the standard error of mean was approximately 0.81NA93.10%Hameed et al. [43]To use deep learning for classification of breast cancer imagesVariants of VGGNet [98] (e.g., fully trained VGG16, fine-tuned VGG16, fully trained VGG19, and fine-tuned VGG19 models)Breast cancer images: 675 for training and 170 for testing8458020The ensemble of fine-tuned VGG16 and VGG19 models offered sensitivity of 97.73% for carcinoma class and overall accuracy of 95.29%. It also offered an F1 score of 95.29%NA95.29%Alantari et al. [48]To detect breast lesions in digital X-ray mammogramsAdopted three deep CNN modelsINbreast [92], DDSM [84]10107020In INbreast [92] mean ACC of 89%, 93%, and 95% for CNN, ResNet50, and Inception-ResNet V2, respectively; 95%, 96%, and 98% for DDSM [146]0.9694.08%Zhang et al. [49]To classify breast massResNet [75], DenseNet [67], VGGNet [98]CBIS-DDSM [128], INbreast [92]31687030Overall ACC of 90.91% and 87.93% obtained from CBIS-DDSM [128] and INbreast [92], respectively0.9689.42%Hassan et al. [59]To classify breast cancer massesModification of AlexNet [22] and GoogLeNet [88]CBIS-DDSM [128], MIAS [85], INbreast [92], etc6007517With CBIS-DDSM [128] and INbreast [92] databases, the modified GoogLeNet achieved ACC of 98.46% and 92.5%, respectively0.9796.98%2021Li et al. [147]To use high-resolution info of imagesMultiview attention-guided multiple instance detection networkBreaKHis [33], BACH [78], PUIH [148]123297030Overall ACC of 94.87%, 91.32%, and 90.45% obtained from BreaKHis [33], BACH [78], and PUIH [148], respectively0.9992.21%Wang et al. [27]To divide breast cancer imagesEmployed a model of CNN and CapsNet [149]BreaKHis [33]79097030ACC of 92.71%, 94.52%, 94.03%, and 93.54% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA93.70%Albashish et al. [30]To analyze VGG16 [98]Employed a variation of VGGNet [98]BreaKHis [33]79099010ACC of 96%, 95.10%, and 87% obtained for polynomial SVM, Radial Basis SVM, and k-nearest neighbors, respectivelyNA92.70%Kundale et al. [150]To classify breast cancer from histology imagesEmployed SURF [82], DSIFT [151], linear coding [152]BreaKHis [33]79097030ACC of 93.35%, 93.86%, 93.73%, and 94.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA93.74%Attallah et al. [153]To classify breast cancer from histopathological imagesEmployed several deep learning techniques including autoencoder [139]BreaKHis [33], ICIAR2018 [78]79097030For BreaKHis [33], ACC of 99.03%, 99.53%, 98.08%, and 97.56% got for 40x, 100x, 200x, and 400x magnification factors, respectively; for ICIAR2018 [78], ACC was 97.93%NA98.43%Burçak et al. [154]To classify breast cancer histopathological imagesStochastic [155], Nesterov [156], Adaptive [157], RMSprop [158], AdaDelta [159], Adam [160]BreaKHis [33]79097030ACC was taken into account. The overall ACC of 97.00%, 97.00%, 96.00%, and 96.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA96.50%Hirra et al. [161]To label breast cancer imagesPatch-based deep belief network [162]HASHI [129]5845230Images from four different data samples achieved an accuracy of 86%NA86.00%Elmannai et al. [42]To extract eminent breast cancer image featuresA combination of two deep CNNsBACH [78]4006020The overall ACC for the subimage classification was 97.29% and for the carcinoma cases the sensitivity achieved was 99.58%NA97.29%Baker and Abu Qutaish [163]To segment breast cancer imagesClustering and global thresholding methodsBACH [78]4007030The maximum ACC obtained from classifiers and neural network using BACH [78] to detect breast cancerNA63.66%Soumik et al. [60]To classify breast cancer imagesEmployed Inception V3 [88]BreaKHis [33]79098020ACC of 99.50%, 98.90%, 98.96% and 98.51% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA98.97%Brancati et al. [50]To analyze gigapixel histopathological imagesEmployed CNN with a compressing path and a learning pathCamelyon16 [164], TUPAC16 [165]8926832AUC values of 0.698, 0.639, and 0.654 obtained for max-pooling, average pooling, and combined attention maps, respectively0.66NAMahmoud et al. [61]To classify breast cancer imagesEmployed transfer learningMammography images [166]75008020Maximum ACC of 97.80% was claimed by using the given dataset [166]. Sensitivity and specificity were estimatedNA94.45%Munien et al. [62]To classify breast cancer imagesEmployed EfficientNet [167]ICIAR2018 [78]4008515Overall ACC of 98.33% obtained from ICIAR2018 [78]. Sensitivity was also taken into accountNA98.33%Boumaraf et al. [63]To analyze breast cancer imagesEmployed ResNet [75] on ImageNet [168] imagesBreaKHis [33]79098020ACC of 94.49%, 93.27%, 91.29%, 89.56% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.15%Saber et al. [64]To detect breast cancerEmployed transfer learning techniqueMIAS [85]3228020Overall ACC, PRS, F1S, and AUC of 98.96%, 97.35%, 97.66%, and 0.995, respectively, got from MIAS [85]0.99598.96%2022Ameh Joseph et al. [169]To classify breast cancer imagesEmployed handcrafted features and dense layerBreaKHis [33]79099010ACC of 97.87% for 40x, 97.60% for 100x, 96.10% for 200x, and 96.84% for 400x demonstrated from BreaKHis [33]NA97.08%Reshma et al. [52]To detect breast cancerEmployed probabilistic transition rules with CNNBreaKHis [33]79099010ACC, PRS, RES, F1S, and GMN of 89.13%, 86.23%, 81.47%, 85.38%, and 85.17% demonstrated from BreaKHis [33]NA89.13%Huang et al. [53]To detect nuclei on breast cancerEmployed mask-region-based CNNH&E images of patients5378020PRS, RES, and F1S of 91.28%, 87.68%, and 89.44% demonstrated from the used datasetNA95.00%Chhipa et al. [170]To learn efficient representationsEmployed magnification prior contrastive similarityBreaKHis [33]79097030Maximum mean ACC of 97.04% and 97.81% were got from patient and image levels, respectively using BreaKHis [33]NA97.42%Zou et al. [171]To classify breast cancer imagesEmployed channel attention module with nondimensionality reductionBreaKHis [33], BACH [78]83099010Average ACC, PRS, RES, and F1S of 97.75%, 95.19%, 97.30%, and 96.30% obtained from BreaKHis [33], respectively. ACC of 85% got from BACH [78]NA91.37%Liu et al. [172]To classify breast cancer imagesEmployed autoencoder and Siamese frameworkBreaKHis [33]79098020Average ACC, PRS, RES, F1S, and RTM of 96.97%, 96.47%, 99.15%, 97.82%, and 335 seconds obtained from BreaKHis [33], respectivelyNA96.97%Jayandhi et al. [54]To diagnose breast cancerEmployed VGG [98] and SVMMIAS [85]3228020Maximum ACC of 98.67% obtained from MIAS [85]. Sensitivity and specificity were also calculatedNA98.67%Sharma and Kumar [55]To classify breast cancer imagesEmployed Xception [135] and SVMBreaKHis [33]20007525Average ACC, PRS, RES, F1S, and AUC of 95.58%, 95%, 95%, 95%, and 0.98 obtained from BreaKHis [33], respectively0.9895.58%Zerouaoui and Idri [56]To classify breast cancer imagesEmployed multilayer perceptron, DenseNet201 [67]BreaKHis [33] and othersNA8020ACC of 92.61%, 92%, 93.93%, and 91.73% on four magnification factors of BreaKHis [33]NA93.85%Soltane et al. [65]To classify breast cancer imagesEmployed ResNet [75]323 colored lymphoma images3235050A total of 27 misclassifications for 323 samples were claimed. PRS, RES, F1S, and Kappa score were estimatedNA91.6%Naik et al. [173]To analyze breast cancer imagesEmployed random forest, k-nearest neighbors, SVM699 whole-slide images6998020Random forest algorithm achieved better result for classifying benign and malignant images from 190 testing samplesNA98.2%Chattopadhyay et al. [174]To classify breast cancer imagesEmployed dense residual dual-shuffle attention networkBreaKHis [33]79098020Average ACC, PRS, RES, and F1S of 96.10%, 96.03%, 96.08%, and 96.02%, respectively, obtained from four different magnification levels of BreaKHis [33]NA96.10%Alruwaili and Gouda [66]To detect breast cancerEmployed the principle of transfer learning, ResNet [75]MIAS [85]3228020Best results for ACC, PRS, RES, F1S, and AUC of 89.5%, 89.5%, 90%, and 89.5% obtained from MIAS [85], respectivelyNA89.5%


### 2.2. Performance Evaluation Metrics

Performance evaluation of any deep learning model is an important task. An algorithm may give very pleasing results when evaluated using a metric (e.g.,ACC), but it may give poor results when evaluated against other metrics (e.g., F1S) [177]. Usually, we use classification accuracy to measure the performance of deep learning algorithms. But it is not enough to determine the model perfectly. For truly judge any deep learning algorithm, we can use nonidentical types of evaluation metrics including classification ACC, AUC, PRS, RES, F1S, RTM, and GMN.(i)
ACC⇒ It is normally defined in terms of error or inaccuracy [178]. It can be calculated using the following equation:(1)ACC=100tn+tptp+tn+fp+fn,wheretn is true negative, tp is true positive, fp is false positive, and fn is false negative. Sometimes, ACC and the percent correct classification (PCC) can be used interchangeably.(ii)
PRS⇒ Its best value is 100 and the worst value is just 0. It can be formulated using the following equation:(2)PRS=100tptp+fp.(iii)
RES⇒ It should ideally be 100 (the highest) for a good classifier. It can be calculated using the following equation:(3)RES=100tptp+fn.(iv)
AUC⇒ It is one of the most widely used metrics for evaluation [177–179]. The AUC of a classifier equals the probability that the classifier ranks a randomly chosen positive sample higher than a randomly chosen negative sample. The AUC varies in value from 0 to 1. If the predictions of a model are 100% wrong, then its AUC=0.00; but if its predictions are 100% correct, then its AUC=1.00.(v)
F1S⇒ It is the harmonic mean between precision and recall. It is also called the F-score or F-measure. It is used in deep learning [177]. It conveys the balance between the precision and the recall. It also tells us how many instances it classifies correctly. Its highest possible value is 1, which indicates perfect precision and recall. Its lowest possible value is 0, when either the precision or the recall is zero. It can be formulated as(4)F1S=21/PRS+1/RES=tptp+fp+fn/2,wherePRS is the number of correct positive results divided by the number of positive results predicted with the classifier and RES is the number of correct positive results divided by the number of all relevant samples.(vi)
RTM⇒ Estimating the RTM complexity of algorithms is mandatory for many applications (e.g., embedded real-time systems [180]). The optimization of the RTM complexity of algorithms in applications is highly expected. The total RTM can prove to be one of the most important determinative performance factors in many software-intensive systems.(vii)
GMN⇒ It indicates the central tendency or typical value of a set of numbers by considering the product of their values instead of using their sum. It can be used to attain a more accurate measure of returns than the mean or arithmetic mean or average. The GMN for any set of numbers x1,x2,x3,…,xm can be defined as(5)GMN=∏i=1mxi1/m=mx1x2x3⋯xm.(viii)
MCC⇒ The Matthews correlation coefficient (MCC) is used as a measure of the quality of binary classifications, introduced by biochemist Brian W. Matthews in 1975.(ix)
κ⇒ The metric of Cohen’s kappa κ can be used to evaluate binary classifications.


## 2.1. Brief Description of Datasets

Accessing relevant images and datasets is one of the key challenges for image analysis researchers. Datasets and benchmarks enable validating and comparing methods for developing smarter algorithms. Recently, several datasets of breast cancer histopathology images have been released for this purpose. Figure2 shows a sample breast cancer histopathological image from BreaKHis [33] dataset of a patient who suffered from papillary carcinoma (malignant) with four magnification levels: (a) 40x, (b) 100x (c) 200x, and (d) 400x [79]. The following list of datasets has been used in the literature as incorporated in Table 2:(i)
BreaKHis [33] ⇒ It is considered as the most popular and clinical valued public breast cancer histopathological dataset. It consists of 7909 breast cancer histopathology images, 2480 benign and 5429 malignant samples, from 82 patients with different magnification factors (e.g., 40x, 100x, 200x, and 400x) [33].(ii)
Bioimaging2015 [122] ⇒ The Bioimaging2015 [122] dataset contained 249 microscopy training images and 36 microscopy testing images in total, equally distributed among the four classes.(iii)
ICIAR2018 [78] ⇒ This dataset, available as part of the BACH grand challenge [78], was an extended version of the Bioimaging2015 dataset [8, 122]. It contained 100 images in each of four categories (i.e., normal, benign, in situ carcinoma, and invasive carcinoma) [8].(iv)
BACH [78] ⇒ The database of BACH holds images obtained from ICIAR2018 Grand Challenge [78]. It consists of 400 images with equal distribution of normal (100), benign (100), in situ carcinoma (100), and invasive carcinoma (100). The high-resolution images are digitized with the same conditions and magnification factor of 200x. In this dataset, images have a fixed size of 2048×1536 pixels [175].(v)
TMA [99] ⇒ The TMA (Tissue MicroArray) database from Stanford University is a public resource with an access to 205161 images. All the whole-slide images have been scanned by a 20x magnification factor for the tissue and 40x for the cells [176].(vi)
Camelyon [97] ⇒ The Camelyon (cancer metastases in lymph nodes) was established based on a research challenge dataset competition in 2016. The Camelyon organizers trained CNNs on smaller datasets for classifying breast cancer in lymph nodes and prostate cancer biopsies. The training dataset consists of 270 whole-slide images; among them 160 are normal slides and 110 slides contain metastases [97].(vii)
PCam [121] ⇒ It is a modified version of the Patch Camelyon (PCam) dataset, which consists of 327680 microscopy images with 96×96-pixel sized patches extracted from the whole-slide images with a binary label hinting the presence of metastatic tissue [8].(viii)
HASHI [129] ⇒ Each image in the dataset of HASHI (high-throughput adaptive sampling for whole-slide histopathology image analysis) [129] has the size of 3002×2384 [161].(ix)
MIAS [85] ⇒ The Mammographic Image Analysis Society (MIAS) database of digital mammograms [85] contains 322 mammogram images, each of which has a size of 1024×1024 pixels with PGM format [59].(x)
INbreast [92] ⇒ The INbreast database has a total of 410 images collected from 115 cases (i.e., patients) indicating benign, malignant, and normal cases having sizes of 2560×3328 or 3328×4084 pixels. It contains 36 benign and 76 malignant masses [92].(xi)
DDSM [84] ⇒ The DDSM [84] dataset was collected by the expert team at the University of South Florida [84]. It contains 2620 scanned film mammography studies. Explicitly, it involves 2620 breast cases (i.e., patients) categorized in 43 different volumes with average size of 3000×4800 pixels [48].(xii)
CBIS-DDSM [128] ⇒ The CBIS-DDSM [128] is an updated version of the DDSM providing easily accessible data and improved region-of-interest segmentation [128, 146]. The CBIS-DDSM dataset comprises 2781 mammograms in the PNG format [49].(xiii)
CMTHis [37] ⇒ The CMTHis (Canine Mammary Tumor Histopathological Image) [37] dataset comprises 352 images acquired from 44 clinical cases of canine mammary tumors.(xiv)
FABCD [133] ⇒ The FABCD (Fully Annotated Breast Cancer Database) [133] consists of 21 annotated images of carcinomas and 19 images of benign tissue taken from 21 patients [130].(xv)
IICBU2008 [87] ⇒ The IICBU2008 (Image Informatics and Computational Biology Unit) malignant lymphoma dataset contains 374 H&E stained microscopy images captured using bright field microscopy [21].(xvi)
VLAD [136] ⇒ The VLAD (Vector of Locally Aggregated Descriptors) dataset [136] consists of 300 annotated images with resolution of 1280×960 [29].(xvii)
LSC [137] ⇒ The LSC (Lymphoma Subtype Classification) [137] dataset has been prepared by pathologists from different laboratories to create a real-world type cohort which contains a larger degree of stain and scanning variances [137]. It consists of 374 images with resolution of 1388×1040 [29].(xviii)
KimiaPath24 [126] ⇒ The official KimiaPath24 [126] dataset consists of a total of 23916 images for training and 1325 images for testing. It is publicly available. It shows various body parts with texture patterns [41].Figure 2
A sample breast cancer histopathological image [79] with four magnification levels of (a) 40x, (b) 100x, (c) 200x, and (d) 400x.Table 2
A succinct survey of deep-learning-based histopathological image classification methods. NA indicates either “not available” or “no answer” from the associated authors.
YearRefAimTechniqueDatasetSampleTraining (%)Testing (%)ResultPerformanceAUCACC2016Chan and Tuszynski [80]To predict tumor malignancy in breast cancerEmployed binarization, fractal dimension, SVMBreaKHis [33]79095050ACC of 97.90%, 16.50%, 16.50%, and 25.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA39.05%Spanhol et al. [33]To classify histopathological imagesEmployed CNN based on AlexNet [81]BreaKHis [33]79097030ACC of 90.0%, 88.4%, 84.6%, and 86.1% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA87.28%Bayram-oglu et al. [38]To classify breast cancer histopathology imagesEmployed single-task CNN and multitask CNNBreaKHis [33]79097030For single-task CNN, ACC of 83.08%, 83.17%, 84.63%, and 82.10%, obtained for 40x, 100x, 200x, and 400x magnification factors, respectively; accordingly, for multitask CNN, ACC of 81.87%, 83.39%, 82.56%, and 80.69%NA82.69%Abbas [77]To diagnose breast massesApplied SURF [82], LBPV [83]DDSM [84], MIAS [85]6004060Overall 92%, 84.20%, 91.50%, and 0.91 obtained for sensitivity, specificity, ACC, and AUC, respectively0.9191.50%2017Song et al. [21]To classify histopathology imagesEmployed a model of CNN, Fisher vector [86], SVMBreaKHis [33], IICBU2008 [87]82837030ACC of 94.42%, 89.49%, 87.25%, and 85.62% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA89.19%Wei et al. [22]To analyze tissue imagesEmployed a modification of GoogLeNet [88]BreaKHis [33]79097525ACC of 97.46%, 97.43%, 97.73%, and 97.74% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.59%Das et al. [23]To classify histopathology imagesEmployed GoogLeNet [88]BreaKHis [33]79098020ACC of 94.82%, 94.38%, 94.67%, and 93.49% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.34%Kahya et al. [89]To identify features of breast cancerEmployed dimensionality reduction, adaptive sparse SVMBreaKHis [33]79097030ACC of 94.97%, 93.62%, 94.54%, and 94.42% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.38%Song et al. [24]To classify histopathology images easilyEmployed CNN-based Fisher vector [86], SVMBreaKHis [33]79097030ACC of 90.02%, 88.90%, 86.90%, and 86.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.03%Gupta and Bhavsar [90]To classify histopathology images.Employed an integrated modelBreaKHis [33]79097030Average ACC of 88.09% and 88.40% obtained for image and patient levels, respectivelyNA88.25%Dhungel et al. [91]To analyze masses in mammogramsApplied multiscale deep belief netsINbreast [92]4106020The best results on the testing set with an ACC got 95% on manual and 91% on the minimal user intervention setup0.7691.03%Spanhol et al. [34]To classify breast cancer imagesUsing deep CNNBreaKHis [33]79007030ACC of 84.30%, 84.35%, 85.25% and 82.10% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA83.96%Han et al. [35]To study breast cancer multiclassificationEmployed class structure based CNNBreaKHis [33]79095050ACC of 95.80%, 96.90%, 96.70%, and 94.9% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA96.08%Sun and Binder [39]To assess performance of H&E stain dat.A comparative study among ResNet-50 [75], CaffeNet [93], and GoogLeNet [88]BreaKHis [33]79097030ACC of 85.75%, 87.03%, and 84.18% obtained for GoogLeNet [88], ResNet-50 [75], and CaffeNet [93], respectivelyNA85.65%Kaymak et al. [94]To organize breast cancer imagesBack-Propagation [95] and Radial Basis Neural Networks [96]176 images from a hospital1766535Overall ACC of 59.0% and 70.4% got from Back-Propagation [95] and Radial Basis [96], respectivelyNA64.70%Liu et al. [47]To detect cancer metastases in imagesEmployed a CNN architectureCamelyon16 [97]1106832An AUC of 97.60 (93.60, 100) obtained on par with Camelyon16 [97] test set performance0.9795.00%Zhi et al. [57]To diagnose breast cancer imagesEmployed a variation of VGGNet [98]BreaKHis [33]79098020ACC of 91.28%, 91.45%, 88.57%, and 84.58% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.97%Chang et al. [58]To solve the limited amount of training dataEmployed CNN model from Inception [88] family (e.g., Inception V3)BreaKHis [33]40177030ACC of 83.00% for benign class and 89.00% for malignant class. AUC of malignant was 93.00% and AUC of benign was also 93.00%0.9386.00%2018Jannesari et al. [6]To classify breast cancer imagesEmployed variations of Inception [88], ResNet [75]BreaKHis [33], 6402 images from TMA [99]143118515With ResNets ACC of 99.80%, 98.70%, 94.80%, and 96.40% obtained for four cancer types. Inception V2 with fine-tuning all layers got ACC of 94.10%0.9996.34%Bardou et al. [7]To classify breast cancer based on histology imagesEmployed CNN topology, data augmentationBreaKHis [33]79097030ACC of 98.33%, 97.12%, 97.85%, and 96.15% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.36%Kumar and Rao [9]To train CNN for using image classificationEmployed CNN topologyBreaKHis [33]79097030ACC of 85.52%, 83.60%, 84.84%, and 82.67% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.16%Das et al. [11]To classify breast histopathology imagesEmployed variation of CNN modelBreaKHis [33]79098020ACC of 89.52%, 89.06%, 88.84%, and 87.67% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.77%Nahid et al. [100]To classify biomedical breast cancer imagesEmployed Boltzmann machine [101], Tamura et al. features [102]BreaKHis [33]79097030ACC of 88.70%, 85.30%, 88.60%, and 88.40% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA87.75%Badejo et al. [103]To classify medical imagesEmployed local phase quantization, SVMBreaKHis [33]79097030ACC of 91.10%, 90.70%, 86.20%, and 84.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.08%Alireza-zadeh et al. [104]To arrange breast cancer imagesThreshold adjacency [105], quadratic analysis [106]BreaKHis [33]79097030ACC of 89.16%, 87.38%, 88.46%, and 86.68% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA87.92%Du et al. [13]To distribute breast cancer imagesEmployed AlexNet [81]BreaKHis [33]79097030ACC of 90.69%, 90.46%, 90.64%, and 90.96% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA90.69%Gandom-kar et al. [14]To model CNN for breast cancer image diagnosisEmployed a variation of ResNet [75] (e.g., ResNet152)BreaKHis [33]77867030ACC of 98.60%, 97.90%, 98.30%, and 97.60% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA98.10%Gupta and Bhavsar [15]To model CNN for breast cancer image diagnosisEmployed DenseNet [67], XGBoost classifier [107]BreaKHis [33]79097030ACC of 94.71%, 95.92%, 96.76%, and 89.11% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.12%Ben-hammou et al. [17]To study CNN for breast cancer imagesEmployed Inception V3 [88] moduleBreaKHis [33]79097030ACC of 87.05%, 82.80%, 85.75%, and 82.70% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.58%Morillo et al. [108]To label breast cancer imagesEmployed KAZE features [109]BreaKHis [33]79097030ACC of 86.15%, 80.70%, 77.95%, and 72.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.20%Chattoraj and Vishwakarma [110]To study breast carcinoma imagesZernike moments [111], entropies of Renyi [112], Yager [113]BreaKHis [33]79097030ACC of 87.7%, 85.8%, 88.0%, and 84.6% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA96.53%Sharma and Mehra [19]To analyze behavior of magnification independent breast cancerEmployed models of VGGNet [98] and ResNet [75] (e.g., VGG16, VGG19, and ResNet50)BreaKHis [33]79099010Pretrained VGG16 with logistic regression classifier showed the best performance with 92.60% ACC, 95.65% AUC, and 95.95% ACC precision score for 90%–10% training-testing data splitting0.9594.28%Zheng et al. [114]To study content-based image retrievalEmployed binarization encoding, Hamming distance [115]BreaKHis [33] and others163097030ACC of 47.00%, 40.00%, 40.00%, and 37.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA41.00%Cascianelli et al. [20]To study features extraction from imagesEmployed dimensionality reduction using CNNBreaKHis [33]79097525ACC of 84.00%, 88.20%, 87.00%, and 80.30% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.88%Mukkamala et al. [116]To study deep model for feature extractionEmployed PCANet [117]BreaKHis [33]79098020ACC of 96.12%, 97.41%, 90.99%, and 85.85% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.59%Mahraban Nejad et al. [51]To retrieve breast cancer imagesEmployed a variation of VGGNet [98], SVMBreaKHis [33]79099802An average ACC of 80.00% was demonstrated from BreaKHis [33]NA80.00%Rakhlin et al. [118]To analyze breast cancer imagesSeveral deep neural networks and gradient boosted trees classifierBACH [78]4007525For 4-class classification task ACC was 87.2% but for 2-class classification ACC was reported to be 93.8%0.9790.50%Almasni et al. [119]To detect breast massesApplied regional deep learning techniqueDDSM [84]6008020Distinguished between benign and malignant lesions with an overall ACC of 97%0.9697.00%2019Kassani et al. [8]To use deep learning for binary classification of breast histology imagesUsage of VGG19 [98], MobileNet [120], and DenseNet [67]BreaKHis [33], ICIAR2018 [78], PCam [121], Bioimaging2015 [122]85948713Multimodel method got better predictions than single classifiers and other algorithms with ACC of 98.13%, 95.00%, 94.64% and 83.10% obtained for BreaKHis [33], ICIAR2018 [78], PCam [121], and Bioimaging2015 [122], respectivelyNA92.72%Alom et al. [10]To classify breast cancer from histopathological imagesInception recurrent residual CNNBreaKHis [33], Bioimaging2015 [122]81587030From BreaKHis [33], ACC of 97.90%, 97.50%, 97.30%, and 97.40%, obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.9897.53%Nahid and Kong [12]To classify histopathological breast imagesEmployed RGB histogram [123] with CNNBreaKHis [33]79098515ACC of 95.00%, 96.60%, 93.500%, and 94.20% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.68%Jiang et al. [16]To use CNN for breast cancer histopathological imagesEmployed CNN, Squeeze-and-Excitation [124] based ResNet [75]BreaKHis [33]79097030The achieved accuracy between 98.87% and 99.34% for the binary classification as well as between 90.66% and 93.81% for the multiclass classification0.9995.67%Sudharshan et al. [18]To use instance learning for image sortingEmployed CNN-based multiple instance learning algorithmBreaKHis [33]79097030ACC of 86.59%, 84.98%, 83.47%, and 82.79% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA84.46%Gupta and Bhavsar [25]To segment breast cancer imagesEmployed ResNet [75] for multilayer feature extractionBreaKHis [33]79097030ACC of 88.37%, 90.29%, 90.54%, and 86.11% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA88.82%Vo et al. [125]To extract visual features from training imagesCombined weak classifiers into a stronger classifierBreaKHis [33], Bioimaging2015 [122]81948713ACC of 95.10%, 96.30%, 96.90%, and 93.80% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA95.56%Qi et al. [32]To organize breast cancer imagesEmployed a CNN network to complete the classification taskBreaKHis [33]79097030ACC of 91.48%, 92.20%, 93.01%, and 92.58% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.32%Talo [41]To detect and classify diseases in imagesDenseNet [67], ResNet [75] (e.g., DenseNet161, ResNet50)KimiaPath24 [126]252418020DenseNet161 pretrained and ResNet50 achieved ACC of 97.89% and 98.87% on grayscale and color images, respectivelyNA98.38%Li et al. [127]To detect invading component in cancer imagesConvolutional autoencoder-based contrast pattern mining framework361 samples of the breast cancer3619010ACC was taken into account. The overall ACC achieved was 76.00%, whereas 77.70% was presented for F1SNA76.00%Ragab et al. [44]To detect breast cancer from imagesAlexNet [81] and SVMDDSM [84], CBIS-DDSM [128]27817030The deep CNN presented an ACC of 73.6%, whereas the SVM demonstrated an ACC of 87.2%0.8873.60%Romero et al. [45]To study cancer imagesA modification of Inception module [88]HASHI [129]1514656337From deep learning networks, an overall ACC of 89.00% was demonstrated along with F1S of 90.00%0.9689.00%Minh et al. [46]To diagnose breast cancer imagesA modification of ResNet [75] and InceptionV3 [88]BACH [78]4007020ACC with 95% for 4 types of cancer classes and ACC with 97.5% for two combined groups of cancer0.9796.25%2020Stanitsas et al. [130]To visualize a health system for cliniciansEmployed region covariance [131], SVM, multiple instance learning [132]FABCD [133], BreaKHis [33]79497015ACC of 91.27% and 92.00% at the patient and image level, respectively0.9891.64%Togacar et al. [26]To analyze breast cancer images rapidlyEmployed a ResNet [75] architecture with attention modulesBreaKHis [33]79098020ACC of 97.99%, 97.84%, 98.51%, and 95.88% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA97.56%Asare et al. [134]To study breast cancer imagesEmployed self-training and self-paced learningBreaKHis [33]79097030ACC of 93.58%, 91.04%, 93.38%, and 91.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.25%Gour et al. [28]To diagnose breast cancer tumors imagesEmployed a modification of ResNet [75]BreaKHis [33]79097030ACC of 90.69%, 91.12%, 95.36%, and 90.24% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.9192.52%Li et al. [29]To grade pathological imagesEmployed a modification of Xception network [135]BreaKHis [33], VLAD [136], LSC [137]85836040ACC of 95.13%, 95.21%, 94.09%, and 91.42% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA93.96%Feng et al. [138]To allocate breast cancer imagesDeep neural-network-based manifold preserving autoencoder [139]BreaKHis [33]79097030ACC of 90.12%, 88.89%, 91.57%, and 90.25% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA90.53%Parvin and Mehedi Hasan [31]To study CNN models for cancer imagesLeNet [140], AlexNet [81], VGGNet [98], ResNet [75], Inception V3 [88]BreaKHis [33]79098020ACC of 89.00%, 92.00%, 94.00% and 90.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.8591.25%Carvalho et al. [141]To classify histological breast imagesEntropies of Shannon [142], Renyi [112], Tsallis [143]BreaKHis [33]49607030ACC of 95.40%, 94.70%, 97.60%, and 95.50% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively0.9995.80%Li et al. [144]To analyze breast cancer imagesEmployed global covariance pooling module [145]BreaKHis [33]79097030ACC of 96.00%, 96.16%, 98.01%, and 95.97% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA94.93%Man et al. [36]To classify cancer imagesUsage of generative adversarial networks, DenseNet [67]BreaKHis [33]79098020ACC of 97.72%, 96.19%, 86.66%, and 85.18% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA91.44%Kumar et al. [37]To classify human breast cancer and canine mammary tumorsEmployed a framework based on a variant of VGGNet [98] (e.g., VGGNet16) and SVMBreaKHis [33] and CMTHis [37]82617030For BreaKHis [33], ACC of 95.94%, 96.22%, 98.15%, and 94.41% obtained for 40x, 100x, 200x, and 400x magnification factors, respectively; the same for CMTHis [37], ACC of 94.54%, 97.22%, 92.07%, and 82.84% obtained0.9596.93%Kaushal and Singla [40]To detect cancerous cells in images.Employed a CNN model of self-training and self-paced learningTotal 50 images of various patients509010ACC was taken into account. Estimation of the standard error of mean was approximately 0.81NA93.10%Hameed et al. [43]To use deep learning for classification of breast cancer imagesVariants of VGGNet [98] (e.g., fully trained VGG16, fine-tuned VGG16, fully trained VGG19, and fine-tuned VGG19 models)Breast cancer images: 675 for training and 170 for testing8458020The ensemble of fine-tuned VGG16 and VGG19 models offered sensitivity of 97.73% for carcinoma class and overall accuracy of 95.29%. It also offered an F1 score of 95.29%NA95.29%Alantari et al. [48]To detect breast lesions in digital X-ray mammogramsAdopted three deep CNN modelsINbreast [92], DDSM [84]10107020In INbreast [92] mean ACC of 89%, 93%, and 95% for CNN, ResNet50, and Inception-ResNet V2, respectively; 95%, 96%, and 98% for DDSM [146]0.9694.08%Zhang et al. [49]To classify breast massResNet [75], DenseNet [67], VGGNet [98]CBIS-DDSM [128], INbreast [92]31687030Overall ACC of 90.91% and 87.93% obtained from CBIS-DDSM [128] and INbreast [92], respectively0.9689.42%Hassan et al. [59]To classify breast cancer massesModification of AlexNet [22] and GoogLeNet [88]CBIS-DDSM [128], MIAS [85], INbreast [92], etc6007517With CBIS-DDSM [128] and INbreast [92] databases, the modified GoogLeNet achieved ACC of 98.46% and 92.5%, respectively0.9796.98%2021Li et al. [147]To use high-resolution info of imagesMultiview attention-guided multiple instance detection networkBreaKHis [33], BACH [78], PUIH [148]123297030Overall ACC of 94.87%, 91.32%, and 90.45% obtained from BreaKHis [33], BACH [78], and PUIH [148], respectively0.9992.21%Wang et al. [27]To divide breast cancer imagesEmployed a model of CNN and CapsNet [149]BreaKHis [33]79097030ACC of 92.71%, 94.52%, 94.03%, and 93.54% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA93.70%Albashish et al. [30]To analyze VGG16 [98]Employed a variation of VGGNet [98]BreaKHis [33]79099010ACC of 96%, 95.10%, and 87% obtained for polynomial SVM, Radial Basis SVM, and k-nearest neighbors, respectivelyNA92.70%Kundale et al. [150]To classify breast cancer from histology imagesEmployed SURF [82], DSIFT [151], linear coding [152]BreaKHis [33]79097030ACC of 93.35%, 93.86%, 93.73%, and 94.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA93.74%Attallah et al. [153]To classify breast cancer from histopathological imagesEmployed several deep learning techniques including autoencoder [139]BreaKHis [33], ICIAR2018 [78]79097030For BreaKHis [33], ACC of 99.03%, 99.53%, 98.08%, and 97.56% got for 40x, 100x, 200x, and 400x magnification factors, respectively; for ICIAR2018 [78], ACC was 97.93%NA98.43%Burçak et al. [154]To classify breast cancer histopathological imagesStochastic [155], Nesterov [156], Adaptive [157], RMSprop [158], AdaDelta [159], Adam [160]BreaKHis [33]79097030ACC was taken into account. The overall ACC of 97.00%, 97.00%, 96.00%, and 96.00% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA96.50%Hirra et al. [161]To label breast cancer imagesPatch-based deep belief network [162]HASHI [129]5845230Images from four different data samples achieved an accuracy of 86%NA86.00%Elmannai et al. [42]To extract eminent breast cancer image featuresA combination of two deep CNNsBACH [78]4006020The overall ACC for the subimage classification was 97.29% and for the carcinoma cases the sensitivity achieved was 99.58%NA97.29%Baker and Abu Qutaish [163]To segment breast cancer imagesClustering and global thresholding methodsBACH [78]4007030The maximum ACC obtained from classifiers and neural network using BACH [78] to detect breast cancerNA63.66%Soumik et al. [60]To classify breast cancer imagesEmployed Inception V3 [88]BreaKHis [33]79098020ACC of 99.50%, 98.90%, 98.96% and 98.51% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA98.97%Brancati et al. [50]To analyze gigapixel histopathological imagesEmployed CNN with a compressing path and a learning pathCamelyon16 [164], TUPAC16 [165]8926832AUC values of 0.698, 0.639, and 0.654 obtained for max-pooling, average pooling, and combined attention maps, respectively0.66NAMahmoud et al. [61]To classify breast cancer imagesEmployed transfer learningMammography images [166]75008020Maximum ACC of 97.80% was claimed by using the given dataset [166]. Sensitivity and specificity were estimatedNA94.45%Munien et al. [62]To classify breast cancer imagesEmployed EfficientNet [167]ICIAR2018 [78]4008515Overall ACC of 98.33% obtained from ICIAR2018 [78]. Sensitivity was also taken into accountNA98.33%Boumaraf et al. [63]To analyze breast cancer imagesEmployed ResNet [75] on ImageNet [168] imagesBreaKHis [33]79098020ACC of 94.49%, 93.27%, 91.29%, 89.56% obtained for 40x, 100x, 200x, and 400x magnification factors, respectivelyNA92.15%Saber et al. [64]To detect breast cancerEmployed transfer learning techniqueMIAS [85]3228020Overall ACC, PRS, F1S, and AUC of 98.96%, 97.35%, 97.66%, and 0.995, respectively, got from MIAS [85]0.99598.96%2022Ameh Joseph et al. [169]To classify breast cancer imagesEmployed handcrafted features and dense layerBreaKHis [33]79099010ACC of 97.87% for 40x, 97.60% for 100x, 96.10% for 200x, and 96.84% for 400x demonstrated from BreaKHis [33]NA97.08%Reshma et al. [52]To detect breast cancerEmployed probabilistic transition rules with CNNBreaKHis [33]79099010ACC, PRS, RES, F1S, and GMN of 89.13%, 86.23%, 81.47%, 85.38%, and 85.17% demonstrated from BreaKHis [33]NA89.13%Huang et al. [53]To detect nuclei on breast cancerEmployed mask-region-based CNNH&E images of patients5378020PRS, RES, and F1S of 91.28%, 87.68%, and 89.44% demonstrated from the used datasetNA95.00%Chhipa et al. [170]To learn efficient representationsEmployed magnification prior contrastive similarityBreaKHis [33]79097030Maximum mean ACC of 97.04% and 97.81% were got from patient and image levels, respectively using BreaKHis [33]NA97.42%Zou et al. [171]To classify breast cancer imagesEmployed channel attention module with nondimensionality reductionBreaKHis [33], BACH [78]83099010Average ACC, PRS, RES, and F1S of 97.75%, 95.19%, 97.30%, and 96.30% obtained from BreaKHis [33], respectively. ACC of 85% got from BACH [78]NA91.37%Liu et al. [172]To classify breast cancer imagesEmployed autoencoder and Siamese frameworkBreaKHis [33]79098020Average ACC, PRS, RES, F1S, and RTM of 96.97%, 96.47%, 99.15%, 97.82%, and 335 seconds obtained from BreaKHis [33], respectivelyNA96.97%Jayandhi et al. [54]To diagnose breast cancerEmployed VGG [98] and SVMMIAS [85]3228020Maximum ACC of 98.67% obtained from MIAS [85]. Sensitivity and specificity were also calculatedNA98.67%Sharma and Kumar [55]To classify breast cancer imagesEmployed Xception [135] and SVMBreaKHis [33]20007525Average ACC, PRS, RES, F1S, and AUC of 95.58%, 95%, 95%, 95%, and 0.98 obtained from BreaKHis [33], respectively0.9895.58%Zerouaoui and Idri [56]To classify breast cancer imagesEmployed multilayer perceptron, DenseNet201 [67]BreaKHis [33] and othersNA8020ACC of 92.61%, 92%, 93.93%, and 91.73% on four magnification factors of BreaKHis [33]NA93.85%Soltane et al. [65]To classify breast cancer imagesEmployed ResNet [75]323 colored lymphoma images3235050A total of 27 misclassifications for 323 samples were claimed. PRS, RES, F1S, and Kappa score were estimatedNA91.6%Naik et al. [173]To analyze breast cancer imagesEmployed random forest, k-nearest neighbors, SVM699 whole-slide images6998020Random forest algorithm achieved better result for classifying benign and malignant images from 190 testing samplesNA98.2%Chattopadhyay et al. [174]To classify breast cancer imagesEmployed dense residual dual-shuffle attention networkBreaKHis [33]79098020Average ACC, PRS, RES, and F1S of 96.10%, 96.03%, 96.08%, and 96.02%, respectively, obtained from four different magnification levels of BreaKHis [33]NA96.10%Alruwaili and Gouda [66]To detect breast cancerEmployed the principle of transfer learning, ResNet [75]MIAS [85]3228020Best results for ACC, PRS, RES, F1S, and AUC of 89.5%, 89.5%, 90%, and 89.5% obtained from MIAS [85], respectivelyNA89.5%


## 2.2. Performance Evaluation Metrics

Performance evaluation of any deep learning model is an important task. An algorithm may give very pleasing results when evaluated using a metric (e.g.,ACC), but it may give poor results when evaluated against other metrics (e.g., F1S) [177]. Usually, we use classification accuracy to measure the performance of deep learning algorithms. But it is not enough to determine the model perfectly. For truly judge any deep learning algorithm, we can use nonidentical types of evaluation metrics including classification ACC, AUC, PRS, RES, F1S, RTM, and GMN.(i)
ACC⇒ It is normally defined in terms of error or inaccuracy [178]. It can be calculated using the following equation:(1)ACC=100tn+tptp+tn+fp+fn,wheretn is true negative, tp is true positive, fp is false positive, and fn is false negative. Sometimes, ACC and the percent correct classification (PCC) can be used interchangeably.(ii)
PRS⇒ Its best value is 100 and the worst value is just 0. It can be formulated using the following equation:(2)PRS=100tptp+fp.(iii)
RES⇒ It should ideally be 100 (the highest) for a good classifier. It can be calculated using the following equation:(3)RES=100tptp+fn.(iv)
AUC⇒ It is one of the most widely used metrics for evaluation [177–179]. The AUC of a classifier equals the probability that the classifier ranks a randomly chosen positive sample higher than a randomly chosen negative sample. The AUC varies in value from 0 to 1. If the predictions of a model are 100% wrong, then its AUC=0.00; but if its predictions are 100% correct, then its AUC=1.00.(v)
F1S⇒ It is the harmonic mean between precision and recall. It is also called the F-score or F-measure. It is used in deep learning [177]. It conveys the balance between the precision and the recall. It also tells us how many instances it classifies correctly. Its highest possible value is 1, which indicates perfect precision and recall. Its lowest possible value is 0, when either the precision or the recall is zero. It can be formulated as(4)F1S=21/PRS+1/RES=tptp+fp+fn/2,wherePRS is the number of correct positive results divided by the number of positive results predicted with the classifier and RES is the number of correct positive results divided by the number of all relevant samples.(vi)
RTM⇒ Estimating the RTM complexity of algorithms is mandatory for many applications (e.g., embedded real-time systems [180]). The optimization of the RTM complexity of algorithms in applications is highly expected. The total RTM can prove to be one of the most important determinative performance factors in many software-intensive systems.(vii)
GMN⇒ It indicates the central tendency or typical value of a set of numbers by considering the product of their values instead of using their sum. It can be used to attain a more accurate measure of returns than the mean or arithmetic mean or average. The GMN for any set of numbers x1,x2,x3,…,xm can be defined as(5)GMN=∏i=1mxi1/m=mx1x2x3⋯xm.(viii)
MCC⇒ The Matthews correlation coefficient (MCC) is used as a measure of the quality of binary classifications, introduced by biochemist Brian W. Matthews in 1975.(ix)
κ⇒ The metric of Cohen’s kappa κ can be used to evaluate binary classifications.


## 3. A Succinct Survey of State of the Art

This section deals with a summary of existing studies apposite for the classification of breast cancer histopathological images followed by a short discussion and our findings.


### 3.1. Summary of Previous Studies

Table2 provides a short summary of previous studies carried out to classify breast cancer from images. Experimental results of miscellaneous deep models in the literature on publicly available datasets demonstrated various degrees of accurate cancer prediction scores. However, as AUC and ACC are the most important metrics for breast cancer histopathological images classification [49], the experimental results in Table 2 take them into account as the performance indices.


### 3.2. Key Techniques and Challenges

The CNNs can be regarded as a variant of the standard neural networks. Instead of using fully connected hidden layers, the CNNs introduce the structure of a special network, which comprises so-called alternating convolution and pooling layers. They were first introduced for overcoming known problems of fully connected deep neural networks when handling high dimensionality structured inputs, such as images or speech. From Table2, it is noticeable that CNNs have become state-of-the-art solutions for breast cancer histology images classification. However, there are still challenges even when using the CNN-based approaches to classify pathological breast cancer images [16], as given below:(i)
Risk of overfitting⇒ The number of parameters of CNN increases rapidly depending on how large the network is, which may lead to poor learning.(ii)
Being cost-intensive⇒ To get a huge number of labeled breast cancer images is very expensive.(iii)
Huge training data⇒ CNNs need to be trained using a lot of images, which might not be easy to find considering that collecting real-world data is a tedious and expensive process.(iv)
Performance degradation⇒ Various hyperparameters have a significant influence on the performance of the CNN model. The model’s parameters need to be tuned properly to achieve a desirable result [75], which usually is not an easy task.(v)
Employment difficulty⇒ In the process of training CNN model, it is usually inevitable to rearrange the learning rate parameters to get a better performance. This makes it arduous for the algorithm to use in real-life applications by nonexpert users [181].Many methods had been proposed in the literature considering the aforementioned challenges. In 2012, AlexNet [81] architecture was introduced for ImageNet Challenge having error rate of 16%. Later various variations of AlexNet [81] with denser network were introduced. Both AlexNet [81] and VGGNet [98] were the pioneering works that demonstrated the potential of deep neural networks [182]. AlexNet was designed by Alex Krizhevsky [81]. It contained 8 layers; the first 5 were convolutional layers, some of them followed by max-pooling layers, and the last 3 were fully connected layers [81]. It was the first large-scale CNN architecture that did well on ImageNet [183] classification. AlexNet [81] was the winner of the ILSVRC [183] classification, the benchmark in 2012. Nevertheless, it was not very deep. SqueezeNet [184] was proposed to create a smaller neural network with fewer parameters that could be easily fit into computer memory and transmitted over a computer network. It achieved AlexNet [81] level accuracy on ImageNet with 50x fewer parameters. It was compressed to less than 0.5 MB (510x smaller than AlexNet [81]) with model compression techniques. The VGG [98] is a deep CNN used to classify images. The VGG19 is a variant of VGG which consists of 19 layers (i.e., 16 convolution layers and 3 fully connected layers, in addition to 5 max-pooling layers and 1 SoftMax layer) [98]. There exist many variants of VGG [98] (e.g., VGG11, VGG16, VGG19, etc.). VGG19 has 19.6 billion FLOPs (floating point operations per second). VGG [98] is easy to implement but slow to train. Nowadays, many deep-learning-based methods are implemented on influential backbone networks; among them, both DenseNet [67] and ResNet [75] are very popular. Due to the longer path between the input layer and the output layer, the information vanishes before reaching its destination. DenseNet [67] was developed to minimize this effect. The key base element of ResNet [75] is the residual block. DenseNet [67] concentrates on making the deep learning networks move even deeper as well as simultaneously making them well organized to train by applying shorter connections among layers. In short, ResNet [75] adopts summation, whereas DenseNet [67] deals with concatenation. Yet, the dense concatenation of DenseNet [67] creates a challenge of demanding high GPU (Graphics Processing Unit) memory and more training time [182]. On the other hand, the identity shortcut that balances training in ResNet [75] curbs its representation dimensions [182]. Compendiously, there is a dilemma in the alternative between ResNet [75] and DenseNet [67] for many applications in terms of performance and GPU resources [182].


### 3.3. Our Findings

Although various deep learning models in Table2 often achieved pretty good scores of AUC and ACC, the models demand a large amount of data but breast cancer diagnosis always suffers from a lack of data. To adopt artificial data is a tentative solution of this issue, but the determination of the best hyperparameters is extremely difficult. Besides efficient deep learning models, the datasets themselves have some limitations, for example, overinterpretation, which cannot be diagnosed using typical evaluation methods based on the ACC of the model. Deep learning models trained on popular datasets (e.g., BreaKHis [33]) may suffer from overinterpretation. In overinterpretation, deep learning models make confident predictions based on details that do not make any sense to humans (e.g., promiscuous patterns and image borders). When deep learning models are trained on datasets, they can make apparently authentic predictions based on both meaningful and meaningless subtle signals. This effect, eventually, can reduce the overall classification performance of deep models. Most probably, this is one of the reasons why any state-of-the-art deep learning model in the literature for classifying breast cancer histopathological images (see Table 2) could not show an ACC of 100%.In addition, the training-testing ratio can regulate the performance of a deep model for image classification. We wish to determine the most popular and/or optimized training-testing ratios for classifying histopathological images using Table2. To this end, we have calculated the usage frequency of the training-testing ratio (i.e., percentage of the number of papers that used the same ratio) by considering data in Table 2 and the following equation:(6)Usagefrequency%=Totalnumberofpapersusedatraining−testingratio100Sumofpapersbothusedandunusedthesametraining−testingratio.Figure3 demonstrates the frequency of usage of training-testing ratio considering data in Table 2. From Figure 3, it is noticeable that the most popular training-testing ratio for histopathological image classification is 70%: 30%. The second-best used training-testing ratio is 80%: 20%, followed by 90%: 10%, 75%: 25%, 50%: 50%, and so on. Figure 4 presents the GMN of ACC for the most frequently used training-testing ratios considering data in Table 2. It shows a different history; in terms of ACC, the rate of 80%: 20% became the best option for the training-testing ratio to classify histopathological images. Explicitly, the GMN of ACC formed like a Gaussian shaped curve and the ratio of 80%: 20% owned its highest peak. To cut a long story short, by considering ACC, the training-testing ratio of 80%: 20% became the finest and the optimal choice for classifying histopathological images.Figure 3
Determination of the most popular training-testing ratios using data from Table2.Figure 4
GMN of ACC for the most popular training-testing ratios deeming data from Table2.


## 3.1. Summary of Previous Studies

Table2 provides a short summary of previous studies carried out to classify breast cancer from images. Experimental results of miscellaneous deep models in the literature on publicly available datasets demonstrated various degrees of accurate cancer prediction scores. However, as AUC and ACC are the most important metrics for breast cancer histopathological images classification [49], the experimental results in Table 2 take them into account as the performance indices.


## 3.2. Key Techniques and Challenges

The CNNs can be regarded as a variant of the standard neural networks. Instead of using fully connected hidden layers, the CNNs introduce the structure of a special network, which comprises so-called alternating convolution and pooling layers. They were first introduced for overcoming known problems of fully connected deep neural networks when handling high dimensionality structured inputs, such as images or speech. From Table2, it is noticeable that CNNs have become state-of-the-art solutions for breast cancer histology images classification. However, there are still challenges even when using the CNN-based approaches to classify pathological breast cancer images [16], as given below:(i)
Risk of overfitting⇒ The number of parameters of CNN increases rapidly depending on how large the network is, which may lead to poor learning.(ii)
Being cost-intensive⇒ To get a huge number of labeled breast cancer images is very expensive.(iii)
Huge training data⇒ CNNs need to be trained using a lot of images, which might not be easy to find considering that collecting real-world data is a tedious and expensive process.(iv)
Performance degradation⇒ Various hyperparameters have a significant influence on the performance of the CNN model. The model’s parameters need to be tuned properly to achieve a desirable result [75], which usually is not an easy task.(v)
Employment difficulty⇒ In the process of training CNN model, it is usually inevitable to rearrange the learning rate parameters to get a better performance. This makes it arduous for the algorithm to use in real-life applications by nonexpert users [181].Many methods had been proposed in the literature considering the aforementioned challenges. In 2012, AlexNet [81] architecture was introduced for ImageNet Challenge having error rate of 16%. Later various variations of AlexNet [81] with denser network were introduced. Both AlexNet [81] and VGGNet [98] were the pioneering works that demonstrated the potential of deep neural networks [182]. AlexNet was designed by Alex Krizhevsky [81]. It contained 8 layers; the first 5 were convolutional layers, some of them followed by max-pooling layers, and the last 3 were fully connected layers [81]. It was the first large-scale CNN architecture that did well on ImageNet [183] classification. AlexNet [81] was the winner of the ILSVRC [183] classification, the benchmark in 2012. Nevertheless, it was not very deep. SqueezeNet [184] was proposed to create a smaller neural network with fewer parameters that could be easily fit into computer memory and transmitted over a computer network. It achieved AlexNet [81] level accuracy on ImageNet with 50x fewer parameters. It was compressed to less than 0.5 MB (510x smaller than AlexNet [81]) with model compression techniques. The VGG [98] is a deep CNN used to classify images. The VGG19 is a variant of VGG which consists of 19 layers (i.e., 16 convolution layers and 3 fully connected layers, in addition to 5 max-pooling layers and 1 SoftMax layer) [98]. There exist many variants of VGG [98] (e.g., VGG11, VGG16, VGG19, etc.). VGG19 has 19.6 billion FLOPs (floating point operations per second). VGG [98] is easy to implement but slow to train. Nowadays, many deep-learning-based methods are implemented on influential backbone networks; among them, both DenseNet [67] and ResNet [75] are very popular. Due to the longer path between the input layer and the output layer, the information vanishes before reaching its destination. DenseNet [67] was developed to minimize this effect. The key base element of ResNet [75] is the residual block. DenseNet [67] concentrates on making the deep learning networks move even deeper as well as simultaneously making them well organized to train by applying shorter connections among layers. In short, ResNet [75] adopts summation, whereas DenseNet [67] deals with concatenation. Yet, the dense concatenation of DenseNet [67] creates a challenge of demanding high GPU (Graphics Processing Unit) memory and more training time [182]. On the other hand, the identity shortcut that balances training in ResNet [75] curbs its representation dimensions [182]. Compendiously, there is a dilemma in the alternative between ResNet [75] and DenseNet [67] for many applications in terms of performance and GPU resources [182].


## 3.3. Our Findings

Although various deep learning models in Table2 often achieved pretty good scores of AUC and ACC, the models demand a large amount of data but breast cancer diagnosis always suffers from a lack of data. To adopt artificial data is a tentative solution of this issue, but the determination of the best hyperparameters is extremely difficult. Besides efficient deep learning models, the datasets themselves have some limitations, for example, overinterpretation, which cannot be diagnosed using typical evaluation methods based on the ACC of the model. Deep learning models trained on popular datasets (e.g., BreaKHis [33]) may suffer from overinterpretation. In overinterpretation, deep learning models make confident predictions based on details that do not make any sense to humans (e.g., promiscuous patterns and image borders). When deep learning models are trained on datasets, they can make apparently authentic predictions based on both meaningful and meaningless subtle signals. This effect, eventually, can reduce the overall classification performance of deep models. Most probably, this is one of the reasons why any state-of-the-art deep learning model in the literature for classifying breast cancer histopathological images (see Table 2) could not show an ACC of 100%.In addition, the training-testing ratio can regulate the performance of a deep model for image classification. We wish to determine the most popular and/or optimized training-testing ratios for classifying histopathological images using Table2. To this end, we have calculated the usage frequency of the training-testing ratio (i.e., percentage of the number of papers that used the same ratio) by considering data in Table 2 and the following equation:(6)Usagefrequency%=Totalnumberofpapersusedatraining−testingratio100Sumofpapersbothusedandunusedthesametraining−testingratio.Figure3 demonstrates the frequency of usage of training-testing ratio considering data in Table 2. From Figure 3, it is noticeable that the most popular training-testing ratio for histopathological image classification is 70%: 30%. The second-best used training-testing ratio is 80%: 20%, followed by 90%: 10%, 75%: 25%, 50%: 50%, and so on. Figure 4 presents the GMN of ACC for the most frequently used training-testing ratios considering data in Table 2. It shows a different history; in terms of ACC, the rate of 80%: 20% became the best option for the training-testing ratio to classify histopathological images. Explicitly, the GMN of ACC formed like a Gaussian shaped curve and the ratio of 80%: 20% owned its highest peak. To cut a long story short, by considering ACC, the training-testing ratio of 80%: 20% became the finest and the optimal choice for classifying histopathological images.Figure 3
Determination of the most popular training-testing ratios using data from Table2.Figure 4
GMN of ACC for the most popular training-testing ratios deeming data from Table2.


## 4. Methods and Materials

This section explains in detail our proposed DenTnet model and its implementation. Figure5 demonstrates a general flowchart of our methodology to classify breast cancer histopathological images automatically.Figure 5
Flowchart of our methodology to classify breast cancer histopathological images.


### 4.1. Architecture of Our Proposed DenTnet

The architecture of our proposed DenTnet is shown in Figure1, which consists of four different blocks, namely, the input volume, training from scratch, transfer learning, and fusion and recognition.


#### 4.1.1. Input Volume

The input is a 3D RGB (three-dimensional red, green, and blue) image with a size of224×224, that is, 224×224×3.


#### 4.1.2. Training from Scratch

Initially, features are extracted from the input images by feeding the input to the convolutional layer. The convolution (conv) layers contain a set of filters (or kernels) parameters, which are learned throughout the training. The size of the filters is usually smaller than the actual image, where each filter convolves with the image and creates an activation map. Thereafter, the pooling layer progressively decreases the spatial size of the representation for reducing the number of parameters in the network. Instead of differentiable functions such as sigmoid and tanh, the network utilizes the ReLU as an activation function. Finally, the extracted features or the output of the last layer from the training from scratch block is then amalgamated with the features extracted from the transfer learning approach. Figure1 includes the design of the DenseNet [67] architecture used to extract the feature using the learning-from-scratch approach.


#### 4.1.3. Transfer Learning

In transfer learning, given that a domainD consists of feature space X and a marginal probability distribution PX, where X = x1,x2,…,xn∈X, and a task T consists of a label space Y and an objective predictive function f: X⟶Y, the corresponding label fx of a new instance x is predicted by function f, where the new tasks denoted by T = Y, fx are learned from the training data consisting of pairs xi and yi, where xi∈X and yi∈Y. When utilizing the learning-from-scratch approach, the extracted features stay in the same distribution. To solve this problem, we amalgamated both learning-from-scratch and the transfer learning approach. The learned parameters are further fine-tuned by retraining the extracted features. This is anticipated to expand the prior knowledge of the network about the data, which might improve the efficiency of the model during training, thereby accelerating the learning speed and also increasing the accuracy of the model. As shown in Figure 1, there is a connection between the blocks of the input volume and transfer learning. The transfer learning approach extracted features from the ImageNet [168] weights. The weight is the parameters (including trainable and nontrainable) learned from the ImageNet [168] dataset. Since transfer learning involves transferring knowledge from one domain to another, we have utilized the ImageNet weight as the models developed in the ImageNet [168] classification competition are measured against each other for performance. Henceforth, the ImageNet weight provides a measure of how good a model is for classification. Besides, the ImageNet weight has already showed a markedly high accuracy [185]. The extracted features are then used by the network before being passed to the fusion and recognition block, where the features are amalgamated with the extracted features from the learning-from-scratch block for recognition.


#### 4.1.4. Fusion and Recognition

The extracted features based on the ImageNet weights are then amalgamated with the features extracted by the block of training from scratch. A global average pooling is performed. Dropout technique helps to prevent a model from overfitting. It is used with dense fully connected layers. The fully connected layer compiles the data extracted by previous layers to form the final output. The last step passes the features through the fully connected layer, which then uses SoftMax to classify the class of the input images.


### 4.2. Implementation Details

#### 4.2.1. Data Preparation

We have adopted data augmentation, stain normalization, and image normalization strategies to optimize the training process. Hereby, we have explained each of them briefly.


#### 4.2.2. Data Augmentation

Due to the limited size of the input samples, the training of our DenTnet was prone to overfitting, which caused low detection rate. One solution to alleviate this issue was the data augmentation, which generated more training data from the existing training set. Dissimilar data augmentation techniques (e.g., horizontal flipping, rotating, and zooming) were applied to datasets for creating more training samples.


#### 4.2.3. Stain Normalization

The breast cancer tissue slices are stained by H&E to differentiate between nuclei stained with purple color and other tissue structures stained with pink and red color to help pathologists analyze the shape of nuclei, density, variability, and overall tissue structure [186]. The H&E staining variability between acquired images exists due to the different staining protocols, scanners, and raw materials. This is a common problem with histological image analysis. Therefore, stain normalization of H&E-stained histology slides was a key step for reducing the color variation and obtaining a better color consistency prior to feeding input images into the DenTnet architecture. Different techniques are available for stain normalization in histological images. We have considered Macenko technique [187] due to its promising performance in many studies to standardize the color intensity of the tissue. This technique was based on a singular value decomposition. A logarithmic function was used to adaptively transform color concentration of the original histopathological image into its optical density (OD) image as OD=−logI/I0, where OD hints the matrix of optical density values, I belongs to the image intensity in red-green-blue space, and I0 addresses the illuminating intensity incident on the histological sample.


#### 4.2.4. Intensity Normalization

Intensity normalization was another important preprocessing step. Its primary aim was to get the same range of values for each input image before feeding to the DenTnet. It also speeded up the convergence of DenTnet. Input images were normalized to the standard normal distribution by min-max normalization (i.e., using one of the most popular ways to normalize data) to the intensity range of [0, 1], which can be computed as(7)xnormalized=x−xminxmax−xmin,where x, xmin, and xmax indicate pixel, minimum, and maximum intensity values of the input image, respectively.


#### 4.2.5. Hardware and Software Requirements

DenTnet was implemented using the TensorFlow and Keras framework [188, 189] and coded in Python using Jupyter Notebook on a Kaggle Private Kernel. The experiment was performed on a machine with the following configuration: Intel® Xeon® CPU @ 2.30 GHz with 16 CPU Cores, 16 GB RAM, and NVIDIA Tesla P100 GPU. We implemented and trained everything on the cloud using Kaggle GPU hours.


#### 4.2.6. Training and Testing Setup

The dataset was divided in a 80%: 20% ratio, where 80% was used for training and the remaining 20% was used for testing. The data used for testing were kept isolated from the training set and never seen by the model during training. To evaluate the images classification, we have computed the recognition rate at the image level over the two different classes: (i) correctly classified images and (ii) the total number of images in the test set.


#### 4.2.7. Training Procedure

In the training of a neural network, a measure of error is required to compute the error between the targeted output and the computed output of training data known as the loss function. An optimization algorithm is needed to minimize this function. We have considered Adam optimizer [190] with numerical stability constant epsilon = None, decay = 0.0, and AMSGrad = True. Table 3 presents the hyperparameter values of the proposed deep learning model. Learning rate (also referred to as step size) signifies the proportion to which weights are updated. A smaller value (e.g., 0.000001) slows down the learning process during training, whereas a larger value (e.g., 0.400) results in faster learning. We have considered a learning rate of 0.001. The exponential decay rates of the first and second moments were estimated to be 0.60 and 0.90, respectively. To update the weights, the number of epochs was set to 50 with 3222 steps per epoch and a batch size of 32. For the BreaKHis [33] dataset, we had a training sample of 103104 images, with 12288 validation samples and 697 testing samples. The training process used 10-fold cross-validation, where one of the samples was used to validate the data and the remaining 9 samples were used to train the DenTnet model. The fully connected layer used 1024 filters with a dropout rate of 0.50. Finally, the last layer used two filters with a SoftMax layer to classify the image into two classes (e.g., benign and malignant). We have used categorical cross-entropy as the objective function to quantify the difference between two probability distributions. The whole training process took more than 4 hours for the breast cancer tissue images.Table 3
List of hyperparameter values for the proposed deep learning model.
ModelHyperparametersBeta_1Beta_2Learning rateEpochBatch sizeEpsilonDecayAMSGradDenTnet0.600.900.0015032None0.0True


## 4.1. Architecture of Our Proposed DenTnet

The architecture of our proposed DenTnet is shown in Figure1, which consists of four different blocks, namely, the input volume, training from scratch, transfer learning, and fusion and recognition.


### 4.1.1. Input Volume

The input is a 3D RGB (three-dimensional red, green, and blue) image with a size of224×224, that is, 224×224×3.


### 4.1.2. Training from Scratch

Initially, features are extracted from the input images by feeding the input to the convolutional layer. The convolution (conv) layers contain a set of filters (or kernels) parameters, which are learned throughout the training. The size of the filters is usually smaller than the actual image, where each filter convolves with the image and creates an activation map. Thereafter, the pooling layer progressively decreases the spatial size of the representation for reducing the number of parameters in the network. Instead of differentiable functions such as sigmoid and tanh, the network utilizes the ReLU as an activation function. Finally, the extracted features or the output of the last layer from the training from scratch block is then amalgamated with the features extracted from the transfer learning approach. Figure1 includes the design of the DenseNet [67] architecture used to extract the feature using the learning-from-scratch approach.


### 4.1.3. Transfer Learning

In transfer learning, given that a domainD consists of feature space X and a marginal probability distribution PX, where X = x1,x2,…,xn∈X, and a task T consists of a label space Y and an objective predictive function f: X⟶Y, the corresponding label fx of a new instance x is predicted by function f, where the new tasks denoted by T = Y, fx are learned from the training data consisting of pairs xi and yi, where xi∈X and yi∈Y. When utilizing the learning-from-scratch approach, the extracted features stay in the same distribution. To solve this problem, we amalgamated both learning-from-scratch and the transfer learning approach. The learned parameters are further fine-tuned by retraining the extracted features. This is anticipated to expand the prior knowledge of the network about the data, which might improve the efficiency of the model during training, thereby accelerating the learning speed and also increasing the accuracy of the model. As shown in Figure 1, there is a connection between the blocks of the input volume and transfer learning. The transfer learning approach extracted features from the ImageNet [168] weights. The weight is the parameters (including trainable and nontrainable) learned from the ImageNet [168] dataset. Since transfer learning involves transferring knowledge from one domain to another, we have utilized the ImageNet weight as the models developed in the ImageNet [168] classification competition are measured against each other for performance. Henceforth, the ImageNet weight provides a measure of how good a model is for classification. Besides, the ImageNet weight has already showed a markedly high accuracy [185]. The extracted features are then used by the network before being passed to the fusion and recognition block, where the features are amalgamated with the extracted features from the learning-from-scratch block for recognition.


### 4.1.4. Fusion and Recognition

The extracted features based on the ImageNet weights are then amalgamated with the features extracted by the block of training from scratch. A global average pooling is performed. Dropout technique helps to prevent a model from overfitting. It is used with dense fully connected layers. The fully connected layer compiles the data extracted by previous layers to form the final output. The last step passes the features through the fully connected layer, which then uses SoftMax to classify the class of the input images.


## 4.1.1. Input Volume

The input is a 3D RGB (three-dimensional red, green, and blue) image with a size of224×224, that is, 224×224×3.


## 4.1.2. Training from Scratch

Initially, features are extracted from the input images by feeding the input to the convolutional layer. The convolution (conv) layers contain a set of filters (or kernels) parameters, which are learned throughout the training. The size of the filters is usually smaller than the actual image, where each filter convolves with the image and creates an activation map. Thereafter, the pooling layer progressively decreases the spatial size of the representation for reducing the number of parameters in the network. Instead of differentiable functions such as sigmoid and tanh, the network utilizes the ReLU as an activation function. Finally, the extracted features or the output of the last layer from the training from scratch block is then amalgamated with the features extracted from the transfer learning approach. Figure1 includes the design of the DenseNet [67] architecture used to extract the feature using the learning-from-scratch approach.


## 4.1.3. Transfer Learning

In transfer learning, given that a domainD consists of feature space X and a marginal probability distribution PX, where X = x1,x2,…,xn∈X, and a task T consists of a label space Y and an objective predictive function f: X⟶Y, the corresponding label fx of a new instance x is predicted by function f, where the new tasks denoted by T = Y, fx are learned from the training data consisting of pairs xi and yi, where xi∈X and yi∈Y. When utilizing the learning-from-scratch approach, the extracted features stay in the same distribution. To solve this problem, we amalgamated both learning-from-scratch and the transfer learning approach. The learned parameters are further fine-tuned by retraining the extracted features. This is anticipated to expand the prior knowledge of the network about the data, which might improve the efficiency of the model during training, thereby accelerating the learning speed and also increasing the accuracy of the model. As shown in Figure 1, there is a connection between the blocks of the input volume and transfer learning. The transfer learning approach extracted features from the ImageNet [168] weights. The weight is the parameters (including trainable and nontrainable) learned from the ImageNet [168] dataset. Since transfer learning involves transferring knowledge from one domain to another, we have utilized the ImageNet weight as the models developed in the ImageNet [168] classification competition are measured against each other for performance. Henceforth, the ImageNet weight provides a measure of how good a model is for classification. Besides, the ImageNet weight has already showed a markedly high accuracy [185]. The extracted features are then used by the network before being passed to the fusion and recognition block, where the features are amalgamated with the extracted features from the learning-from-scratch block for recognition.


## 4.1.4. Fusion and Recognition

The extracted features based on the ImageNet weights are then amalgamated with the features extracted by the block of training from scratch. A global average pooling is performed. Dropout technique helps to prevent a model from overfitting. It is used with dense fully connected layers. The fully connected layer compiles the data extracted by previous layers to form the final output. The last step passes the features through the fully connected layer, which then uses SoftMax to classify the class of the input images.


## 4.2. Implementation Details

### 4.2.1. Data Preparation

We have adopted data augmentation, stain normalization, and image normalization strategies to optimize the training process. Hereby, we have explained each of them briefly.


### 4.2.2. Data Augmentation

Due to the limited size of the input samples, the training of our DenTnet was prone to overfitting, which caused low detection rate. One solution to alleviate this issue was the data augmentation, which generated more training data from the existing training set. Dissimilar data augmentation techniques (e.g., horizontal flipping, rotating, and zooming) were applied to datasets for creating more training samples.


### 4.2.3. Stain Normalization

The breast cancer tissue slices are stained by H&E to differentiate between nuclei stained with purple color and other tissue structures stained with pink and red color to help pathologists analyze the shape of nuclei, density, variability, and overall tissue structure [186]. The H&E staining variability between acquired images exists due to the different staining protocols, scanners, and raw materials. This is a common problem with histological image analysis. Therefore, stain normalization of H&E-stained histology slides was a key step for reducing the color variation and obtaining a better color consistency prior to feeding input images into the DenTnet architecture. Different techniques are available for stain normalization in histological images. We have considered Macenko technique [187] due to its promising performance in many studies to standardize the color intensity of the tissue. This technique was based on a singular value decomposition. A logarithmic function was used to adaptively transform color concentration of the original histopathological image into its optical density (OD) image as OD=−logI/I0, where OD hints the matrix of optical density values, I belongs to the image intensity in red-green-blue space, and I0 addresses the illuminating intensity incident on the histological sample.


### 4.2.4. Intensity Normalization

Intensity normalization was another important preprocessing step. Its primary aim was to get the same range of values for each input image before feeding to the DenTnet. It also speeded up the convergence of DenTnet. Input images were normalized to the standard normal distribution by min-max normalization (i.e., using one of the most popular ways to normalize data) to the intensity range of [0, 1], which can be computed as(7)xnormalized=x−xminxmax−xmin,where x, xmin, and xmax indicate pixel, minimum, and maximum intensity values of the input image, respectively.


### 4.2.5. Hardware and Software Requirements

DenTnet was implemented using the TensorFlow and Keras framework [188, 189] and coded in Python using Jupyter Notebook on a Kaggle Private Kernel. The experiment was performed on a machine with the following configuration: Intel® Xeon® CPU @ 2.30 GHz with 16 CPU Cores, 16 GB RAM, and NVIDIA Tesla P100 GPU. We implemented and trained everything on the cloud using Kaggle GPU hours.


### 4.2.6. Training and Testing Setup

The dataset was divided in a 80%: 20% ratio, where 80% was used for training and the remaining 20% was used for testing. The data used for testing were kept isolated from the training set and never seen by the model during training. To evaluate the images classification, we have computed the recognition rate at the image level over the two different classes: (i) correctly classified images and (ii) the total number of images in the test set.


### 4.2.7. Training Procedure

In the training of a neural network, a measure of error is required to compute the error between the targeted output and the computed output of training data known as the loss function. An optimization algorithm is needed to minimize this function. We have considered Adam optimizer [190] with numerical stability constant epsilon = None, decay = 0.0, and AMSGrad = True. Table 3 presents the hyperparameter values of the proposed deep learning model. Learning rate (also referred to as step size) signifies the proportion to which weights are updated. A smaller value (e.g., 0.000001) slows down the learning process during training, whereas a larger value (e.g., 0.400) results in faster learning. We have considered a learning rate of 0.001. The exponential decay rates of the first and second moments were estimated to be 0.60 and 0.90, respectively. To update the weights, the number of epochs was set to 50 with 3222 steps per epoch and a batch size of 32. For the BreaKHis [33] dataset, we had a training sample of 103104 images, with 12288 validation samples and 697 testing samples. The training process used 10-fold cross-validation, where one of the samples was used to validate the data and the remaining 9 samples were used to train the DenTnet model. The fully connected layer used 1024 filters with a dropout rate of 0.50. Finally, the last layer used two filters with a SoftMax layer to classify the image into two classes (e.g., benign and malignant). We have used categorical cross-entropy as the objective function to quantify the difference between two probability distributions. The whole training process took more than 4 hours for the breast cancer tissue images.Table 3
List of hyperparameter values for the proposed deep learning model.
ModelHyperparametersBeta_1Beta_2Learning rateEpochBatch sizeEpsilonDecayAMSGradDenTnet0.600.900.0015032None0.0True


## 4.2.1. Data Preparation

We have adopted data augmentation, stain normalization, and image normalization strategies to optimize the training process. Hereby, we have explained each of them briefly.


## 4.2.2. Data Augmentation

Due to the limited size of the input samples, the training of our DenTnet was prone to overfitting, which caused low detection rate. One solution to alleviate this issue was the data augmentation, which generated more training data from the existing training set. Dissimilar data augmentation techniques (e.g., horizontal flipping, rotating, and zooming) were applied to datasets for creating more training samples.


## 4.2.3. Stain Normalization

The breast cancer tissue slices are stained by H&E to differentiate between nuclei stained with purple color and other tissue structures stained with pink and red color to help pathologists analyze the shape of nuclei, density, variability, and overall tissue structure [186]. The H&E staining variability between acquired images exists due to the different staining protocols, scanners, and raw materials. This is a common problem with histological image analysis. Therefore, stain normalization of H&E-stained histology slides was a key step for reducing the color variation and obtaining a better color consistency prior to feeding input images into the DenTnet architecture. Different techniques are available for stain normalization in histological images. We have considered Macenko technique [187] due to its promising performance in many studies to standardize the color intensity of the tissue. This technique was based on a singular value decomposition. A logarithmic function was used to adaptively transform color concentration of the original histopathological image into its optical density (OD) image as OD=−logI/I0, where OD hints the matrix of optical density values, I belongs to the image intensity in red-green-blue space, and I0 addresses the illuminating intensity incident on the histological sample.


## 4.2.4. Intensity Normalization

Intensity normalization was another important preprocessing step. Its primary aim was to get the same range of values for each input image before feeding to the DenTnet. It also speeded up the convergence of DenTnet. Input images were normalized to the standard normal distribution by min-max normalization (i.e., using one of the most popular ways to normalize data) to the intensity range of [0, 1], which can be computed as(7)xnormalized=x−xminxmax−xmin,where x, xmin, and xmax indicate pixel, minimum, and maximum intensity values of the input image, respectively.


## 4.2.5. Hardware and Software Requirements

DenTnet was implemented using the TensorFlow and Keras framework [188, 189] and coded in Python using Jupyter Notebook on a Kaggle Private Kernel. The experiment was performed on a machine with the following configuration: Intel® Xeon® CPU @ 2.30 GHz with 16 CPU Cores, 16 GB RAM, and NVIDIA Tesla P100 GPU. We implemented and trained everything on the cloud using Kaggle GPU hours.


## 4.2.6. Training and Testing Setup

The dataset was divided in a 80%: 20% ratio, where 80% was used for training and the remaining 20% was used for testing. The data used for testing were kept isolated from the training set and never seen by the model during training. To evaluate the images classification, we have computed the recognition rate at the image level over the two different classes: (i) correctly classified images and (ii) the total number of images in the test set.


## 4.2.7. Training Procedure

In the training of a neural network, a measure of error is required to compute the error between the targeted output and the computed output of training data known as the loss function. An optimization algorithm is needed to minimize this function. We have considered Adam optimizer [190] with numerical stability constant epsilon = None, decay = 0.0, and AMSGrad = True. Table 3 presents the hyperparameter values of the proposed deep learning model. Learning rate (also referred to as step size) signifies the proportion to which weights are updated. A smaller value (e.g., 0.000001) slows down the learning process during training, whereas a larger value (e.g., 0.400) results in faster learning. We have considered a learning rate of 0.001. The exponential decay rates of the first and second moments were estimated to be 0.60 and 0.90, respectively. To update the weights, the number of epochs was set to 50 with 3222 steps per epoch and a batch size of 32. For the BreaKHis [33] dataset, we had a training sample of 103104 images, with 12288 validation samples and 697 testing samples. The training process used 10-fold cross-validation, where one of the samples was used to validate the data and the remaining 9 samples were used to train the DenTnet model. The fully connected layer used 1024 filters with a dropout rate of 0.50. Finally, the last layer used two filters with a SoftMax layer to classify the image into two classes (e.g., benign and malignant). We have used categorical cross-entropy as the objective function to quantify the difference between two probability distributions. The whole training process took more than 4 hours for the breast cancer tissue images.Table 3
List of hyperparameter values for the proposed deep learning model.
ModelHyperparametersBeta_1Beta_2Learning rateEpochBatch sizeEpsilonDecayAMSGradDenTnet0.600.900.0015032None0.0True


## 5. Experimental Results and Comparison on BreaKHis Dataset

This section demonstrates the experimental results achieved from classifying the breast cancer histopathology (i.e., BreaKHis [33]) images using our proposed DenTnet model.Figure6 shows the performance curves obtained during the training of DenTnet using BreaKHis [33] dataset. A normalized confusion matrix for the classification of breast cancer test set images is illustrated in Figure 7(a). The main reason for confusion between benign and malignant breast tissues is their similar textures or expression. Henceforth, careful description of texture is required to remove the confusion between the two classes. For binary classification, 5 images only were misclassified, indicating that DenTnet achieved the highest and best ACC of 99.28%. Figures 7(b) and 7(c) demonstrate the ROC curve and precision-recall curve for classification of benign and malignant images from BreaKHis [33] dataset, respectively. AUC of 0.99, sensitivity of 97.73%, and specificity 100% have been reported. Table 4 lists the complete classification report of DenTnet. It achieved an ACC of 99.28%.Figure 6
(a) hints ACC and (b) shows loss charts of DenTnet during training.
(a)(b)Figure 7
(a) hints confusion matrix for benign and malignant classification, (b) shows ROC curve, and (c) demonstrates precision-recall curve.
(a)(b)(c)Table 4
Classification results by counting all evaluation criteria.
TypePRSRESF1SSupportBenign0.981.000.99216Malignant1.000.990.99481Micro mean0.990.990.99697Macro mean0.990.990.99697Weighted mean0.990.990.99697Table1 compares the results obtained by several methods. The methods of Togacar et al. [26], Parvin et al. [31], Man et al. [36], Soumik et al. [60], Liu et al. [172], Zerouaoui and Idri [56], and Chattopadhyay et al. [174] were centered on mainly CNN models, but they were tested against the same training-testing ratio of 80%: 20% on the BreaKHis dataset [33]. However, Boumaraf et al. [63] suggested a transfer-learning-based method deeming the residual CNN ResNet-18 as a backbone model with block-wise fine-tuning strategy and obtained a mean ACC of 92.15% applying a training-testing ratio of 80%: 20% on BreaKHis dataset [33]. From Table 1, it is notable that DenTnet [ours] achieved the best ACC on the same ground.


## 6. Generalization Ability Evaluation of Proposed DenTnet

What would be the performance of the proposed DenTnet compared with other types of cancer or disease datasets? To evaluate the generalization ability of DenTnet, this section presents the experimental result obtained not only from the dataset of BreaKHis [33] but also from additional datasets of Malaria [191], CovidXray [192], and SkinCancer [193].


### 6.1. Datasets Irrelevant to Breast Cancer

The three following datasets are not related to breast cancer. Herewith, their primary aim is to evaluate the generalization ability of our proposed method DenTnet:(i)
Malaria [191] ⇒ This dataset contains a total of 27558 infected and uninfected images for malaria.(ii)
SkinCancer [193] ⇒ This dataset contains balanced images from benign skin moles and malignant skin moles. The data consist of two folders, each containing 1800 pictures 224×244 from the two types of mole.(iii)
CovidXray [192] ⇒ Corona (COVID-19) virus affects the respiratory system of healthy individual. The chest X-ray is one of the key imaging methods to identify the coronavirus. This dataset contains chest X-ray of healthy versus pneumonia (Corona) infected patients along with few other categories including SARS (Severe Acute Respiratory Syndrome), Streptococcus, and ARDS (Acute Respiratory Distress Syndrome) with a goal of predicting and understanding the infection.Figure8 specifies some sample images from Malaria [191], SkinCancer [193], and CovidXray [192] datasets.Figure 8
(a), (b), and (c) specify images of Malaria [191], SkinCancer [193], and CovidXray [192] datasets, respectively.
(a)(b)(c)


### 6.2. Experimental Results Comparison

Using four datasets in the experiment, DenTnet has been compared with six widely used and well-known deep learning models, namely, AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184]. To evaluate and analyze the performance of DenTnet, four different cases are considered. The first case is the evaluation of different deep learning methods, which are trained and tested on BreaKHis [33] dataset. The second case studies the performance of the deep-learning-based classification methods that are trained and tested on Malaria [191] dataset. The third case is to train and test the deep learning models on SkinCancer [193] dataset. The final one is to understand and analyze the performance of the deep learning models on CovidXray [192] dataset. The overall results are tabulated in Tables 5–9. Besides, the RTM in seconds of various datasets using the deep learning models is shown in Table 10.Table 5
ACC of various methods deeming four different datasets.
ModelsACC of various datasetsGMN of ACCBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.92680.97380.87140.85260.90490.0951ResNet [75]0.98570.98320.90450.89900.94220.0578VGG16 [98]0.97850.98060.85010.85760.91450.0855VGG19 [98]0.97850.98110.85120.92790.93280.0672Inception V3 [88]0.97840.98790.85870.89980.92960.0704SqueezeNet [184]0.97560.94980.82880.80160.88580.1142DenTnet [ours]0.99280.98650.91570.89420.94630.0537Table 6
PRS of various methods deeming four different datasets.
ModelsPRS of various datasetsGMN of PRSBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.93170.96560.84170.87440.90210.0979ResNet [75]0.99370.97930.91670.86670.93770.0623VGG16 [98]0.99360.98880.90550.85330.93340.0666VGG19 [98]0.98140.97530.80830.98720.93480.0652Inception V3 [88]0.98290.97130.85120.97960.94460.0554SqueezeNet [184]0.98540.97780.88710.77990.90360.0964DenTnet [ours]0.97000.98480.92580.86410.93500.0650Table 7
RES of various methods deeming four different datasets.
ModelsRES of various datasetsGMN of RESBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.96470.98120.91540.88800.93660.0634ResNet [75]0.98540.98670.90100.96850.95970.0403VGG16 [98]0.97510.97180.82500.98460.93670.0633VGG19 [98]0.98750.98650.90650.90590.94570.0543Inception V3 [88]0.98540.98190.88740.94910.95010.0499SqueezeNet [184]0.97920.91970.78610.95140.90590.0941DenTnet [ours]0.98960.98790.92080.96290.96490.0351Table 8
F1S of various methods deeming four different datasets.
ModelsF1S of various datasetsGMN of F1SBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.94790.97340.87700.88110.91890.0811ResNet [75]0.98960.98300.91290.91470.94940.0506VGG16 [98]0.98430.98030.86340.91430.93420.0658VGG19 [98]0.98450.98090.85460.94480.93970.0603Inception V3 [88]0.98440.97240.86930.90770.93220.0678SqueezeNet [184]0.98230.94790.83360.85710.90310.0969DenTnet [ours]0.99480.98640.92330.91080.95310.0469Table 9
AUC of various methods deeming four different datasets.
ModelsAUC of various datasetsGMN of AUCBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.900.970.870.850.89640.1036ResNet [75]0.990.980.900.910.94410.0559VGG16 [98]0.980.980.860.850.91540.0846VGG19 [98]0.970.980.850.910.92600.0740Inception V3 [88]0.970.970.890.870.92390.0761SqueezeNet [184]0.970.950.830.750.87030.1297DenTnet [ours]0.990.990.910.900.94650.0535Table 10
RTM of various methods deeming four different datasets.
ModelsRTM in seconds of various datasetsGMN of RTMBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]AlexNet [81]075734100141313282762.8ResNet [75]168893556079926833368.5VGG16 [98]134197698145010813567.2VGG19 [98]235027115125512944059.4Inception V3 [88]144047357132911893597.3SqueezeNet [184]200804140133918643795.3DenTnet [ours]110837102087315193196.3According to the results in terms of GMN of ACC, RES, F1S, and AUC as shown in Tables5–9, respectively, the proposed DenTnet architecture provides the best scores as compared to AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184]. On the other hand, DenTnet gets the third best result. Moreover, in most of the cases, AlexNet [81] obtains the lowest results.


### 6.3. Performance Evaluation

The deepening of deep models makes their parameters rise rapidly, which may lead to overfitting of the model. To take the edge off the overfitting problem, predominantly a large number of dataset images are required as the training set. Considering a small dataset, it is possible to reduce the risk of overfitting of the model by reducing the parameters and augmenting the dataset. Accordingly, DenTnet used fewer parameters along with the dense connections in the construction of the model, instead of the direct connections among the hidden layers of the network. As DenTnet used fewer parameters, it attenuated the vanishing gradient descent and strengthened the feature propagation. Consequently, the proposed DenTnet outperformed its alternative state-of-the-art methods. Yet, its runtime was a bit longer in Malaria [191] and SkinCancer [193] datasets as compared to ResNet [75]. The main reason why the DenTnet model may require more time is that it uses many small convolutions in the network, which can run slower on GPU than compact large convolutions with the same number of GFLOPS. Still, DenTnet includes fewer parameters compatibility when compared to ResNet [75]. Henceforth, it is more efficient in solving the problem of overfitting. In general, all of the used algorithms suffered from some degree of overfitting problem on all datasets. We minimized such problems by reducing the batch size and adjusting the learning rate and the dropout rate. In some cases, the proposed DenTnet predicted fewer positive samples as compared to ResNet [75]. This is due to the lack of its conservative designation of the positive class. Thus, the GMN PRS of the proposed DenTnet was about 2% lower than that of ResNet [75].As VGG16 [98] is easier to implement, many deep learning image classification problems benefit from the technique by using the network either as a sole model or as a backbone architecture to classify images. While VGG19 [98] is better than the VGG16 [98] model, they are both very slow to train—for example, a ResNet with 34 layers only requires 18% of operations as a VGG with 19 layers (around half the layers of the ResNet) will require [194]. Regarding AlexNet [81], the model struggled to scan all features as it is not very deep, resulting in poor performance. The SqueezeNet [184] model achieved approximately the same performance as the AlexNet [81] model. VGG19 [98] and Inception V3 [88] showed almost the same level of effectiveness. Although the ResNet [75] model has proven to be a powerful tool for image classification and is usually fast, it has been shown to take a long time to train. Concisely, using all benefits of DenseNet [67] with optimization, DenTnet obtained the highest GMN ACC of 0.9463, RES of 0.9649, F1S of 0.9531, and AUC of 0.9465 from all four datasets. This implies that DenTnet has the best generalization ability compared to its alternative methods.Often, it is important to measure that certain deep learning models are more efficient and practical as compared to their alternatives. Seemingly, it is difficult to measure such superiority from the obtained experimental results in Tables5–10. Nonetheless, nonparametric statistical test can make a clear picture of this issue.


## 6.1. Datasets Irrelevant to Breast Cancer

The three following datasets are not related to breast cancer. Herewith, their primary aim is to evaluate the generalization ability of our proposed method DenTnet:(i)
Malaria [191] ⇒ This dataset contains a total of 27558 infected and uninfected images for malaria.(ii)
SkinCancer [193] ⇒ This dataset contains balanced images from benign skin moles and malignant skin moles. The data consist of two folders, each containing 1800 pictures 224×244 from the two types of mole.(iii)
CovidXray [192] ⇒ Corona (COVID-19) virus affects the respiratory system of healthy individual. The chest X-ray is one of the key imaging methods to identify the coronavirus. This dataset contains chest X-ray of healthy versus pneumonia (Corona) infected patients along with few other categories including SARS (Severe Acute Respiratory Syndrome), Streptococcus, and ARDS (Acute Respiratory Distress Syndrome) with a goal of predicting and understanding the infection.Figure8 specifies some sample images from Malaria [191], SkinCancer [193], and CovidXray [192] datasets.Figure 8
(a), (b), and (c) specify images of Malaria [191], SkinCancer [193], and CovidXray [192] datasets, respectively.
(a)(b)(c)


## 6.2. Experimental Results Comparison

Using four datasets in the experiment, DenTnet has been compared with six widely used and well-known deep learning models, namely, AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184]. To evaluate and analyze the performance of DenTnet, four different cases are considered. The first case is the evaluation of different deep learning methods, which are trained and tested on BreaKHis [33] dataset. The second case studies the performance of the deep-learning-based classification methods that are trained and tested on Malaria [191] dataset. The third case is to train and test the deep learning models on SkinCancer [193] dataset. The final one is to understand and analyze the performance of the deep learning models on CovidXray [192] dataset. The overall results are tabulated in Tables 5–9. Besides, the RTM in seconds of various datasets using the deep learning models is shown in Table 10.Table 5
ACC of various methods deeming four different datasets.
ModelsACC of various datasetsGMN of ACCBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.92680.97380.87140.85260.90490.0951ResNet [75]0.98570.98320.90450.89900.94220.0578VGG16 [98]0.97850.98060.85010.85760.91450.0855VGG19 [98]0.97850.98110.85120.92790.93280.0672Inception V3 [88]0.97840.98790.85870.89980.92960.0704SqueezeNet [184]0.97560.94980.82880.80160.88580.1142DenTnet [ours]0.99280.98650.91570.89420.94630.0537Table 6
PRS of various methods deeming four different datasets.
ModelsPRS of various datasetsGMN of PRSBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.93170.96560.84170.87440.90210.0979ResNet [75]0.99370.97930.91670.86670.93770.0623VGG16 [98]0.99360.98880.90550.85330.93340.0666VGG19 [98]0.98140.97530.80830.98720.93480.0652Inception V3 [88]0.98290.97130.85120.97960.94460.0554SqueezeNet [184]0.98540.97780.88710.77990.90360.0964DenTnet [ours]0.97000.98480.92580.86410.93500.0650Table 7
RES of various methods deeming four different datasets.
ModelsRES of various datasetsGMN of RESBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.96470.98120.91540.88800.93660.0634ResNet [75]0.98540.98670.90100.96850.95970.0403VGG16 [98]0.97510.97180.82500.98460.93670.0633VGG19 [98]0.98750.98650.90650.90590.94570.0543Inception V3 [88]0.98540.98190.88740.94910.95010.0499SqueezeNet [184]0.97920.91970.78610.95140.90590.0941DenTnet [ours]0.98960.98790.92080.96290.96490.0351Table 8
F1S of various methods deeming four different datasets.
ModelsF1S of various datasetsGMN of F1SBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.94790.97340.87700.88110.91890.0811ResNet [75]0.98960.98300.91290.91470.94940.0506VGG16 [98]0.98430.98030.86340.91430.93420.0658VGG19 [98]0.98450.98090.85460.94480.93970.0603Inception V3 [88]0.98440.97240.86930.90770.93220.0678SqueezeNet [184]0.98230.94790.83360.85710.90310.0969DenTnet [ours]0.99480.98640.92330.91080.95310.0469Table 9
AUC of various methods deeming four different datasets.
ModelsAUC of various datasetsGMN of AUCBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]SuccessFailureAlexNet [81]0.900.970.870.850.89640.1036ResNet [75]0.990.980.900.910.94410.0559VGG16 [98]0.980.980.860.850.91540.0846VGG19 [98]0.970.980.850.910.92600.0740Inception V3 [88]0.970.970.890.870.92390.0761SqueezeNet [184]0.970.950.830.750.87030.1297DenTnet [ours]0.990.990.910.900.94650.0535Table 10
RTM of various methods deeming four different datasets.
ModelsRTM in seconds of various datasetsGMN of RTMBreaKHis [33]Malaria [191]SkinCancer [193]CovidXray [192]AlexNet [81]075734100141313282762.8ResNet [75]168893556079926833368.5VGG16 [98]134197698145010813567.2VGG19 [98]235027115125512944059.4Inception V3 [88]144047357132911893597.3SqueezeNet [184]200804140133918643795.3DenTnet [ours]110837102087315193196.3According to the results in terms of GMN of ACC, RES, F1S, and AUC as shown in Tables5–9, respectively, the proposed DenTnet architecture provides the best scores as compared to AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184]. On the other hand, DenTnet gets the third best result. Moreover, in most of the cases, AlexNet [81] obtains the lowest results.


## 6.3. Performance Evaluation

The deepening of deep models makes their parameters rise rapidly, which may lead to overfitting of the model. To take the edge off the overfitting problem, predominantly a large number of dataset images are required as the training set. Considering a small dataset, it is possible to reduce the risk of overfitting of the model by reducing the parameters and augmenting the dataset. Accordingly, DenTnet used fewer parameters along with the dense connections in the construction of the model, instead of the direct connections among the hidden layers of the network. As DenTnet used fewer parameters, it attenuated the vanishing gradient descent and strengthened the feature propagation. Consequently, the proposed DenTnet outperformed its alternative state-of-the-art methods. Yet, its runtime was a bit longer in Malaria [191] and SkinCancer [193] datasets as compared to ResNet [75]. The main reason why the DenTnet model may require more time is that it uses many small convolutions in the network, which can run slower on GPU than compact large convolutions with the same number of GFLOPS. Still, DenTnet includes fewer parameters compatibility when compared to ResNet [75]. Henceforth, it is more efficient in solving the problem of overfitting. In general, all of the used algorithms suffered from some degree of overfitting problem on all datasets. We minimized such problems by reducing the batch size and adjusting the learning rate and the dropout rate. In some cases, the proposed DenTnet predicted fewer positive samples as compared to ResNet [75]. This is due to the lack of its conservative designation of the positive class. Thus, the GMN PRS of the proposed DenTnet was about 2% lower than that of ResNet [75].As VGG16 [98] is easier to implement, many deep learning image classification problems benefit from the technique by using the network either as a sole model or as a backbone architecture to classify images. While VGG19 [98] is better than the VGG16 [98] model, they are both very slow to train—for example, a ResNet with 34 layers only requires 18% of operations as a VGG with 19 layers (around half the layers of the ResNet) will require [194]. Regarding AlexNet [81], the model struggled to scan all features as it is not very deep, resulting in poor performance. The SqueezeNet [184] model achieved approximately the same performance as the AlexNet [81] model. VGG19 [98] and Inception V3 [88] showed almost the same level of effectiveness. Although the ResNet [75] model has proven to be a powerful tool for image classification and is usually fast, it has been shown to take a long time to train. Concisely, using all benefits of DenseNet [67] with optimization, DenTnet obtained the highest GMN ACC of 0.9463, RES of 0.9649, F1S of 0.9531, and AUC of 0.9465 from all four datasets. This implies that DenTnet has the best generalization ability compared to its alternative methods.Often, it is important to measure that certain deep learning models are more efficient and practical as compared to their alternatives. Seemingly, it is difficult to measure such superiority from the obtained experimental results in Tables5–10. Nonetheless, nonparametric statistical test can make a clear picture of this issue.


## 7. Nonparametric Statistical Analysis

Figure9 depicts performance evaluation of various algorithms deeming the numerical values of the ineffectualness metrics and RTM from Table 11. It is noted that, for a better visualization purpose, the RTM scores in Figure 9 use log-normal distribution [195] with a mean of 10 and standard deviation of 1. However, from this graph, it is extremely hard to rank each algorithm. However, statistically, it is possible to show that one algorithm is better than its alternatives. Friedman test [196] and its derivatives (e.g., Iman-Davenport test [197]) are normally referred to as examples of the most well-known nonparametric tests for multiple comparisons. The mathematical equations of Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests can be found in the works of Quade [199] and Westfall and Young [200]. Friedman test [196] takes measures in preparation for ranking of a set of algorithms with performance in descending order. But it can solely inform us about the appearance of differences among all samples of results under comparison. Henceforth, its alternatives (e.g., Friedman’s aligned rank test [198] and Quade test [199]) can give us further information. Consequently, we have performed the tests of Friedman [196], Friedman’s aligned rank [198], and Quade [199] for average rankings based on the features of our experimental study. On rejecting null-hypotheses, we have continued to use post hoc procedures to find the special pairs of algorithms that give idiosyncrasies. In the case of 1×N comparisons, the post hoc procedures make up for Bonferroni-Dunn’s [201], Holm’s [202], Hochberg’s [203], Hommel’s [204, 205], Holland and Copenhaver’s [206], Rom’s [207], Finner’s [208], and David Li’s [209] procedures, whereas the post hoc procedures of Nemenyi [210], Shaffer [211], and Bergmann-Hommel [212] are involved in N×N comparisons. The details can be found in the works of Bergmann and Hommel [212], García and Herrera [213], and Hommel and Bernhard [205].Figure 9
Plotting of the numerical values using data from Table11.Table 11
Summary of performance failure and RTM scores of miscellaneous deep learning algorithms.
ModelsGMN scores of performance failureGMN of RTMACCPRSRESF1SAUCAlexNet [81]0.09510.09790.06340.08110.10362762.8ResNet [75]0.05780.06230.04030.05060.05593368.5VGG16 [98]0.08550.06660.06330.06580.08463567.2VGG19 [98]0.06720.06520.05430.06030.07404059.4Inception V3 [88]0.07040.05540.04990.06780.07613597.3SqueezeNet [184]0.11420.09640.09410.09690.12973795.3DenTnet [ours]0.05370.06500.03510.04690.05353196.3


### 7.1. Average Ranking of Algorithms

To get the nonparametric statistical test results, Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests have been applied to the results of seven models in Table 11. Explicitly, statistical tests have been applied to a matrix with dimension of 7×6, where 7 is the number of models and 6 is the number of parameters (as 6 datasets while applied to the statistical software environment [214]) in each model. Table 12 shows the average ranking computed by using Friedman [196], Friedman’s aligned rank [198], and Quade [199] nonparametric statistical tests. The nonparametric Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests determine whether there were significant differences among various models taking data from Table 11. These tests provide the average ranking of all algorithms; that is, the best performing algorithm gets the highest rank of 1, the second-best algorithm gets the rank of 2, and so on.Table 12
Average ranking of each algorithm using nonparametric statistical tests. The best results are shown in bold.
AlgorithmsMultiple comparison testsFriedman ranking [196]Friedman’s aligned ranking [198]Quade ranking [199]AlexNet [81]5.333326.00004.6189ResNet [75]2.166709.00002.2857VGG16 [98]4.666727.83334.6191VGG19 [98]4.000021.66674.3333Inception V3 [88]3.666722.16674.0952SqueezeNet [184]6.666736.66676.6667DenTnet [ours]1.500007.16671.3809Various statistics24.50000023.1025575.274194p value0.0004220.0007630.000820Figure10 makes a visualization of the average rankings using the data in Table 12. From Figure 10, it is noticeable that the algorithm of DenTnet [ours] became the best performing one, with the longest bars of 0.6667, 0.1395, and 0.7242 for Friedman test [196], Friedman’s aligned rank test [198], and Quade test [199], respectively. This indicates that the algorithm of DenTnet [ours] gives great performance for the solution of underlaying problems of classifying breast cancer histopathological images from four different datasets. Friedman [196] statistic considered reduction performance (distributed according to chi-square with 6 degrees of freedom) of 24.500000. Friedman’s aligned [198] statistic considered reduction performance (distributed according to chi-square with 6 degrees of freedom) of 23.102557. Iman-Davenport [197] statistic considered reduction performance (distributed according to F-distribution with 6 and 30 degrees of freedom) of 10.652174. Quade [199] statistic considered reduction performance (distributed according to F-distribution with 6 and 30 degrees of freedom) of 5.274194. The p values computed through Friedman statistic, Friedman’s aligned statistic, Iman-Davenport statistic, and Quade statistic are 0.000422, 0.000762847204, 0.000002458229, and 0.000820133186, respectively.Figure 10
Plotting of average rankings data from Table12, where each value x is plotted as 1/x to visualize the highest ranking with the tallest bar.Table13 demonstrates the results obtained on post hoc comparisons of adjusted p values; α=0.05 and α=0.10. Using level of significance α=0.05, (i) Bonferroni-Dunn’s [201] procedure rejects those hypotheses that have an unadjusted p value ≤0.008333; (ii) Holm’s [202] procedure rejects those hypotheses that have an unadjusted p value ≤0.016667; (iii) Hochberg’s [203] procedure rejects those hypotheses that have an unadjusted p value ≤0.0125; (iv) Hommel’s [204] procedure rejects those hypotheses that have an unadjusted p value ≤0.016667; (v) Holland’s [206] procedure rejects those hypotheses that have an unadjusted p value ≤0.016952; (vi) Rom’s [207] procedure rejects those hypotheses that have an unadjusted p value ≤0.013109; (vii) Finner’s [208] procedure rejects those hypotheses that have an unadjusted p value ≤0.033617; and (viii) Li’s [209] procedure rejects those hypotheses that have an unadjusted p value ≤0.021422.Table 13
Results achieved on post hoc comparisons for adjustedp values, with α=0.05 and α=0.10.
IndexAlgorithmsp valuesα=0.05α=0.10Holm [202]Shaffer [211]Holm [202]Shaffer [211]1VGG19 [98] versus Inception V3 [88]0.7892680.0500000.0500000.1000000.1000002ResNet [75] versus DenTnet [ours]0.5929800.0250000.0250000.0500000.0500003VGG16 [98] versus VGG19 [98]0.5929800.0166670.0166670.0333330.0333334AlexNet [81] versus VGG16 [98]0.5929800.0125000.0166670.0250000.0333335VGG16 [98] versus Inception V3 [88]0.4226780.0100000.0166670.0200000.0333336AlexNet [81] versus SqueezeNet [184]0.2850490.0083330.0083330.0166670.0166677AlexNet [81] versus VGG19 [98]0.2850490.0071430.0071430.0142860.0142868ResNet [75] versus Inception V3 [88]0.2291020.0062500.0062500.0125000.0125009AlexNet [81] versus Inception V3 [88]0.1814490.0055560.0055560.0111110.01111110ResNet [75] versus VGG19 [98]0.1415790.0050000.0050000.0100000.01000011VGG16 [98] versus SqueezeNet [184]0.1088090.0045450.0045450.0090910.00909112Inception V3 [88] versus DenTnet [ours]0.0823520.0041670.0041670.0083330.00833313VGG19 [98] versus DenTnet [ours]0.0450210.0038460.0038460.0076920.00769214ResNet [75] versus VGG16 [98]0.0450210.0035710.0035710.0071430.00714315VGG19 [98] versus SqueezeNet [184]0.0325090.0033330.0033330.0066670.00666716Inception V3 [88] versus SqueezeNet [184]0.0161570.0031250.0033330.0062500.00666717VGG16 [98] versus DenTnet [ours]0.0111180.0029410.0033330.0058820.00666718AlexNet [81] versus ResNet [75]0.0111180.0027780.0033330.0055560.00666719AlexNet [81] versus DenTnet [ours]0.0021160.0026320.0033330.0052630.00666720ResNet [75] versus SqueezeNet [184]0.0003090.0025000.0033330.0050000.00666721SqueezeNet [184] versus DenTnet [ours]0.0000340.0023810.0023810.0047620.004762


### 7.2. Post Hoc Procedures:1×N Comparisons

In the case of1×N comparisons, the post hoc procedures consist of Bonferroni-Dunn’s [201], Holm’s [202], Hochberg’s [203], Hommel’s [204, 205], Holland and Copenhaver’s [206], Rom’s [207], Finner’s [208], and David Li’s [209] procedures. In these tests, multiple comparison post hoc procedures have been considered for comparing the control algorithm of DenTnet [ours] with others. The results have been shown by computing p values for each comparison. Table 14 depicts the obtained p values using the ranks computed by nonparametric Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests. All tests have demonstrated significant improvements of DenTnet [ours] over AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184] counting each and every post hoc procedure. Besides, David Li’s [209] procedure had the greatest performance, reaching the lowest p value in the comparisons.Table 14
Adjustedp values for various tests considering DenTnet [ours] as control method.
TestsAlgorithmsNot1×N post hoc procedures and p valuesadjusted1-2 step-procedureStep-down proceduresStep-up proceduresp valuespBonf [201]pLi [209]pHolm [202]pHol [206]pFinn [208]pHoch [203]pHom [204]pRom [207]Friedman [196]SqueezeNet [184]0.0000340.0002060.0000840.0002060.0002060.0002060.0002060.0002060.000196AlexNet [81]0.0021160.0126940.0051710.0105780.0105330.0063330.0105780.0105780.010060VGG16 [98]0.0111180.0667050.0265880.0444700.0437340.0221120.0444700.0444700.042403VGG19 [98]0.0450210.2701250.0995950.1350630.1290730.0667650.1350630.1235280.135063Inception V3 [88]0.0823520.4941130.1682810.1647040.1579230.0979900.1647040.1647040.164704ResNet [75]0.5929803.5578810.5929800.5929800.5929800.5929800.5929800.592980.592980F. al. rank [198]SqueezeNet [184]0.0000310.0001870.0001520.0001870.0001870.0001870.0001870.0001870.000178VGG16 [98]0.0035250.0211470.0169640.0176230.0174990.0105360.0176230.0176230.016759AlexNet [81]0.0078370.0470230.0369540.0313480.0309820.0156130.0313480.0313480.029891Inception V3 [88]0.0341930.2051550.1434040.1025780.0991100.0508480.0812770.0683850.081277VGG19 [98]0.0406380.2438300.1659520.1025780.0991100.0508480.0812770.0812770.081277ResNet [75]0.7957584.7745450.7957580.7957580.7957580.7957580.7957580.7957580.795758Quade [199]SqueezeNet [184]0.0278790.1672720.0867790.1672720.1560380.1560380.1672720.1672720.159049AlexNet [81]0.1779391.0676320.3775310.8896930.6245770.4444630.5176180.3882130.517618VGG16 [98]0.1779391.0676320.3775310.8896930.6245770.4444630.5176180.3882130.517618VGG19 [98]0.2193481.3160860.4278030.8896930.6245770.4444630.5176180.4386950.517618Inception V3 [88]0.2588091.5528530.4686930.8896930.6245770.4444630.5176180.5176180.517618ResNet [75]0.7066174.2397010.7066170.8896930.7066170.7066170.7066170.7066170.706617


### 7.3. Post Hoc Procedures:N×N Comparisons

In the case ofN×N comparisons, the post hoc procedures consist of Nemenyi’s [210], Shaffer’s [211], and Bergmann-Hommel’s [212] procedures. Table 15 presents 21 hypotheses of equality among 7 different algorithms and p values achieved. Using level of significance α=0.05, (i) Nemenyi’s [210] procedure rejects those hypotheses that have an unadjusted p value ≤0.002381; (ii) Holm’s [202] procedure rejects those hypotheses that have an unadjusted p value ≤0.002778; (iii) Shaffer’s [211] procedure rejects those hypotheses that have an unadjusted p value ≤0.002381; and (iv) Bergmann’s [212] procedure rejects those hypotheses of AlexNet [81] versus DenTnet [ours], ResNet [75] versus SqueezeNet [184], and SqueezeNet [184] versus DenTnet [ours]. On the other hand, considering α=0.10, (i) Nemenyi’s [210] procedure rejects those hypotheses that have an unadjusted p value ≤0.004762; (ii) Holm’s [202] procedure rejects those hypotheses that have an unadjusted p value ≤0.005556; (iii) Shaffer’s [211] procedure rejects those hypotheses that have an unadjusted p value ≤0.004762; and (iv) Bergmann’s [212] procedure rejects those hypotheses of AlexNet [81] versus DenTnet [ours], ResNet [75] versus SqueezeNet [184], and SqueezeNet [184] versus DenTnet [ours].Table 15
Adjustedp values of tests for multiple comparisons among all methods.
IndexHypothesisN×N post hoc procedures and p valuesUnadjustedNemenyi [210]Holm [202]Shaffer [211]Bergmann [212]1SqueezeNet [184] versus DenTnet [ours]0.0000340.0007210.0007210.0007210.0007212ResNet [75] versus SqueezeNet [184]0.0003090.0064790.0061710.0046280.0046283AlexNet [81] versus DenTnet [ours]0.0021160.0444280.0401970.0317340.0317344AlexNet [81] versus ResNet [75]0.0111180.2334690.2001160.1667630.1111765VGG16 [98] versus DenTnet [ours]0.0111180.2334690.2001160.1667630.1222936Inception V3 [88] versus SqueezeNet [184]0.0161570.3392960.2585110.2423540.1777267VGG19 [98] versus SqueezeNet [184]0.0325090.6826980.4876420.4876420.2925858ResNet [75] versus VGG16 [98]0.0450210.9454390.6302920.4952300.3151469VGG19 [98] versus DenTnet [ours]0.0450210.9454390.6302920.4952300.40518810Inception V3 [88] versus DenTnet [ours]0.0823521.7293970.9882270.9058740.49411311VGG16 [98] versus SqueezeNet [184]0.1088092.2849981.1969041.1969040.65285712ResNet [75] versus VGG19 [98]0.1415792.9731561.4157891.4157890.65285713AlexNet [81] versus Inception V3 [88]0.1814493.8104331.6330431.6330431.27014414ResNet [75] versus Inception V3 [88]0.2291024.8111401.8328151.6330431.27014415AlexNet [81] versus VGG19 [98]0.2850495.9860381.9953461.9953461.27014416AlexNet [81] versus SqueezeNet [184]0.2850495.9860381.9953461.9953461.42524717VGG16 [98] versus Inception V3 [88]0.4226788.8762402.1133902.1133901.69071218AlexNet [81] versus VGG16 [98]0.59298012.4525822.3719202.3719201.77894019VGG16 [98] versus VGG19 [98]0.59298012.4525822.3719202.3719201.77894020ResNet [75] versus DenTnet [ours]0.59298012.4525822.3719202.3719201.77894021VGG19 [98] versus Inception V3 [88]0.78926816.5746292.3719202.3719201.778940


### 7.4. Critical Distance Diagram from Nemenyi [210] Test

Nemenyi [210] test is very conservative with a low power, and hence it is not a recommended choice in practice [215]. Nevertheless, it has a unique advantage of having an associated plot to demonstrate the results of fair comparison. Figure 11 depicts the Nemenyi [210] post hoc critical distance diagrams at three distinct levels of significance α values. If the distance between algorithms is less than the critical distance, then there is no statistically significant difference between them. The diagrams in Figures 11(a) and 11(b) associated with α=0.10 with the critical distance of 3.3588 and with α=0.05 with the critical distance of 3.6768, respectively, are identical, whereas the diagram in Figure 11(c) related to α=0.01 with the critical distance of 4.3054 is different. Any two algorithms are considered as significantly different if their performance variation is greater than the critical distance. To this end, from Figure 11, it is noticeable that, at α=0.01, both SqueezeNet [184] versus DenTnet [ours] and SqueezeNet [184] versus ResNet [75] are remarkably different, while other pairs are not remarkably divergent as their performance differences are less than 4.3054. As compared to ResNet [75], DenTnet [ours] differs from SqueezeNet [184] by a greater distance. On the other hand, SqueezeNet [184] versus DenTnet [ours] and AlexNet [81] versus DenTnet [ours] are significantly different at both α=0.10 and α=0.05, whereas SqueezeNet [184] versus ResNet [75] is significantly dissimilar at those α values. Straightforwardly, DenTnet [ours] is outstandingly unalike both SqueezeNet [184] and AlexNet [81], but ResNet [75] is not outstandingly unalike AlexNet [81]. This implies that the method of DenTnet [ours] outperforms that of ResNet [75], which also agrees with the finding in Figure 10.Figure 11
Nemenyi [210] post hoc critical distance diagrams for three α values using data in Table 11.
(a)(b)(c)


### 7.5. Reasons of Superiority

In this study, DenseNet [67] was a great choice as it was very compact and deep. It used less training parameters and reduced the risk of model overfitting and improved the learning rate. In the dense block of DenTnet, the outputs from the previous layers were concatenated instead of using the summation. This type of concatenation helped to markedly speed up the processing of data for large number of columns. The dense block of DenTnet contained convolution and nonlinear layers, which applied several optimization techniques (e.g., dropout and BN). DenTnet scaled to hundreds of layers, while exhibiting no optimization difficulties. Overall, this model was applied to a very large number of preprocessed augmented images from BreaKHis [33], Malaria [191], SkinCancer [193], and CovidXray [192] datasets. To the best of our knowledge, no other studies in the literature had such an edge. Additionally, the use of data augmentation approach in this study positively affected the performance of the model due to expansion in the size of training data, which is the foremost requirement of a deep network for its proper working. Our DenTnet was well trained through various parameters’ tuning. For example, in the case of BreaKHis [33], unlike other existing models, our model was trained on all the magnifications combined (40x, 100x, 200x, and 400x) to avoid any loss of generality.In sum and substance, based on the aforementioned experimental and nonparametric statistical test results, it is, therefore, possible to conclude that the proposed DenTnet [ours] outperformed AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184] in terms of computational speed. Significantly, the accuracy achieved by the proposed DenTnet [ours] surpassed those of existing state-of-the-art models in classifying images of the BreaKHis [33], Malaria [191], SkinCancer [193], and the CovidXray [192] dataset.


### 7.6. Limitation of Proposed Model and Methodology

Despite these promising results, questions remain as to whether the proposed DenTnet model could be utilized to classify categorical images. Moreover, DenTnet was tested with one breast cancer dataset (i.e., BreaKHis [33]) only. Although the generalization ability of DenTnet with three non-breast-cancer-related datasets was studied in Section 6, it is unknown whether DenTnet can generalize to other state-of-the-art breast cancer datasets. Future work should, therefore, investigate the efficacy and generalizability of DenTnet with datasets along with multiclass labels, as well as other publicly available breast cancer datasets (e.g., the most recently introduced MITNET dataset [216]).The classification effect of breast cancer histopathological images of any deep learning methodology is related to the features and many studies predominantly focused on how to develop good feature descriptors and better extract features. Different from traditional handcrafted feature-based models, DenTnet can automatically extract more abstract features. Nevertheless, it is worth noting that although the proposed DenTnet has addressed the cross-domain problem by utilizing the transfer learning approach, features extracted in the methodology are solely deep-network-based features, which are extracted by feeding images directly to the model. However, feeding deep models directly with images would not generalize as the models consider color distribution of an image. It is understood that local information can be captured from color images using Local Binary Pattern (LBP) [217]. Therefore, future work can use multiple types of features by combining the features extracted by the proposed method with LBP features to address this issue.


## 7.1. Average Ranking of Algorithms

To get the nonparametric statistical test results, Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests have been applied to the results of seven models in Table 11. Explicitly, statistical tests have been applied to a matrix with dimension of 7×6, where 7 is the number of models and 6 is the number of parameters (as 6 datasets while applied to the statistical software environment [214]) in each model. Table 12 shows the average ranking computed by using Friedman [196], Friedman’s aligned rank [198], and Quade [199] nonparametric statistical tests. The nonparametric Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests determine whether there were significant differences among various models taking data from Table 11. These tests provide the average ranking of all algorithms; that is, the best performing algorithm gets the highest rank of 1, the second-best algorithm gets the rank of 2, and so on.Table 12
Average ranking of each algorithm using nonparametric statistical tests. The best results are shown in bold.
AlgorithmsMultiple comparison testsFriedman ranking [196]Friedman’s aligned ranking [198]Quade ranking [199]AlexNet [81]5.333326.00004.6189ResNet [75]2.166709.00002.2857VGG16 [98]4.666727.83334.6191VGG19 [98]4.000021.66674.3333Inception V3 [88]3.666722.16674.0952SqueezeNet [184]6.666736.66676.6667DenTnet [ours]1.500007.16671.3809Various statistics24.50000023.1025575.274194p value0.0004220.0007630.000820Figure10 makes a visualization of the average rankings using the data in Table 12. From Figure 10, it is noticeable that the algorithm of DenTnet [ours] became the best performing one, with the longest bars of 0.6667, 0.1395, and 0.7242 for Friedman test [196], Friedman’s aligned rank test [198], and Quade test [199], respectively. This indicates that the algorithm of DenTnet [ours] gives great performance for the solution of underlaying problems of classifying breast cancer histopathological images from four different datasets. Friedman [196] statistic considered reduction performance (distributed according to chi-square with 6 degrees of freedom) of 24.500000. Friedman’s aligned [198] statistic considered reduction performance (distributed according to chi-square with 6 degrees of freedom) of 23.102557. Iman-Davenport [197] statistic considered reduction performance (distributed according to F-distribution with 6 and 30 degrees of freedom) of 10.652174. Quade [199] statistic considered reduction performance (distributed according to F-distribution with 6 and 30 degrees of freedom) of 5.274194. The p values computed through Friedman statistic, Friedman’s aligned statistic, Iman-Davenport statistic, and Quade statistic are 0.000422, 0.000762847204, 0.000002458229, and 0.000820133186, respectively.Figure 10
Plotting of average rankings data from Table12, where each value x is plotted as 1/x to visualize the highest ranking with the tallest bar.Table13 demonstrates the results obtained on post hoc comparisons of adjusted p values; α=0.05 and α=0.10. Using level of significance α=0.05, (i) Bonferroni-Dunn’s [201] procedure rejects those hypotheses that have an unadjusted p value ≤0.008333; (ii) Holm’s [202] procedure rejects those hypotheses that have an unadjusted p value ≤0.016667; (iii) Hochberg’s [203] procedure rejects those hypotheses that have an unadjusted p value ≤0.0125; (iv) Hommel’s [204] procedure rejects those hypotheses that have an unadjusted p value ≤0.016667; (v) Holland’s [206] procedure rejects those hypotheses that have an unadjusted p value ≤0.016952; (vi) Rom’s [207] procedure rejects those hypotheses that have an unadjusted p value ≤0.013109; (vii) Finner’s [208] procedure rejects those hypotheses that have an unadjusted p value ≤0.033617; and (viii) Li’s [209] procedure rejects those hypotheses that have an unadjusted p value ≤0.021422.Table 13
Results achieved on post hoc comparisons for adjustedp values, with α=0.05 and α=0.10.
IndexAlgorithmsp valuesα=0.05α=0.10Holm [202]Shaffer [211]Holm [202]Shaffer [211]1VGG19 [98] versus Inception V3 [88]0.7892680.0500000.0500000.1000000.1000002ResNet [75] versus DenTnet [ours]0.5929800.0250000.0250000.0500000.0500003VGG16 [98] versus VGG19 [98]0.5929800.0166670.0166670.0333330.0333334AlexNet [81] versus VGG16 [98]0.5929800.0125000.0166670.0250000.0333335VGG16 [98] versus Inception V3 [88]0.4226780.0100000.0166670.0200000.0333336AlexNet [81] versus SqueezeNet [184]0.2850490.0083330.0083330.0166670.0166677AlexNet [81] versus VGG19 [98]0.2850490.0071430.0071430.0142860.0142868ResNet [75] versus Inception V3 [88]0.2291020.0062500.0062500.0125000.0125009AlexNet [81] versus Inception V3 [88]0.1814490.0055560.0055560.0111110.01111110ResNet [75] versus VGG19 [98]0.1415790.0050000.0050000.0100000.01000011VGG16 [98] versus SqueezeNet [184]0.1088090.0045450.0045450.0090910.00909112Inception V3 [88] versus DenTnet [ours]0.0823520.0041670.0041670.0083330.00833313VGG19 [98] versus DenTnet [ours]0.0450210.0038460.0038460.0076920.00769214ResNet [75] versus VGG16 [98]0.0450210.0035710.0035710.0071430.00714315VGG19 [98] versus SqueezeNet [184]0.0325090.0033330.0033330.0066670.00666716Inception V3 [88] versus SqueezeNet [184]0.0161570.0031250.0033330.0062500.00666717VGG16 [98] versus DenTnet [ours]0.0111180.0029410.0033330.0058820.00666718AlexNet [81] versus ResNet [75]0.0111180.0027780.0033330.0055560.00666719AlexNet [81] versus DenTnet [ours]0.0021160.0026320.0033330.0052630.00666720ResNet [75] versus SqueezeNet [184]0.0003090.0025000.0033330.0050000.00666721SqueezeNet [184] versus DenTnet [ours]0.0000340.0023810.0023810.0047620.004762


## 7.2. Post Hoc Procedures:1×N Comparisons

In the case of1×N comparisons, the post hoc procedures consist of Bonferroni-Dunn’s [201], Holm’s [202], Hochberg’s [203], Hommel’s [204, 205], Holland and Copenhaver’s [206], Rom’s [207], Finner’s [208], and David Li’s [209] procedures. In these tests, multiple comparison post hoc procedures have been considered for comparing the control algorithm of DenTnet [ours] with others. The results have been shown by computing p values for each comparison. Table 14 depicts the obtained p values using the ranks computed by nonparametric Friedman [196], Friedman’s aligned rank [198], and Quade [199] tests. All tests have demonstrated significant improvements of DenTnet [ours] over AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184] counting each and every post hoc procedure. Besides, David Li’s [209] procedure had the greatest performance, reaching the lowest p value in the comparisons.Table 14
Adjustedp values for various tests considering DenTnet [ours] as control method.
TestsAlgorithmsNot1×N post hoc procedures and p valuesadjusted1-2 step-procedureStep-down proceduresStep-up proceduresp valuespBonf [201]pLi [209]pHolm [202]pHol [206]pFinn [208]pHoch [203]pHom [204]pRom [207]Friedman [196]SqueezeNet [184]0.0000340.0002060.0000840.0002060.0002060.0002060.0002060.0002060.000196AlexNet [81]0.0021160.0126940.0051710.0105780.0105330.0063330.0105780.0105780.010060VGG16 [98]0.0111180.0667050.0265880.0444700.0437340.0221120.0444700.0444700.042403VGG19 [98]0.0450210.2701250.0995950.1350630.1290730.0667650.1350630.1235280.135063Inception V3 [88]0.0823520.4941130.1682810.1647040.1579230.0979900.1647040.1647040.164704ResNet [75]0.5929803.5578810.5929800.5929800.5929800.5929800.5929800.592980.592980F. al. rank [198]SqueezeNet [184]0.0000310.0001870.0001520.0001870.0001870.0001870.0001870.0001870.000178VGG16 [98]0.0035250.0211470.0169640.0176230.0174990.0105360.0176230.0176230.016759AlexNet [81]0.0078370.0470230.0369540.0313480.0309820.0156130.0313480.0313480.029891Inception V3 [88]0.0341930.2051550.1434040.1025780.0991100.0508480.0812770.0683850.081277VGG19 [98]0.0406380.2438300.1659520.1025780.0991100.0508480.0812770.0812770.081277ResNet [75]0.7957584.7745450.7957580.7957580.7957580.7957580.7957580.7957580.795758Quade [199]SqueezeNet [184]0.0278790.1672720.0867790.1672720.1560380.1560380.1672720.1672720.159049AlexNet [81]0.1779391.0676320.3775310.8896930.6245770.4444630.5176180.3882130.517618VGG16 [98]0.1779391.0676320.3775310.8896930.6245770.4444630.5176180.3882130.517618VGG19 [98]0.2193481.3160860.4278030.8896930.6245770.4444630.5176180.4386950.517618Inception V3 [88]0.2588091.5528530.4686930.8896930.6245770.4444630.5176180.5176180.517618ResNet [75]0.7066174.2397010.7066170.8896930.7066170.7066170.7066170.7066170.706617


## 7.3. Post Hoc Procedures:N×N Comparisons

In the case ofN×N comparisons, the post hoc procedures consist of Nemenyi’s [210], Shaffer’s [211], and Bergmann-Hommel’s [212] procedures. Table 15 presents 21 hypotheses of equality among 7 different algorithms and p values achieved. Using level of significance α=0.05, (i) Nemenyi’s [210] procedure rejects those hypotheses that have an unadjusted p value ≤0.002381; (ii) Holm’s [202] procedure rejects those hypotheses that have an unadjusted p value ≤0.002778; (iii) Shaffer’s [211] procedure rejects those hypotheses that have an unadjusted p value ≤0.002381; and (iv) Bergmann’s [212] procedure rejects those hypotheses of AlexNet [81] versus DenTnet [ours], ResNet [75] versus SqueezeNet [184], and SqueezeNet [184] versus DenTnet [ours]. On the other hand, considering α=0.10, (i) Nemenyi’s [210] procedure rejects those hypotheses that have an unadjusted p value ≤0.004762; (ii) Holm’s [202] procedure rejects those hypotheses that have an unadjusted p value ≤0.005556; (iii) Shaffer’s [211] procedure rejects those hypotheses that have an unadjusted p value ≤0.004762; and (iv) Bergmann’s [212] procedure rejects those hypotheses of AlexNet [81] versus DenTnet [ours], ResNet [75] versus SqueezeNet [184], and SqueezeNet [184] versus DenTnet [ours].Table 15
Adjustedp values of tests for multiple comparisons among all methods.
IndexHypothesisN×N post hoc procedures and p valuesUnadjustedNemenyi [210]Holm [202]Shaffer [211]Bergmann [212]1SqueezeNet [184] versus DenTnet [ours]0.0000340.0007210.0007210.0007210.0007212ResNet [75] versus SqueezeNet [184]0.0003090.0064790.0061710.0046280.0046283AlexNet [81] versus DenTnet [ours]0.0021160.0444280.0401970.0317340.0317344AlexNet [81] versus ResNet [75]0.0111180.2334690.2001160.1667630.1111765VGG16 [98] versus DenTnet [ours]0.0111180.2334690.2001160.1667630.1222936Inception V3 [88] versus SqueezeNet [184]0.0161570.3392960.2585110.2423540.1777267VGG19 [98] versus SqueezeNet [184]0.0325090.6826980.4876420.4876420.2925858ResNet [75] versus VGG16 [98]0.0450210.9454390.6302920.4952300.3151469VGG19 [98] versus DenTnet [ours]0.0450210.9454390.6302920.4952300.40518810Inception V3 [88] versus DenTnet [ours]0.0823521.7293970.9882270.9058740.49411311VGG16 [98] versus SqueezeNet [184]0.1088092.2849981.1969041.1969040.65285712ResNet [75] versus VGG19 [98]0.1415792.9731561.4157891.4157890.65285713AlexNet [81] versus Inception V3 [88]0.1814493.8104331.6330431.6330431.27014414ResNet [75] versus Inception V3 [88]0.2291024.8111401.8328151.6330431.27014415AlexNet [81] versus VGG19 [98]0.2850495.9860381.9953461.9953461.27014416AlexNet [81] versus SqueezeNet [184]0.2850495.9860381.9953461.9953461.42524717VGG16 [98] versus Inception V3 [88]0.4226788.8762402.1133902.1133901.69071218AlexNet [81] versus VGG16 [98]0.59298012.4525822.3719202.3719201.77894019VGG16 [98] versus VGG19 [98]0.59298012.4525822.3719202.3719201.77894020ResNet [75] versus DenTnet [ours]0.59298012.4525822.3719202.3719201.77894021VGG19 [98] versus Inception V3 [88]0.78926816.5746292.3719202.3719201.778940


## 7.4. Critical Distance Diagram from Nemenyi [210] Test

Nemenyi [210] test is very conservative with a low power, and hence it is not a recommended choice in practice [215]. Nevertheless, it has a unique advantage of having an associated plot to demonstrate the results of fair comparison. Figure 11 depicts the Nemenyi [210] post hoc critical distance diagrams at three distinct levels of significance α values. If the distance between algorithms is less than the critical distance, then there is no statistically significant difference between them. The diagrams in Figures 11(a) and 11(b) associated with α=0.10 with the critical distance of 3.3588 and with α=0.05 with the critical distance of 3.6768, respectively, are identical, whereas the diagram in Figure 11(c) related to α=0.01 with the critical distance of 4.3054 is different. Any two algorithms are considered as significantly different if their performance variation is greater than the critical distance. To this end, from Figure 11, it is noticeable that, at α=0.01, both SqueezeNet [184] versus DenTnet [ours] and SqueezeNet [184] versus ResNet [75] are remarkably different, while other pairs are not remarkably divergent as their performance differences are less than 4.3054. As compared to ResNet [75], DenTnet [ours] differs from SqueezeNet [184] by a greater distance. On the other hand, SqueezeNet [184] versus DenTnet [ours] and AlexNet [81] versus DenTnet [ours] are significantly different at both α=0.10 and α=0.05, whereas SqueezeNet [184] versus ResNet [75] is significantly dissimilar at those α values. Straightforwardly, DenTnet [ours] is outstandingly unalike both SqueezeNet [184] and AlexNet [81], but ResNet [75] is not outstandingly unalike AlexNet [81]. This implies that the method of DenTnet [ours] outperforms that of ResNet [75], which also agrees with the finding in Figure 10.Figure 11
Nemenyi [210] post hoc critical distance diagrams for three α values using data in Table 11.
(a)(b)(c)


## 7.5. Reasons of Superiority

In this study, DenseNet [67] was a great choice as it was very compact and deep. It used less training parameters and reduced the risk of model overfitting and improved the learning rate. In the dense block of DenTnet, the outputs from the previous layers were concatenated instead of using the summation. This type of concatenation helped to markedly speed up the processing of data for large number of columns. The dense block of DenTnet contained convolution and nonlinear layers, which applied several optimization techniques (e.g., dropout and BN). DenTnet scaled to hundreds of layers, while exhibiting no optimization difficulties. Overall, this model was applied to a very large number of preprocessed augmented images from BreaKHis [33], Malaria [191], SkinCancer [193], and CovidXray [192] datasets. To the best of our knowledge, no other studies in the literature had such an edge. Additionally, the use of data augmentation approach in this study positively affected the performance of the model due to expansion in the size of training data, which is the foremost requirement of a deep network for its proper working. Our DenTnet was well trained through various parameters’ tuning. For example, in the case of BreaKHis [33], unlike other existing models, our model was trained on all the magnifications combined (40x, 100x, 200x, and 400x) to avoid any loss of generality.In sum and substance, based on the aforementioned experimental and nonparametric statistical test results, it is, therefore, possible to conclude that the proposed DenTnet [ours] outperformed AlexNet [81], ResNet [75], VGG16 [98], VGG19 [98], Inception V3 [88], and SqueezeNet [184] in terms of computational speed. Significantly, the accuracy achieved by the proposed DenTnet [ours] surpassed those of existing state-of-the-art models in classifying images of the BreaKHis [33], Malaria [191], SkinCancer [193], and the CovidXray [192] dataset.


## 7.6. Limitation of Proposed Model and Methodology

Despite these promising results, questions remain as to whether the proposed DenTnet model could be utilized to classify categorical images. Moreover, DenTnet was tested with one breast cancer dataset (i.e., BreaKHis [33]) only. Although the generalization ability of DenTnet with three non-breast-cancer-related datasets was studied in Section 6, it is unknown whether DenTnet can generalize to other state-of-the-art breast cancer datasets. Future work should, therefore, investigate the efficacy and generalizability of DenTnet with datasets along with multiclass labels, as well as other publicly available breast cancer datasets (e.g., the most recently introduced MITNET dataset [216]).The classification effect of breast cancer histopathological images of any deep learning methodology is related to the features and many studies predominantly focused on how to develop good feature descriptors and better extract features. Different from traditional handcrafted feature-based models, DenTnet can automatically extract more abstract features. Nevertheless, it is worth noting that although the proposed DenTnet has addressed the cross-domain problem by utilizing the transfer learning approach, features extracted in the methodology are solely deep-network-based features, which are extracted by feeding images directly to the model. However, feeding deep models directly with images would not generalize as the models consider color distribution of an image. It is understood that local information can be captured from color images using Local Binary Pattern (LBP) [217]. Therefore, future work can use multiple types of features by combining the features extracted by the proposed method with LBP features to address this issue.


## 8. Conclusion

We presented that, for classifying breast cancer histopathological images, the most popular training-testing ratio was 70%: 30%, while the best performance was indicated by the training-testing ratio of 80%: 20%. We proposed a novel approach named DenTnet to classify histopathology images using training-testing ratio of 80%: 20%. DenTnet achieved a very high classification accuracy on the BreaKHis dataset. Several impediments of existing state-of-the-art methods including the requirement of high computation and the utilization of the identical feature distribution were attenuated. To test the generalizability of DenTnet, we conducted experiments on three additional datasets (Malaria, SkinCancer, and CovidXray) with varying difficulties. Experimental results on all four datasets demonstrated that DenTnet achieved a better performance in terms of accuracy and computational speed than a large number of effective state-of-the-art classification methods (AlexNet, ResNet, VGG16, VGG19, InceptionV3, and SqueezeNet). These findings contributed to our understanding of how a lightweight model could be used to improve the accuracy and accelerate the learning process of images, including histopathology image classification on using the wild state-of-the-art datasets. Future work shall investigate the efficacy of DenTnet on datasets with multiclass labels.



---

*Source: 8904768-2022-10-10.xml*