# Using artificial immune algorithm for fast convergence of multi layer perceptron in breast cancer diagnosis application

Rima Daoudi, Khalifa Djemal, Abdelkader Benyettou

► To cite this version:

Rima Daoudi, Khalifa Djemal, Abdelkader Benyettou. Using artificial immune algorithm for fast convergence of multi layer perceptron in breast cancer diagnosis application. 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA 2015), Nov 2015, Orléans, France. pp.341-345, 10.1109/IPTA.2015.7367161. hal-01277528

HAL Id: hal-01277528

https://hal.science/hal-01277528

Submitted on 12 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Using Artificial Immune Algorithm for Fast Convergence of Multi Layer Perceptron in Breast Cancer Diagnosis Application

Rima Daoudi¹,², Khalifa Djemal¹ and Abdelkader Benyettou²

¹ IBISC Laboratory, University Of Evry Val d'Essonne, France

e-mail: rima.daoudi@ibisc.univ-evry.fr, khalifa.djemal@ibisc.univ-evry.fr

² SIMPA Laboratory, University of Sciences and Technologies of Oran, Algeria

e-mail: aek.benyettou@univ-usto.dz

**Abstract**—In this paper, a Multi Layer Perceptron (MLP) based Artificial Immune System (AIS) is presented for breast cancer classification. The proposed algorithm integrates clonal selection principle of AIS in MLP learning to reduce its computational costs and accelerate its convergence to a Mean Squared Error Threshold (MSE<sub>th</sub>) set by the user. Applied on the Wisconsin Diagnosis Breast Cancer database (WDBC), the results show that combining Artificial Immune Systems and Neural Networks is effective. Indeed, a significant reduction of computation time has been obtained with a slight improvement of classification accuracy.

**Keywords**—Multi Layer Perceptron, Artificial Immune System, Classification, Breast Cancer, Mean Squared Error, Speed Convergence.

## I. INTRODUCTION

The correct diagnosis of breast cancer has become a major problem in the medical field since this type of cancer is ranked among the leading causes of death among the ladies. Although some risk reduction can be achieved through prevention, strategies in this sense are not able to eliminate the majority of breast cancers that occur in low- and middle-income countries. Early detection remains the main way to fight against this disease; it improves the chances of survival as well as breast cancer outcome. Indeed, if the cancerous cells are detected before spreading to other organs, the survival rate for patients is more than 97% [1]. For an ultimate diagnosis, the patient undergoes several tests which go from the mammography until the biopsy in certain difficult cases. In order to help experts to reduce the number of errors they can do and minimize the number of unnecessary biopsies, Computer-Aided Diagnosis systems (CAD) are increasingly used to directly reach the final diagnosis.

Several articles have been published presenting new CAD systems, authors in [2] proposed an automatic diagnosis system for detecting breast cancer based on Association Rules (AR) and Neural Network, they used AR for reducing the dimension of breast cancer database and the MLP neural network as classifier, the classification accuracies were 95.6% for four inputs and 97.4% for eight inputs. In [3] three different methods, Optimized Learning Vvector quantization (LVQ), big LVQ, and Artificial Immune Recognition System (AIRS), were applied and the obtained accuracies are 96.7%, 96.8%, and

97.2%, respectively. Classification accuracy obtained by the application of a feature extraction algorithm using a hybrid of K-Means and Support Vector Machine algorithms in [4] is 97.38%. Work in paper [5] presented an Artificial Meta-plasticity Neural Network reaching a classification accuracy of 99.26%. Authors in [6] presented a decision support system for early breast cancer detection; the proposed method employs a Correlation Feature Selection procedure and AIRS algorithm for classification, the classification accuracy obtained is 83.33%. Work in [7] proposed a new artificial immune algorithm which uses the clonal selection principle obtaining a classification accuracy of 93.4%. Several Improvements on the artificial immune algorithm CLONALG were applied in [8] and [9], classification accuracies were 94.40% and 98.58% respectively. This work aims to use the advantages of Artificial Immune Systems (AIS), to enhance the learning of Multi Layer Perceptron neural network in order to perform a quick and efficient diagnosis of breast cancer.

### A. Multi Layer Perceptron Neural Network (MLP)

Artificial Neural Networks (ANNs) are computational models that simulate the functions of recognition of human forms. They are powerful parallel models that provide excellent resolution of problems in many areas. Neurons are interconnected with connection links which have weights that are multiplied by the signal transmitted in the network [10]. The output of each network is determined by use of an activating function as the Sigmoid or Gaussian [11].

Among all the various types of ANN architectures, Multi Layer Perceptron is the well known and the most frequently used [12]. Basically MLP network is composed of an input layer, one or more hidden layers and one output layer. It is a supervised learning model which uses the Back Propagation (BP) algorithm to improve its efficiency. That consists of adjusting the network weights in order to reduce the Mean Squared Error (MSE) obtained by:

$$MSE = \frac{\sum_{k=1}^{N} (D_k - Y_k)^2}{N} \quad (1)$$

Where N is the total number of data, D<sub>k</sub> is the desired output of the k<sup>th</sup> data, and Y<sub>k</sub> is the actual output of the k<sup>th</sup> data.
data obtained by:

$$Y_k = f(\sum_i W_i \cdot X_i) \qquad (2)$$

Where $X_i$ corresponds to the input vector, and $W_i$ is the weight vector and $f$ the activation function.

Despite its great success, MLPs showed some limitations and problems during training. A serious drawback of MLPs is the slow rate of convergence [13]. Several modifications have been proposed to enhance this algorithm, but no one is able of producing satisfactory results with fast convergence, thus the search for an approach to speed-up its convergence and improve its performance still remains important.

## B. Artificial Immune System (AIS)

As ANNs, models of Artificial Immune Systems have been mainly developed for analyzing the behavior of the Natural Immune System (NIS) in the field of medicine, but later its properties as pattern recognition, robustness; its hybrid structures, distributed processing and self-organization took the attention of some researchers. AIS is a distributed system that can perform classification tasks, recognition and learning using the extraction process, communication and memorization.

The major part of AIS work to date has been the development of three algorithms derived from more simplified models; negative selection [14], clonal selection [15] and immune networks [16]. The immune principles used in the most AIS algorithms are: evaluation of affinity, selection, cloning and mutation. The goal is to build a consistent set of memory cells that will be used in classification step for the recognition of unknown examples by the system.

Because the human immune system operates as a cognitive mechanism that recognizes patterns that may not be recognized by the nervous system, the AIS can be used as a complement to the neural network architectures. As well, new structures based on immunology can be proposed as an alternative to ANNs, or structures of hybrid networks composed of both systems.

In this study, a Multi Layer Perceptron based Artificial Immune System is presented for classifying breast lesions in Benign / Malignant. This method consists of using the clonal selection principle in MLP learning to accelerate its convergence for a specified error without decreasing its accuracy.

The rest of the paper is organized as follows: in the next section, we present the Wisconsin Diagnosis Breast Cancer database used in evaluation. The description of the proposed algorithm is given in section 3. Section 4 provides a discussion of the obtained results and we conclude our work in section 5.

## II. WISCONSIN DIAGNOSIS BREAST CANCER DATABASE (WDBC)

In this study, the Wisconsin Diagnosis Breast Cancer Database is used [17]. WDBC consists of data from 569 breast fine needle aspirate (FNA) cases containing 32 descriptive

features where the two first features correspond to a unique identification number and the diagnosis status (benign or malignant). The rest 30 features are computed from a digitized image of a FNA of a breast mass. They describe characteristics of the cell nuclei present in the image. Samples of FNA images are shown in figure 1. The case distribution includes 357 cases of benign breast changes and 212 cases of malignant breast cancer. The descriptive features are recorded with four significant digits and include:

1) Radius = radius (mean of distances from center to points on the perimeter)

2) Texture = texture (standard deviation of gray-scale values)

3) Perimeter = perimeter of the cell nucleus

4) Area = area of the cell nucleus

5) Smoothness = smoothness (local variation in radius lengths)

6) Compactness = compactness (perimeter²/area - 1.0)

7) Concavity = concavity (severity of concave portions of the contour)

8) Concavepts = concave points (number of concave portions of the contour)

9) Symmetry = symmetry (measure of symmetry of the cell nucleus)

10) FracDim = fractal dimension ("coastline approximation" - 1)

<figure><img src="image_2.png" /><figcaption>Fig. 1. Images taken using the Fine Needle Aspirate test</figcaption></figure>

## III. PROPOSED MLP-AIS ALGORITHM

In this section, the MLP based AIS approach. In order to increase convergence speed and reduce computational costs without affecting accuracy, this study uses a Clonal Selection Artificial Immune System to assist the MLP to converge more quickly towards a Mean Squared Error Threshold ($MSE_{th}$) set by the user. The proposed AIS-MLP algorithm adds the principles of cloning and mutation to the training algorithm of MLP to help in search for optimal weights that minimize the mean squared error. The main purpose of an immune system is to recognize foreign cells that attack the body, if the difference between the antibody and the foreign cell (antigen) is little, affinity between this antigen and the antibody is great which mean that recognition is more likely. For a neural network the objective is to update the connection weights in order to minimize the distance between the obtained output and the desired output. This distance is represented by Mean Squared Error (MSE) between the two outputs, in our algorithm the
affinity itself the MSE (1). We present in next subsections a description of each step of the proposed algorithm.

### A. Normalization

Before starting the algorithm, the database is normalized in [0, 1], according to the following equation:

$$R = \frac{X_i - X_{min}}{X_{max} - X_{min}} \qquad (3)$$

Where $X_{min}$ is the minimum of the data $X_i$ for all i, $X_{max}$ is the maximum of the data $X_i$ for all i, and $X_i$ is the $i^{th}$ data of data X.

### B. MLP Weights Initialization

Randomly generate the initial weights of the neural network. The goal of this work is to update these weights in every iteration of the algorithm to find the optimal weights that minimize the mean squared error in minimal time. The choice of these initial weights directly affects the results of the algorithm. Calculate the MSE using these initial weights.

### C. Repeat until the MSE <= MSE<sub>th</sub> set by the user

1) Backpropagation and weights update : To correct obtained error, we must change the weight $W_{j,i}$ so as to minimize the error E calculated by:

$$E = \sum (D_j - Y_j)^2 \qquad (4)$$

Where $D_j$ and $Y_j$ are respectively the desired and the actual outputs of the $j^{th}$ neuron.

The principle of backpropagation is used to calculate $\Delta W_{j,i}$ to update the weights of MLP according to the equation:

$$W_{j,i+1} = W_{j,i} + \Delta W_{j,i} \qquad (5)$$

Where:

* The index j represents the neuron for which the weight is to adapt.

* The index i represents a neuron of the previous layer relative to neuron j.

The variation of $\Delta W_{j,i}$ will depend on the gradient of this error, according to the gradient descent algorithm. This can be expressed by the following equation:

$$\Delta W_{j,i} = -\mu \frac{\partial E}{\partial W_{j,i}} \qquad (6)$$

With $0 \le \mu \le 1$ representing the Learning Step-Size. A detailed description of PB algorithm can be found in [18]

2) Evaluation: This step consists in computing the mean squared error using the new updated weights ($MSE_{BP}$).

3) Cloning: In AIS cloning consists in creating copies of selected antibodies to increase their competitiveness. At the end of this step we dispose of a number of identical copies of the updated weights matrix, these clones shall undergo to the operator of mutation in the following step.

4) Mutation: The mutation is a random change in one or more positions of the attribute values of the clone. This process is intended to train the clones to become better or worse antibodies, less good clones are eliminated by selection, there will remain only the best. In our algorithm, the mutation consists of reviewing some clones values by a random process in order to find a better weight matrix than the original (the updated one).

5) Evaluation and selection of the best mutated clone: In this step we compute the affinity of all the mutated clones in the previous step and we select the one who maximizes the value of affinity (minimizes MSE). A comparison between this minimal MSE and MSEBP calculated in step C.2 is made, if the affinity of the best clone is better than that of the original one, then an update of the weights matrix is done by replacing the original matrix by the best mutated clone and the $MSE_{BP}$ by the MSE of this best clone.

At the end of the iteration, we test if it fulfils the criterion for termination ($MSE \le MSE_{th}$). If not, we return to step C.1. If yes, the algorithm ends. The final weights matrix represents the solution of the algorithm i.e. the optimal weights of the MLP which will be used for the classification of breast cancer. The algorithmic description of the proposed modified MLP is given by pseudo-code shown in algorithm 1.

### Algorithm 1 Pseudo-code of the proposed MLP-AIS algorithm

A. Normalization of data

B. Generate Random Initial MLP weights

while $MSE < MSE_{th}$ do

1) Backpropagation and weights update

2) Evaluation (Calculate MSEBP using the updated weights)

3) Clone the new weights matrix

4) Mutate the clones

5) Evaluation and selection of the best mutated clone

if $MSE_{BestClone} < MSE_{BP}$ then

Weights matrix = Best Clone

$MSE = MSE_{BestClone}$

end if

end while(iteration)

## IV. EXPERIMENTAL RESULTS

The overall results of this research are determined in this section. First we present the parameters used in evaluation, then all achieved tests are given and discussed in subsequent sections.
## A. Algorithm parameters

In our analysis the MLP model is composed of 30 inputs which represent the 30 features of the WDBC, and one hidden layer. The Mean squared Error ($MSE_{th}$) Threshold was set to $10^{-4}$ and the activation function used is the sigmoid function defined by:

$$f(x) = \frac{1}{1 + e^{-x}} \qquad (7)$$

It is necessary to control two parameters of the MLP: the Number of Neurons in the Hidden Layer, and the Learning Step-Size ($\mu$). To determine the best combination of these two parameters, we selected five values for the Learning Rate (LR): 0.1, 0.2, 0.5, 0.8 and 1 and five other values for the Number of Neurons in the Hidden Layer (NNHL): 2, 5, 10, 15 and 20 neurons. The classification accuracies of each test are listed in table 1.

<table><caption>Table 1. Classification accuracies of different values of training rate and hidden neurons on WDBC</caption><thead><tr><th>NNHL</th><th>0.1</th><th>0.25</th><th>0.5</th><th>0.75</th><th>1</th></tr></thead><tbody><tr><th>2</th><td>98.50</td><td>98.50</td><td>98.12</td><td>97.75</td><td>96.25</td></tr><tr><th>5</th><td>98.50</td><td>97.75</td><td>98.32</td><td>93.63</td><td>90.63</td></tr><tr><th>10</th><td>98.50</td><td>99.06</td><td>89.51</td><td>82.39</td><td>72.28</td></tr><tr><th>15</th><td>96.62</td><td>95.88</td><td>77.15</td><td>56.92</td><td>56.92</td></tr><tr><th>20</th><td>95.88</td><td>93.63</td><td>53.08</td><td>43.44</td><td>44.19</td></tr></tbody></table>

From table 1 we can see that the best classification result is obtained with a learning step-size of 0.25 and 10 neurons in the hidden layer. For the AIS algorithm, we fixed the number of clones to 5 and the affinity measure is the MSE. Table 2 summarizes the setting used in evaluation.

<table><caption>Table 2. Optimal parameters used in evaluation</caption><thead><tr><th>Parameter</th><th>Value</th></tr></thead><tbody><tr><td>Number of inputs</td><td>30</td></tr><tr><td>Number of hidden layers</td><td>1</td></tr><tr><td>MSE<sub>Threshold</sub></td><td>10<sup>-4</sup></td></tr><tr><td>μ</td><td>0.25</td></tr><tr><td>NNHL</td><td>10</td></tr><tr><td>Number of clones</td><td>5</td></tr><tr><td>Affinity measure</td><td>MSE</td></tr></tbody></table>

## B. 2) Results and Discussion

Based on tests performed above to set the parameters of the algorithm, the performance of AIS-MLP is studied on WDBC using in Matlab® (Matlab R2013a 1.1.0.604. We shared the database into 75% for training and 25% for the test. Table 3 presents the results of five successive runs of MLP and AIS-MLP using different examples for train and test in each run. The average of this five runs is taken as final result.

From the table above it is easily noticeable that the proposed approach greatly reduces the convergence time of the MLP to the ($MSE_{th}$) set by the user, with a slight improvement of

<table><caption>Table 3. Average of Performances of 5 successive runs of MLP and AIS-MLP on WDBC</caption><thead><tr><th>Algorithm</th><th>Accuracy (%)</th><th>iterations.Nb</th><th>Execution Time (s)</th></tr></thead><tbody><tr><td>MLP</td><td>99.32</td><td>2705</td><td>44.01</td></tr><tr><td>AIS-MLP</td><td>99.39</td><td>670</td><td>22.84</td></tr></tbody></table>

classification accuracy. The necessary number of iterations to converge is reduced by more than 75%, and execution time is reduced by 50%. This reduction of computational costs is done thanks to the mutation operator that helps the algorithm to converge faster to the desired error by changing the values of the weight of MLP by a random process, and update the weights only if they have better affinity than their original cell. Figures 2 and 3 plot respectively comparisons of convergence curves and computational costs of the two approaches on WDBC.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Comparison between Convergence curves of MLP and AIS-MLP on WDBC</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. Comparison between computational costs of MLP and AIS-MLP on WDBC</figcaption></figure>

To validate our method, Table 4 provides final evaluation based on the comparison of classification accuracies of our algorithm and previous methods from literature applied on WDBC. As can be seen from the results AIS-MLP approach obtains excellent classification accuracy.
<table><caption>Table 4. Classification accuracies obtained with our method and other classifiers from literature on WDBC</caption><thead><tr><th>Author</th><th>Method</th><th>Classification Accuracy</th></tr></thead><tbody><tr><td rowspan="2">[19]</td><td>MNN</td><td>98.22%</td></tr><tr><td>RBFNN</td><td>97.25%</td></tr><tr><td>[20]</td><td>SVM</td><td>96.32%</td></tr><tr><td rowspan="2">This study</td><td>MLP</td><td>99.32%</td></tr><tr><td>AIS-MLP</td><td>99.39%</td></tr></tbody></table>

## V. CONCLUSION

In this paper, we presented an improved MLP neural network algorithm. The approach uses the Artificial Immune Systems to find the optimal weights of the neural network which minimize the MSE in a minimum of time. Based on the principles of Back Propagation of MLP and the mutation of the AISs, the proposed hybrid algorithm shows a great performance obtaining a total classification accuracy of 99.39% while significantly reducing the calculation costs comparing to the classical MLP neural network. Moreover, the results show that the combination of the AIS and the MLP can enhance the performance of the neural networks. Based on this work, we can say that the proposed approach can be used as a diagnostic tool in conjunction with other medical tests for early detection of breast cancer.

## REFERENCES

[1] American Cancer Society Hompage. [Online]. Available: http://www.cancer.org

[2] M. Karabatak and M. C. Ince, "An expert system for detection of breast cancer based on association rules and neural network," *Expert Systems with Applications*, vol. 36, no. 2, pp. 3465-3469, 2009.

[3] D. Goodman, L. Boggess, and A. Watkins, "Artificial immune system classification of multiple-class problems," *Proceedings of the artificial neural networks in engineering ANNIE*, vol. 2, pp. 179-183, 2002.

[4] B. Zheng, S. W. Yoon, and S. S. Lam, "Breast cancer diagnosis based on feature extraction using a hybrid of k-means and support vector machine algorithms," *Expert Systems with Applications*, vol. 41, no. 4, pp. 1476-1482, 2014.

[5] A. Marcano-Cedeño, J. Quintanilla-Domínguez, and D. Andina, "Wbcd breast cancer database classification applying artificial metaplasticity neural network," *Expert Systems with Applications*, vol. 38, no. 8, pp. 9573-9579, 2011.

[6] C. Katsis, I. Gkogkou, C. Papadopoulos, Y. Goletsis, and P. Boufounou, "Using artificial immune recognition systems in order to detect early breast cancer," *International Journal of Intelligent Systems and Applications (IJISA)*, vol. 5, no. 2, p. 34, 2013.

[7] A. Sharma and D. Sharma, "Clonal selection algorithm for classification," in *Artificial Immune Systems*. Springer, 2011, pp. 361-370.

[8] R. Daoudi, K. Djemal, and A. Benyettou, "An immune-inspired approach for breast cancer classification," in *Engineering Applications of Neural Networks*. Springer, 2013, pp. 273-281.

[9] —, "Cells clonal selection for breast cancer classification," in *Systems, Signals & Devices (SSD), 2013 10th International Multi-Conference on*. IEEE, 2013, pp. 1-4.

[10] E. R. Kandel, J. H. Schwartz, T. M. Jessell et al., *Principles of neural science*. McGraw-Hill New York, 2000, vol. 4.

[11] D. Andina, A. Jevtić, A. Marcano, and J. B. Adame, "Error weighting in artificial neural networks learning interpreted as a metaplasticity model," in *Bio-inspired Modeling of Cognitive Tasks*. Springer, 2007, pp. 244-252.

[12] N. A. Hamid, N. M. Nawi, R. Ghazali, and M. N. M. Salleh, "Accelerating learning performance of back propagation algorithm by using adaptive gain together with adaptive momentum and adaptive learning rate on classification problems," in *Ubiquitous Computing and Multimedia Applications*. Springer, 2011, pp. 559-570.

[13] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," *DTIC Document, Tech. Rep.*, 1985.

[14] D. Dasgupta and N. S. Majumdar, "Anomaly detection in multidimensional data using negative selection algorithm," in *Computational Intelligence, Proceedings of the World on Congress on*, vol. 2. IEEE, 2002, pp. 1039-1044.

[15] L. N. De Castro and F. J. Von Zuben, "The clonal selection algorithm with engineering applications," in *Proceedings of GECCO*, vol. 2000, 2000, pp. 36-39.

[16] L. N. de Castro and F. J. Von Zuben, "ainet: an artificial immune network for data analysis," *Data mining: a heuristic approach*, vol. 1, pp. 231-259, 2001.

[17] U. M. L. Repository. Breast cancer wisconsin (diagnostic) data set. [Online]. Available: https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/

[18] S. Abid, A. Mouelhi, and F. Fnaiech, "Accelerating the multilayer perceptron learning with the davidon fletcher powell algorithm," in *Neural Networks, 2006. IJCNN'06. International Joint Conference on*. IEEE, 2006, pp. 3389-3394.

[19] H. Vazirani, A. Shukla, R. Kala, and R. Tiwari, "Diagnosis of breast cancer by modular neural network," in *Computer Science and Information Technology (ICCSIT), 2010 3rd IEEE International Conference on*, vol. 7. IEEE, 2010, pp. 115-119.

[20] H. Guo and A. K. Nandi, "Breast cancer diagnosis using genetic programming generated feature," *Pattern Recognition*, vol. 39, no. 5, pp. 980-987, 2006.