# AI-Based Classification Algorithm of Infrared Images of Patients with Spinal Disorders

Anna Poplavska, Valentina Vassilenko, Oleksandr Poplavskyi, Diogo Casal

► To cite this version:

Anna Poplavska, Valentina Vassilenko, Oleksandr Poplavskyi, Diogo Casal. AI-Based Classification Algorithm of Infrared Images of Patients with Spinal Disorders. 12th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Jul 2021, Costa de Caparica, Portugal. pp.316-323, 10.1007/978-3-030-78288-7_30. hal-03685915

HAL Id: hal-03685915

https://inria.hal.science/hal-03685915

Submitted on 2 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
AI-based Classification Algorithm of Infrared Images
of Patients with Spinal Disorders

Anna Poplavska¹,², Valentina Vassilenko¹,², Oleksandr Poplavskyi³, Diogo Casal⁴

¹ Laboratory of Instrumentation, Biomedical Engineering and Radiation Physics (LIBPHYS), NOVA School of Science and Technology - NOVA University Lisbon

² NMT, S.A., Parque Tecnológico de Cantanhede, Núcleo 04, Lote 3, 3060 - 197 Cantanhede, Portugal

³ Kyiv National University of Construction and Architecture, Ukraine

⁴ NOVA Medical School|Faculdade de Ciências Médicas (NMS|FCM)

**Abstract.** Infrared thermal imaging is a non-destructive, non-invasive technique that has shown to be effective in the detection and pre-clinical diagnosis of a variety of disorders. Nowadays, some medical applications have already been successfully implemented in pre-clinic diagnostics using thermography based on AI algorithms to support decision-based medical tasks. Though, the massive amount of image types, disease variety, and numerous individual anatomical features of the human body continue to give researchers more challenging jobs that still need to be solved. This paper proposes a novel methodology using a convolutional neural network (CNN) for analyzing with high accuracy infrared thermal images from the spine region for quick screening and disease classification of patients.

**Keywords:** biomedical applications, medical thermography, AI decision making, convolutional neural network, spinal disorders

# 1 Introduction

Medical applications based on AI algorithms have been successfully implemented in monitoring and pre-clinic diagnostics of complex medical data. Advances in AI development will change approach clinical problem-solving and seem to make it possible to address issues like misdiagnosis or detecting disease on the last stage. However, while some AI applications can compete with doctors to solve challenging medical tasks, they still must also be completely incorporated, implemented, and improved to be utilized in daily medical routines. The enormous amount of image types, disease variety, and numerous individual anatomical features of the human body continue to give researchers challenging tasks and undergoing extensive evaluation.

Infrared thermal imaging (IRTI) techniques due to their non-invasive, non-destructive, and painless way have obvious benefits among other diagnostic methods. IRTI in the medical field can give high-quality thermal images for disease monitoring
and pre-clinical diagnosis in real-time by displaying thermal anomalies present in the body. Some recent publications discussed the clinical utility of infrared thermography for spinal diseases, as diagnoses and evaluate pain symptoms [1], scoliosis [2,3], rheumatic diseases [4], spinal disorders [5,6], etc. But the majority of research methods use manual feature extraction and traditional methods of classification.

In recent years, thermal data starting to be applied by researchers as input features for AI classifiers that tried automatically solve challenging tasks in the medical field. Convolutional neural network (CNN) for detecting Alzheimer's disease [7], Artificial Neural Networks (ANN) for detecting diabetes disease [8], Support Vector Machines (SVM) for breast cancer detection [9], k-Nearest Neighbor (k-NN) to segment hands, aiding in the evaluation of this health problem, Decision Trees (DT) for the detection of the existing rheumatoid arthritis disease for improved diagnosis [10], Fuzzy methods [11], etc. are some examples of this type of learning. Most research works based on the developing AI algorithms for medical thermal images are focusing mainly on breast cancer detection [12-14] through a considerable number of open-access databases. A breast cancer diagnosis is a topic with the highest number of publications for the thermal imaging application with ML algorithms. The latest article focused on the systematic review and analysis of the application of ML classifiers using infrared thermography for biomedical applications published in 2021 identified 719 records through database searching. After proper review and analysis of the eligibility criteria from identified records, the quantitative synthesis was performed only for 34 of the 68 encountered publications [15] and only one article related to the back problems [16]. There were not found publications on implementing the AI analysis for infrared image classification of patients with any spinal disorders. Consequently, the present research study aimed to develop an AI algorithm for the classification of infrared images database of patients with spinal disorders to healthy and unhealthy classes which were not done before.

## 2 Relationship to Applied AI Systems

Based on the statistical analysis of the global disease database 266 million individuals (3.63%) worldwide have degenerative spine disease, from which 39 million individuals (0.53%) were found to have spondylolisthesis, 403 million (5.5%) individuals with symptomatic disc degeneration, and 103 million (1.41%) individuals with spinal stenosis annually [17]. Spinal disorders significantly limit mobility and agility, increasing the disability and leading to early retirement, lower well-being levels, and reduced ability to participate in society. Degenerative spine conditions (DSC) are characterized by the progressive loss of normal spine structure and function over time, i.e. treatment varying depending on the type and severity of the disorder. [17]. The treatment process began at a later stage leads to a long rehabilitation process or painful operation treatment. Early detection of spinal problems is not only more likely to continue a healthy human lifestyle but is also essential for better care and a lower treatment period.

The development of medical applications based on AI algorithms using infrared image data is a growing and promising research worldwide. These applications have
enormous potential in medical imaging technology, treatment process, medical meta-
analysis, and diagnostics. The fundamental difference between our research and other
works (based on traditional feature selection or classification) is that there is no
necessity to define features in the pre-processed dataset since the CNN is responsible
for determining which image features belong to which class. A significant contribution
of the current research consists of developing of methodology for the classification of
our infrared images database of patients with spinal disorders based on a CNN
algorithm, which was not done before.

## 3 Materials and Methods

To build an efficient algorithm, firstly, computer systems must be fed structured data, i.e each pixel from the image has a label identifiable to the algorithm. After the CNN has been exposed to a sufficient number of data sets and labels, the performance results are dissected to ensure precision. The algorithm typically includes known data, allowing evaluation of the algorithm's capacity to find out the correct answer. Based on the testing results, CNN can be modified or fed more data to improve outcome results.

**3.1. Sample Size**

The dataset used in this work comprises 480 infrared images of the region of interest of patients with different types of spinal disorders and 220 infrared images of the healthy participants from two different European countries, Portugal and Ukraine, were obtained [4,5]. The dataset was divided into two groups as healthy or with the spinal disease, as is shown in fig.1:

<figure><img src="image_2.png" /><figcaption>Fig. 1. Example of the spinal thermogram of participant (a) with spinal disease and (b) healthy</figcaption></figure>

Since to use CNN architecture the database is necessary to be balanced, images of
the healthy set were increased to 320. CNN classification in our approach was
performed using each image of each patient.

**3.2. Measurement Protocol**

The majority of the measurements were performed under supervision by the medical
staff of the Center of Medical Rehabilitation & Sports Medicine, Vinnytsia, Ukraine,
hence to the experimental protocol described elsewhere [4,5,18]. The infrared (IR)
images were obtained in a laboratory room with a maintained temperature of 21°C and humidity of 50%, approximately, using IR camera FLIR® E6 (320x240) with <0.06°C thermal sensitivity. The emissivity parameter for the skin (0.98) was set in the thermal camera settings. Before the measurements the thermal camera was switched on for 15 minutes, as well as each volunteer had to stay at the ambient temperature for accommodation. As a backdrop, a black matte curtain was used. Each participant, during the measurements, stood in the same position, the arms down and facing the wall with the head straight ahead.

### 3.3. Deep Learning Model

The Convolutional Neural Network (CNN) is a class of DL based on the idea that the model function properly fits a local understanding of the image. The main types of learning are: supervised (reference label must be accessible to each image), semi-supervised (reference label have only a subset of all images), and unsupervised learning (reference label is unavailable), as is shown in fig.2.:

<figure><img src="image_3.png" /><figcaption>Fig. 2. The main types of learning</figcaption></figure>

CNN is defined as the integral of the product of the two functions after one is reversed and shifted. The integral is evaluated for all values of shift, producing the convolution function [3,15]. A digital image during image processing can be viewed as a discrete function of a two-dimensional area, marked as f(x,y). Assuming the existence of a two-dimensional convolution function g(x, y), the output image z(x,y), which can be used to extract image features is represented by the following formula:

$$z(x, y) = f(x, y) * g(x, y) \qquad (1)$$

Similarly, in deep learning (DL) applications the input image is a multi-dimensional array of 3 × image width × image length when the input is a color image containing RGB three channels and the convolution kernel is defined in the learning algorithm as the accounting. The computational argument is a multi-dimensional array as well. After, when two-dimensional images are fed into the program, the following formula is used to calculate the corresponding convolution operation:

$$z(x, y) = f(x, y) * g(x, y) = \sum_{t} \sum_{h} f(t, h)g(x - t, y - h) \qquad (2)$$

If a convolution kernel of m×n dimension array is given, the formula is:
$$z(x, y) = f(x, y) * g(x, y) = \sum_{t=0}^{t=m} \sum_{h=0}^{h=n} f(t, h)g(x - t, y - h) \quad (3)$$

where f – the input image G.

CNN architecture consists typically of a convolution layer, a non-linear activation layer, and a max-pooling layer [1,4,15]. To create a set of linear activations, the convolution layer operations are executed in parallel. The convolution operation's goal is to extract high-level features from the input image. Then each linear activation is executed through the most used activation functions rectified linear activation function. The pooling layer, like the convolutional layer, is in charge of reducing the spatial size of the convolved feature. Through dimensionality reduction, the computational power required to process the data is reduced. Furthermore, it is useful for extracting dominant features, allowing the model to be successfully trained. The process of training a CNN algorithm is similar to that of a feedforward neural network until a loss function is calculated, each thermal image is directed via the layers. Then the losses are backpropagated into the network, changing the weights using gradient descent methods, and the action is repeated until a predetermined number of training epochs are completed.

## 4 Discussion and Results

During this research, we have designed a CNN algorithm for thermal images to classify the participants into healthy or with spinal disorders. The entire dataset was split into subsets of the training, the test, and the validation sets, as is shown in fig.3:

<figure><img src="image_4.png" /><figcaption>Fig. 3. The schematic of dataset pre-processing</figcaption></figure>

Our dataset composed of 800 IR images was divided into two categories: to estimate a final model 40 test images were used to fit on the training dataset, and 760 training images to fit the model. This proportion was obtained empirically.
Before an AI-based classification algorithm was implemented, all database images were treated by our previously developed algorithm [4]. Using pre-processed infrared images, CNN was trained and tested on selected regions of interest (ROI) of the human body. The main algorithm steps for thermal images classification using a CNN are shown in fig.4:

<figure><img src="image_5.png" /><figcaption>Fig. 4. The algorithm steps for thermal images classification using a CNN</figcaption></figure>

The CNN architecture was constructed with a 0:001 learning rate and a 0:75 dropout rate. Two Convolutional Layers with the values of the 5x5 convolutional kernel and 32 outputs were used in the proposed architecture, followed by two Max Pooling Layers that calculates the maximum value in each patch of each feature map. The ReLU was used to keep the computation needed to operate the neural network from growing exponentially. The output layer divided the database into categories: healthy and with the disease, and is a completely connected layer. Cross-validation ensures that each image in the database is evaluated in train and test sets. To most effectively use our deep learning model an extension to stochastic gradient descent Adam Optimizer was used. As our algorithm has only two label classes (healthy and with the disease), binary cross-entropy class was used to calculate loss between true labels and predicted labels. After training the CNN on the training set and evaluating it on the test set was making a single prediction on a randomly chosen thermal image.

As the database has a small number of images the methodology is composed of a set of 80 experiments based on thermal images, each time randomly mixed of the training set and test set to make a single prediction. Table 1 is shown the results of descriptive statistical analysis of developed methodology for the classification of infrared images database of patients with spinal disorders based on a CNN.

<table><caption>Table 1. The results after using CNN on our infrared image database</caption><thead><tr><th>Experim.</th><th>Accuracy</th><th>Sensibility</th><th>Specificity</th><th>PPV</th><th>NPV</th><th>Time (ms/step)</th></tr></thead><tbody><tr><td>1.</td><td>0.96</td><td>0.93</td><td>0.94</td><td>0.93</td><td>0.93</td><td>89</td></tr><tr><td>2.</td><td>1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>169</td></tr><tr><td>3.</td><td>0.94</td><td>0.93</td><td>0.94</td><td>0.95</td><td>0.96</td><td>173</td></tr><tr><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td></tr><tr><td>Mean</td><td>0.95</td><td>0.94</td><td>0.93</td><td>0.94</td><td>0.95</td><td>152</td></tr></tbody></table>

The obtained results showed a good outcome with 95% accuracy and 94% specificity, which is a significant impetus to continue work on the classification of infrared images database of patients with spinal disorders based on a CNN algorithm.
On another side, according to the International Statistical Classification of Diseases and Related Health Problems [21], the study group could be divided into disease classes, as follows: 1) patients with deforming dorsopathies (kyphosis, lordosis, scoliosis, etc.); 2) patients with spondylopathies (osteochondrosis, spondylolisthesis, etc.), and 3) patients with other dorsopathies (torticollis, leg length discrepancy, etc.). However, in that case, the sample set would be small, which in turn has led to the impossibility to train an AI algorithm. So, as a negative point, it can be mentioned a relatively small present dataset of thermal images of the spine and the absence of open-access database information.

Meanwhile, the reported results show the potential for implementing IR thermal imaging of patients with spinal disorders based on a CNN algorithm.

## 5 Conclusions

Medical applications based on AI algorithms using infrared image data can be successfully used for spinal disease classification. One limitation of the present study was a relatively small quantity of IR images and the absence of an open-access spine thermograms database with the label by types of the diseases. However, the first obtained results allow assuming that our methodology can easily be applied to a much larger set of data, conducting even more precise results.

In subsequent researches, we expect to increase the database of the spine thermograms to classify them not only by two classes (healthy and with the disease) but specify by the type of disease to which it belongs, as well as to use different CNN architectures to compare results.

**Acknowledgments.** The authors would like to express their deepest gratitude to all volunteers and medical staff from the Center of Medical Rehabilitation & Sports Medicine, Vinnytsia, Ukraine, for participating in the study. Special thanks to Professor Petro F. Kolisnyk and Professor Sergei P. Kolisnyk from National Pirogov Memorial Medical University for assistance, and Professor Dr. Sergei V. Pavlov from Vinnytsia National Technical University for help in research work. This work was financed by Fundação para a Ciência e Tecnologia (FCT, Portugal) and NMT, S.A. as the Ph.D. grant (PD/BDE/142791/2018) of the Doctoral NOVA 14H Program.

## References

1. E. F. J. Ring, "History of thermology and thermography: Pioneers and progress," *Thermal. Int.*, vol. 22, no. 3, pp. 3-7, 2012.

2. G. Kwok *et al.*, "Postural Screening for Adolescent Idiopathic Scoliosis with Infrared Thermography," *Sci. Rep.*, vol. 7, no. 1, pp. 1-8, 2017, doi:10.1038/s41598-017-14556-w.

3. A. Lubkowska and E. Gajewska, "Temperature distribution of selected body surfaces in scoliosis based on static infrared thermography," *Int. J. Environ. Res. Public Health*, vol. 17, no. 23, pp. 1-16, 2020, doi: 10.3390/ijerph17238913.

4. A. Poplavska *et al.*, "Algorithm for Automated Segmentation and Feature Extraction of
Thermal Images,” in *Technological Innovation for Life Improvement*, 2020, pp. 378–386.

5. V. Vassilenko et al., “Automated features analysis of patients with spinal diseases using medical thermal images,” in *Optical Fibers and Their Applications 2020*, 2020, vol. 11456, pp. 116–124, doi: 10.1117/12.2569780.

6. H. E. John, V. Niumsawatt, W. M. Rozen, and I. S. Whitaker, “Clinical applications of dynamic infrared thermography in plastic surgery: A systematic review,” *Gland Surg.*, vol. 5, no. 2, pp. 122–132, 2016, doi: 10.3978/j.issn.2227-684X.2015.11.07.

7. J. Wen et al., “Overview of classification of Alzheimer’s disease” *Med Image Anal.*, 63, 2020.

8. U. Thirunavukkarasu et al., “Human Tongue Thermography Could Be a Prognostic Tool for Prescreening the Type II Diabetes Mellitus,” *Evidence-based Complement. Altern. Med.*, vol. 2020, 2020, doi: 10.1155/2020/3186208.

9. S. V. Francis, M. Sasikala, and S. Saranya, “Detection of breast abnormality from thermograms using curvelet transform based feature extraction,” *J. Med. Syst.*, vol. 38, no. 4, 2014, doi: 10.1007/s10916-014-0023-3.

10. M. Frize and A. Ogungbemile, “Estimating rheumatoid arthritis activity with infrared image analysis,” *Stud. Health Technol. Inform.*, vol. 180, pp. 594–598, 2012, doi: 10.3233/978-1-61499-101-4-594.

11. R. N. Kvyetnyy et al., “Group decision support system based on Bayesian network,” *Prz. Elektrotechniczny*, vol. 96, no. 9, pp. 123–128, 2020, doi: 10.15199/48.2020.09.26.

12. R. Roslidar et al., “A Review on Recent Progress in Thermal Imaging and Deep Learning Approaches for Breast Cancer Detection,” *IEEE Access*, vol. 8, pp. 116176–116194, 2020, doi: 10.1109/ACCESS.2020.3004056.

13. Vardasca, Magalhaes, and Mendes, “Biomedical Applications of Infrared Thermal Imaging: Current State of Machine Learning Classification,” *Proceedings*, vol. 27, no. 1, p. 46, 2019, doi: 10.3390/proceedings2019027046.

14. M. De Freitas Oliveira Baffa and L. Grassano Lattari, “Convolutional Neural Networks for Static and Dynamic Breast Infrared Imaging Classification,” *Proc.-31st Conf.Graph. Patt. Images, SIBGRAPI 2018*, p. 174–181, 2019, doi: 10.1109/SIBGRAPI.2018.00029.

15. C. Magalhaes, J. Mendes, and R. Vardasca, “Meta-analysis and systematic review of the application of machine learning classifiers in biomedical applications of infrared thermography,” *Appl. Sci.*, vol. 11, no. 2, pp. 1–18, 2021, doi: 10.3390/app11020842.

16. R. Koprowski, “Automatic analysis of the trunk thermal images from healthy subjects and patients with faulty posture,” *Comput. Biol. Med.*, vol. 62, pp. 110–118, 2015, doi: 10.1016/j.compbiomed.2015.04.017.

17. V. M. Ravindra et al., “Degenerative Lumbar Spine Disease: Estimating Global Incidence and Worldwide Volume,” *Glob. Spine J.*, vol. 8, no. 8, pp. 784–794, 2018, doi: 10.1177/2192568218770769.

18. J. Christiansen and W. Dudley, “International Academy of Clinical Thermology Quality Assurance Guidelines Standards and Protocols in Clinical Thermographic Imaging,” no. January, pp. 1–35, 2015, doi: 10.13140/RG.2.2.28341.78562.

19. S. T. Kakileti, A. Dalmia, and G. Manjunath, “Exploring deep learning networks for tumor segmentation in infrared images,” *Quant. Infrared Thermogr. J.*, vol. 17, no. 3, pp. 153–168, 2020, doi: 10.1080/17686733.2019.1619355.

20. F. J. Fern et al., “Detection of Breast Cancer Using Infrared Thermography and Deep Neural Networks,” *Lect. Notes Comput. Sci.*, vol. 11466 LNBI, pp. 514–523, 2019, doi:10.1007/978-3-030-17935-9_46.

21. B. Khoury, C. Kogan, and S. Daouk, “International Classification of Diseases 11th Edition (ICD-11),” in *Encyclopedia of Personality and Individual Differences*, V. Zeigler-Hill and T. K. Shackelford, Eds. Cham: Springer International Publishing, 2017, pp. 1–6.