# Algorithm for Automated Segmentation and Feature Extraction of Thermal Images

Anna A. Poplavska, Valentina B. Vassilenko, Oleksandr A. Poplavskyi, Sergei

V. Pavlov, Fernando Pimentel-Santos

► To cite this version:

Anna A. Poplavska, Valentina B. Vassilenko, Oleksandr A. Poplavskyi, Sergei V. Pavlov, Fernando Pimentel-Santos. Algorithm for Automated Segmentation and Feature Extraction of Thermal Images. 11th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Jul 2020, Costa de Caparica, Portugal. pp.378-386, 10.1007/978-3-030-45124-0_36. hal-03741551

HAL Id: hal-03741551

https://inria.hal.science/hal-03741551v1

Submitted on 1 Aug 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Algorithm for Automated Segmentation and Feature
Extraction of Thermal Images

Anna A. Poplavska¹,², Valentina B. Vassilenko¹,², Oleksandr A. Poplavskyi³,
Sergei V. Pavlov⁴, Fernando M. Pimentel-Santos⁵

¹ Laboratory of Instrumentation, Biomedical Engineering and Radiation Physics (LIBPHYS), NOVA School of Science and Technology - NOVA University Lisbon

² NMT, S.A., Parque Tecnológico de Cantanhede, Núcleo 04, Lote 3, 3060 - 197 Cantanhede, Portugal

³ Kiev National University of Construction and Architecture, Ukraine

⁴ Vinnytsia National Technical University, Ukraine

⁵ NOVA Medical School|Faculdade de Ciências Médicas (NMS|FCM), Chronic Diseases Research Center (CEDOC), Portugal

**Abstract.** Medical infrared thermal imaging techniques can provide the high-quality images for monitoring and pre-clinical diagnostic of the diseases by showing the thermal abnormalities available in the body. Its biggest advantage is non-contact, non-invasive and very fast way of use. However, the incorrect interpretation of the thermal images via simple observation or manual analysis and detection approximate regions of interest lead to the numerous false positive results. The main objective of this research work is to develop an algorithm for automated image processing and analysis of the extracted features on thermal images for screening or pre-diagnosis of the diseases. This work presents the results of our previously developed Copyright algorithm applied to thermal images obtained from the patients with Axial Spondyloarthritis. This processing and analysis of the extracted features from thermal images can offer a novel quick and non-invasive tool for diagnosis and monitoring of rheumatoid diseases.

**Keywords:** medical thermal images, image processing, automated segmentation, rheumatic diseases

# 1 Introduction

Medical infrared thermal imaging (MITI) is currently a rapid developing technique that can provide the images for monitoring and pre-clinical diagnostic of thermal abnormalities present in the body in real time. MITI technique is a non-contact, noninvasive, imaging procedure which offers the two-dimensional temperature spatial distribution measurement.

With modern technology, a single image may contain several thousands of temperature points recorded in a fraction of a second. It has been used to study number of diseases where the increased skin temperature can signal the presence of inflammation, or where blood flow is increased or decreased due to a clinical
abnormality. MITI were widely tested in the following areas of medical diagnostics, as: thermoregulation study [1], muscular pain [2], breast cancer [3], vascular disorder [4], rheumatoid diseases [5], osteoarthritis [6,7], Raynaud's phenomenon [8] etc. Various thermography studies have been performed during the last decade for the assessment of thermal symmetry in hand regions [9]. Recently the thermographic technology was successfully applied in animal models as a tool for the quick evaluation of inflammation after invasive intervention in reconstructive plastic surgery [10].

In medical IRT, the abnormalities can be diagnosed by visual and subjective analysis, but the human based diagnoses are more likely to errors due to visual exhaustion, negligence, mental workload, etc. The segmentation by Regions of Interesse (ROI) is often an initial step in thermographic medical analysis, where the automatic approaches could lead to fast and highly reproducible outcomes [11]. Due to different geometrical shapes and proportions of body parts, it's not possible to capture the ROIs properly without background while data acquisition. Based on the procedure used to define these parameters, the segmentation process can be categorized as: a) manual, where software is used to manually mark the ROI with pre-defined or user-defined shapes; b) semiautomated, when the human intervention is required to reduce the search space for automatic segmentation or to inspect visually and modify the automatically obtained ROI; c) fully-automated, where the different approaches, like thresholding, clustering, geometric and thermal features are used for automatic determination of parameters which define the ROI.

Thermography shows the exact location of thermal abnormalities related to physiological and metabolic processes; therefore, it has a great potential in medical use. However, MITI technology still has a limited use in diagnostic due to a lot of false negative and false positive rates. The problem is not connected to the equipment because the infrared (IR) cameras with a high thermal sensitivity, above the 0.06°C, provide the high-quality images. One of the main problems is the wrong interpretation of the obtained results due to the lack of automated feature extraction on thermal images for specific diseases. Therefore, the main objective of this research work was to develop an algorithm for automated segmentation and extracted features on thermal images in order to support medical decision-making and help to specify feature that can be significant signs for disease classification.

## 2 Contribution to Life Improvement

The information extracted from the IR images after its proper analysis is the basis for identifying, monitoring or pre-diagnostic of different types of diseases. To automate this process, modern devices have enough capacity to provide superficial analysis using the special analytical tools. However, the huge amount of diseases and various individual anatomical thermal features of human body cause difficulties to implement one or few well-known algorithms for all cases.

The important contribution of the present work is development of a novel algorithm for the IR image segmentation and feature extraction for each cohort of patient with specific disease, related with the inflammation processes in the human body. Ankylosing spondylitis (AS) is one of subtypes of axial spondyloarthritis that primarily affects the spine, although other joints can become involved. The most common early symptoms
of AS are frequent pain and stiffness in the lower back. Early diagnosis is critical for establishing a proper therapy that halts the natural progression of the disease and clinical evaluation requires complementary radiological information through a combination of techniques (radiography, computed tomography and MRI) [12].

The algorithm suggested in this paper combined with IR thermal camera can offer a novel quick and non-invasive tool for diagnosing and monitoring some kinds of rheumatoid diseases. Another important task of this work is to improve protocol of thermal measurements and increase a database of thermal images of patients with specific diseases that may contribute to the use of medical thermography in hospitals in the future.

## 3 Materials and Methods

Present study was carried out in partnership with the Center for the Study of Chronic Diseases of the Faculty of Medical Sciences of the Nova University of Lisbon (CEDOC/FCM-UNL) in the research project: "MyoSpA: o papel do músculo nas espondiloartropatias, um novo paradigma" [13].

The participant's cohort of 48 volunteers, 16 females and 32 males, between 18 to 70 years old, an average height 1.72±0.1 (m) and weight 74.6±0.1 (kg). The participants were divided into two groups: I group of 22 individuals with AS disease and II control group with 26 participants. The pain syndrome had more than 90% of participants with AS and 9.1% were without pain, thus signalizing that this disease has a big influence of daily lifestyle. On the other side, in the control group, without AS diagnosis, the 65.4% of participants had pain syndrome, which signalized that a lot of people are suffering from the back pain even without associated diagnosis. Almost all participants who had pain syndrome with AS disease indicated the pain in the lumbar region or along of the all spine.

The measurements were performed in the ambient temperature using a thermal camera FLIR® E6, with a thermal sensitivity of <0.06°C. An IR resolution of 160x120 pixels was interpolated to 320x240 resolution within the camera electronics. The camera was switched on 15 minutes before acquisition. The emissivity parameter on the camera was set as that for the skin, ε = 0.98. The camera lens was placed in parallel to the frontal plane of the subject's body at the distance of 1.5 m. The measurements have been carried out in the laboratory room with no access of solar radiation and/or any heaters, the room temperature was about 21°C and humidity was maintained at approximately 50%. The black matte curtain was used as background to prevent any infrared reflections and to make the background uniform.

All volunteers were completed a questionnaire regarding the general information and agreement to participate in the project. The volunteers were asked not to smoke, abstain from alcohol or coffee, not to perform physical activity, use skin creams or spray on the research area 6 hours before the examination. Before measurement, each participant had to stay undressed for 10 minutes for the purpose of accommodation to the ambient temperature. During the measurements each participant was in the same standing position, facing the wall, the feet were together, the head was looking straight ahead and the arms were down to the sides.
## 4 Methodology

The medical images edge detection (MIED) is an important step toward understanding image features. With the increasing numbers of medical images techniques, the use of computers for processing and analysis of images has become critical necessary. The MIED is extensively used in image segmentation when images are divided into areas of interest. This information has greatly increased the knowledge of normal and pathological anatomy and are the critical components in diagnostic and treatment planning [11-14,16].

Ankylosing spondylitis (AS) primarily affects the spine and the sacroiliac joints that connect the lower spine to the pelvis, resulting in pain in the lower back, hips, and buttocks. So, our work was concentrated on the posterior side of the back, the target region was specified in the thermal abnormalities in lumbar region, as one of the possible feature of AS disease. The principal steps that were taking into consideration during this study are shown in the fig.1:

<figure><img src="image_3.png" /><figcaption>Fig. 1. Algorithm for automated segmentation and feature extraction of thermal image</figcaption></figure>

Thermal images were obtained in pseudo colors and, as the first step, the infrared images were converted to a grayscale, using the formula:

$$ \text{Gray} = 0.2126\text{R} + 0.7152\text{G} + 0.0722\text{B} \quad (1) $$

The linear Gauss filter was applied to blur images and remove the noise:

$$ g(x, y) = \frac{1}{2\pi\sigma^2} e^{-\frac{(x^2+y^2)}{2\sigma^2}} \quad (2) $$

where x is the distance from the origin in the horizontal axis, y is the distance from the origin in the vertical axis, and σ is the standard deviation of the Gaussian distribution.

In our work the Sobel's operator [14] (fig.4, a) was used to detect edge of human body in each thermal image. The I_m defined as the source image, and G_x and G_y were two images which at each point contain the vertical and horizontal derivative approximations respectively, the computations were as follows:
$$G_x = \begin{bmatrix} -1 & 0 & 1 \\ -2 & 0 & 2 \\ -1 & 0 & 1 \end{bmatrix} * I_m \quad \text{and} \quad G_y = \begin{bmatrix} 1 & 2 & 1 \\ 0 & 0 & 0 \\ -1 & -2 & -1 \end{bmatrix} * I_m \quad (3)$$

As example, in the Fig 2 is shown a comparison between results from applying of different gradient operators into thermal image of patient with AS.

<figure><img src="image_4.png" /><figcaption>Fig.2. Example of gradient operators: a) Prewitt; b) Canny; c) Sobel</figcaption></figure>

In the healthy persons the temperature distribution on the back is usually symmetrical relatively to the vertebral column (VC). Asymmetry on the region of temperature distribution relative to the VC is the main criteria on diagnosis of an inflammatory process or chronic disease. From the areas with an abnormally high or low temperature, the symptoms of many diseases can be recognized in the early stages of their occurrence.

Thus, after detecting the edges of the human body, the axes of vertical symmetry X and horizontal Y were found (fig.4, b) using the formulas:

$$X_s = \frac{\sum_{i=1}^{n-1} \frac{|L_i + R_i|}{2}}{n} \quad Y_s = \frac{\sum_{j=1}^{m-1} \frac{|Lw_j + A_j|}{2}}{m} \quad (4)$$

where n and m are the numbers of pixels in Lᵢ columns and rows, respectively; Rᵢ are the coordinates of left and right edge pixels of each row, Lwⱼ, Aⱼ – coordinates of lower and upper back edge pixels of each column.

Segmentation of medical images is an important challenging task in detecting the temperature abnormalities present in the body. Despite the huge effort in solving it, there is no single approach that can generally solve the problem of segmentation for the huge among of image varieties. In this study the K-means clustering algorithm was used for segmentation of infrared thermal images. This algorithm is based on minimization of the objective function using the formula:

$$J(V) = \sum_{i=1}^{c} \sum_{j=1}^{c_i} (\|x_i - v_j\|)^2 \quad (5)$$

where $\|x_i - v_j\|$ is the Euclidean distance between xᵢ and vⱼ, cᵢ – the number of data points in iᵗʰ cluster and c – the number of cluster centers. The algorithm consists in choosing the number of cluster K, and then implying the K-means clustering to the image. In the fig.3 is shown an example of results obtained after applying the proposed k-mean clustering algorithm onto thermal images of the back of healthy person and one with AS disease. It could be clearly noticed from the cluster 5 that in healthy patient were not detected inflammatory zone, in opposite, as in patient that had AS disease. After ROI detection the targeted region was selected from vertical Ys axis of symmetry
starting on the cervical part and ending on the sacral spine. This region was divided into two parts: left and right lower back. The areas of regions and horizontal Xs coordinates were calculated by using the formula (5).

<figure><img src="image_5.png" /><figcaption>Fig.3. Results of k-mean clustering for IR images: a) normal case; b) patient with AS</figcaption></figure>

In the fig.4. is shown results of automated segmentation and feature extraction of thermal image on one patient with AS disease.

<figure><img src="image_6.png" /><figcaption>Fig.4. Image processing: a) edge detection, b) Xs and Ys axis of symmetry, c) ROI-segmentation</figcaption></figure>

## 4 Discussion Results

The deep and objective diagnostic information, like severity and type of a disease can only be obtained by statistical analysis of the ROI. The segmentation of ROI is often an initial step in thermographic analysis, where the automatic approaches lead to fast and highly reproducible outcomes [14,15]. Due to different geometrical shapes and proportions of body parts, it is not possible to capture the ROIs properly without the background during the process of data acquisition. The human back thermal picture varies individually, and it is difficult to state the exact features for some disease. This requires a large amount of measurements of significant cohort of patients with one specific disease. As thermal asymmetry distribution is not good enough for manual analysis, which can be appropriate only when a significant strong difference in color
on thermal images are observed for the feature extraction and segmentation, the ROI needs to be solved automatically. The most significant benefit of the developed algorithm is automated segmentation of ROI with analyzing the side of asymmetry based on the central axis of the spine. The proposed algorithm for automatic segmentation of body regions is consistent with the solutions provided by other research groups [14,15]. During its developing the main question were considered:

i) entire region segmentation, where the edge detection and thresholding techniques are commonly used to extract the body region from background [16];

ii) anatomical region segmentation, geometric and local thermal features to locate the specific anatomical regions [17];

iii) diseased area segmentation, in order to locate the hottest or coldest regions [18].

The developed experimental protocol for IR thermal measurements turns to be appropriate for measurements in the hospitals, as well as it was well accepted by the participants. Analyzing features from the IR images of the patients with AS diseases we observe that for almost all patients with AS the significant feature was an abnormal temperature distribution among lumbar region or/and along all spine. The skin temperature distribution of those healthy as well as individuals with associate diagnosis represents the different pattern. Another important result observed in this study: the thermal images of the participants with the back pain complains clearly demonstrated the difference of temperature exactly in the places indicated by the participants. This allows to assume that thermography clearly shows the area of pain syndrome or thermal abnormalities. An additional information which can be obtained from thermal measurements: it was noticed that a lot of women has an area of higher temperature as a result of a bra strap that is too tight, that can cause poor posture, back and neck pain, shoulder grooves leading to numbness in the fingers and restrict circulation to muscle. It should be mentioned that the local temperature abnormalities and the asymmetry of the IR thermal image depends on the disease stage and severity, as well as the period of the disease treatment process.

# 5 Conclusions

The paper presents an algorithm for IR image segmentation as well as its features extraction. Its application to the cohort of patient with specific disease, ankylosing spondylitis (AS), shows that it can be used for automatic thermal image-processing, segmentation and feature. Proposed solution solves problem of ROI's detection by automatic segmentation and the location of thermal abnormalities in the patients' back. The experimental protocol used in present for the IR thermal camera measurements shows a good correlation with a medical observation. However, the methodology needs to be validated in the further research study by increasing the database of IR images and statistical analysis. The developed algorithm and measurements protocol with combination of IR thermography can offer a novel quick and non-invasive tool for diagnosis and monitoring of rheumatoid diseases in real time for saving lives.

**Acknowledgment.** The authors would like to thank the Fundação para a Ciência e Tecnologia (FCT, Portugal) and NMT, S.A. for co-financing the PhD grant (PD/BDE/142791/2018) of the Doctoral NOVA 14H Program. Special thanks to Diogo
Casal from NOVA Medical School – NOVA University of Lisbon for assistance in some
tasks that need to be solved from the medical point of view.

References

1. Bouzida, N., Bendada, A., Maldague, X.P.: Visualization of body thermoregulation by infrared imaging. Journal of Thermal Biology. 34, 120–126, 2009.

2. Park, J., Hyun, J.K., Seo, J.: The effectiveness of digital infrared thermographic imaging in patients with shoulder impingement syndrome. Journal of Shoulder and Elbow Surgery.16, 548–554, 2007.

3. Cherkas, L.F., Carter, L., Spector, et al.: Use of thermographic criteria to identify Raynaud's phenomenon in a populationsetting. Journal of Rheumatology. 30, 720–722, 2003.

4. Holey, L.A., Dixon, J., Selfe, J.: An exploratory thermographic investigation of the effects of connective tissue massage on autonomic function. Journal of Manipulative and Physiological Therapeutics. 34, 457–462, 2011.

5. Denoble, A.E, Hall, N., Pieper, C.F., et al.: Patellar skin surface temperature by thermography reflects knee osteoarthritis severity Clin. Med. Insights: Arthritis and Musculoskeletal Disord. 3, 69–75, 2010.

6. Ammer, K.: Diagnosis of Raynaud's phenomenon by thermography. Skin Res. Tech. 2, 182-185, 2006.

7. Vardasca, R.: Hand thermogram standardization with barycentric warp model. Proceedings of the 4th research student workshop, University of Glamorgan, South Wales. 73–75, 2009.

8. Vardasca, R., Ring, E., Plassmann, P., et al.: Thermal symmetry of the upper and lower extremities in healthy subjects. Thermol Int. 22(2), 53–60, 2012.

9. Casal, D., Mota-Silva, E., Pais, D., Iria, I., Videira, P.A., Tanganho, D., Custódio Alves, S., Mascarenhas-Lemos, L., Martins-Ferreira J., Ferraz-Oliveira, M., Vassilenko, V., Goyri O'Neill, J.: Optimization of an Arterialized Venous Fasciocutaneous Flap in the Abdomen of the Rat. Plastic and Reconstructive Surgery. Global open. 2017.

10. Casal, D., Mota-Silva, E., Iria, I., et al.: Reconstruction of a 10-mm-long median nerve gap in an ischemic environment using autologous conduits with different patterns of blood supply: A comparative study in the rat. PLoS ONE. 13(4):e0195692, 2018.

11. Singh J., Arora A.S.: A framework for enhancing the thermographic evaluation on characteristic areas for paranasal sinusitis detection. Infrared Phys Technol 85:457-464, 2017.

12. Polidori, G., Kinne, M., Mereu, T., Beaumont, F., Kinne, M.: Medical Infrared Thermography in back pain osteopathic management. Complementary Therapies in Medicine. 39, 19–23, 2018.

13. Ferreira-Dos-Santos, G., Domingues, L., et al.: Tools for the Screening of Sarcopenia: A Head-to-Head Comparison by the NOVA SarcoAging Group – Archives of Physical Medicine and Rehabilitation. 99, 110, 2018.

14. Poplavskyi, A.A., Vassilenko, V.B., Poplavska, A.A.: Automatic threshold definition for dynamic images. Certificate of registration of computer program. №50151, 2013.

15. Duarte A., Carrão L., Espanha M.,et al.: Segmentation algorithms for thermal images. Procedia Technol 16:1560–1569, 2014.

16. Fournet D., Redortier B., Havenith G.: A method for whole-body skin temperature mapping in humans. Thermol. Int. 22:157–159, 2016.

17. Nandagopan G.L., Bhargavi Haripriya A.: Implementation and comparison of two image segmentation techniques on thermal foot images and detection of ulceration using asymmetry. Int Conf Commun Signal Process ICCSP 2016: 356–360, 2016, 2016.

18. Mohiyuddin N., Warhade K.K.: Segmentation of thermal images for evaluation of rheumatoid arthritis disease. Int J Emerg Eng Res Technol 2:35–44, 2014.