dataesr/scientific-paragraphs-categorization huggingface dataset:
Deep learning image segmentation As the respective densities of native healthy bone, neo-formed bone and calcium and magnesium phosphate bone cements are comparable, their gray values in µCT are often superimposed. Image segmentation by thresholding to obtain reliable quantitative information is consequently di cult and time-consuming. Therefore, µCT data were processed by neural networks with the segmentation wizard of Dragon y software (ORS, Canada)  (7, 28). The work ow can be described as follows: 1. Prearchitectured neural networks were generated with a depth level of 5 and a patch size of 32. In particular, Dragon y U-Net 3D was used to improve the robustness of the results. 2. Manual segmentation of a representative region of interest (ROI) was performed as initial training data. 3.
Statistical analysis All tests were performed using GraphPad Prism version 8. 0. 0 (GraphPad Software, San Diego, California, USA). Nonparametric tests were selected when a Gaussian distribution of values could not be found by a normality test or when the sample size did not satisfy parametric test requirements. The H0 hypothesis was rejected when p-value < 0. 05. Outlier values determined by the ROUT method  (29)  were removed from the analysis.
Early detection of physical frailty and infectious diseases in seniors is important to avoid any fatal drawback and promptly provide them with the necessary healthcare. One of the major symptoms of viral infections is elevated body temperature. In this work, preparation and implementation of multi-age thermal faces dataset is done to train different "You Only Look Once" (YOLO) object detection models (YOLOv5,6 and 7) for eye detection. Eye detection allows scanning for the most accurate temperature in the face, which is the inner canthus temperature. An approach using an elderly thermal dataset is performed in order to produce an eye detection model specifically for elderly people. An application of transfer learning is applied from a multi-age YOLOv7 model to an elderly YOLOv7 model. The comparison of speed, accuracy, and size between the trained models shows that the YOLOv7 model performed the best (Mean average precision at Intersection over Union of 0.
Object detection deep-learning methods have better performance in the case of large-scale data training. From these methods, we have R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN and YOLO  [10]. YOLO is a real-time object monitoring algorithm, due to its small size and fast operational speed in comparison to other object detection alternatives. In addition, YOLO can detect objects in videos, making it a better choice in real-time detection scenarios  [11]. Several techniques that aim for eye detection in thermal images were proposed in the literature. For example, Hussein et al. worked on a training cascade-based classifier, feeding it by Haar, histogram of oriented gradient (HoG), and local binary patterns (LBP) features from 1000 images taken from the Natural Visible and Infrared Facial Expression Database (NVIE).
The different architectures of YOLO versions 5,6 and 7 are trained and compared in this work. At first, multi-age images collected from online image datasets and captured thermal images are used for the training and testing processes. Then, transfer learning is carried out, starting from the multi-age model weights, to train a new model on the limited number of elderly faces in thermal images. This is followed by an inner canthus temperature extraction method. The rest of the paper is organized as follows. Section 2 introduces the materials and describes the methodology. Section 3 shows the results, followed by a discussion in Section 4. A conclusion with work perspectives is finally presented in Section 5.
Materials and Methods The general methodology of this work consists of several steps. The first step is the data collection and preparation procedure, followed by labelling the eyes in the dataset. After data annotation, images are introduced to data augmentation techniques to increase their number and avoid the model's overfitting. Then, training and testing of YOLO different models is carried out. Finally, the detected eye area is scanned for the inner canthus temperature. These steps are applied once on a multi-age dataset and another time on an elderly dataset. Figure  1  shows a flowchart of the methodology proposed in our work wherein the final step will be performed later, retrospectively.
The Multi-Ages Dataset The multi-age dataset consists of 1827 thermal images, a combination of thermal images captured in our lab (104 images of 11 individuals) and the TFW testing indoor dataset (1723 images of 23 individuals). The merging of two different datasets is performed in order to obtain larger and more varied samples of face thermal images. This method will ensure a more generalized model  [19]. The multi-age dataset consists of 1827 thermal images, a combination of thermal images captured in our lab (104 images of 11 individuals) and the TFW testing indoor dataset (1723 images of 23 individuals). The merging of two different datasets is performed in order to obtain larger and more varied samples of face thermal images. This method will ensure a more generalized model  [19]. Data collection in the lab is carried out using a TROTEC IC060 thermal camera with a resolution of 152 × 115.
The equipment's sensor is an uncooled microbolometer focal plane array with a minimum focus distance = 0. 1m and a spatial resolution = 2. 2 mrad. The captured images in SAT file format are transferred to the PC using a USB data cable. TROTEC provides ICReport software which allows exportation of the images and their temperature data in JPG and xlsx Excel spreadsheet formats, respectively. A total of 104 images of 11 individuals are captured. Five males and six females aged between 22 and 30 contributed to data collection. The images are captured in our lab, with the camera being parallel to the participants and placed at a distance of 1m  [20]. The ambient temperature is maintained at 20 • C (±1 • C) and measured using a thermometer. All our participants were asked to spend 15 min in the room in order to prevent the thermal effect of the external environment, and the same setup conditions were preserved for all captures.
The experiment was explained to the volunteers, who agreed to let their images be utilized for research and publication purposes. The second dataset, named the "annotated thermal faces in the wild dataset (TFW)", is a visual and thermal dataset containing images of 147 subjects of different age groups. It counts 9982 images collected in controlled and uncontrolled and indoor and outdoor environments. The indoor environment's temperature is maintained at 25 °C. The images are taken using FLIR T540 thermal camera with a resolution of 464 × 348 pixels. Note that the TFW dataset is originally split to training, testing, and validation in order to train a YOLOv5 model to detect individuals' faces and their landmarks (pupils, nose, and borders of mouth). Therefore, we have chosen its indoor unlabeled testing dataset, which contain 2160 images of 30 individuals.
However, 7 individuals are wearing eyeglasses; accordingly their images are eliminated  [21]. This leaves the dataset with 1723 images of 23 individuals with different head rotations (upward, downward, left and right). The second dataset, named the "annotated thermal faces in the wild dataset (TFW)", is a visual and thermal dataset containing images of 147 subjects of different age groups. It counts 9982 images collected in controlled and uncontrolled and indoor and outdoor environments. The indoor environment's temperature is maintained at 25 • C. The images are taken using FLIR T540 thermal camera with a resolution of 464 × 348 pixels. Note that the TFW dataset is originally split to training, testing, and validation in order to train a YOLOv5 model to detect individuals' faces and their landmarks (pupils, nose, and borders of mouth).
The Elderly Dataset (TFW, Tufts, IRDatabase) The preparation of the elderly dataset is done by collecting images from three different online thermal datasets: TFW  [21] , Tufts  [22, 23]  and IRDatabase  [24]. TFW database is collected in the Institute of Smart Systems and Artificial Intelligence, Nazarbayev University, Kazakhstan. The second database, IRdatabase is collected in RWTH Aachen University in Germany. The third database, Tufts, is built in Tufts University, Boston, USA. The elderly participants in the three previous datasets are staff, faculty members, and/or their family members, alongside students' family members. This results in a total of 656 images for 20 individuals aged 60 and above. The distribution of the sources of the database is presented in Table  1. We used Roboflow online tool to label our data  [25].
The labelling consists of two classes: one eye and two eyes. The class one eye includes each eye in two separate bounding boxes; however, the class two eyes includes both eyes in the same bounding box. Figure  3  shows an example of a labeled image where the pink boxes refer to the one eye class and the yellow box refers to the two eyes class. 2. 1. 2. The Elderly Dataset (TFW, Tufts, IRDatabase) The preparation of the elderly dataset is done by collecting images from three different online thermal datasets: TFW  [21] , Tufts  [22, 23]  and IRDatabase  [24]. TFW database is collected in the Institute of Smart Systems and Artificial Intelligence, Nazarbayev University, Kazakhstan. The second database, IRdatabase is collected in RWTH Aachen University in Germany. The third database, Tufts, is built in Tufts University, Boston, USA.
The elderly participants in the three previous datasets are staff, faculty members, and/or their family members, alongside students' family members. This results in a total of 656 images for 20 individuals aged 60 and above. The distribution of the sources of the database is presented in Table  1. We used Roboflow online tool to label our data  [25]. The labelling consists of two classes: one eye and two eyes. The class one eye includes each eye in two separate bounding boxes; however, the class two eyes includes both eyes in the same bounding box. Figure  3  shows an example of a labeled image where the pink boxes refer to the one eye class and the yellow box refers to the two eyes class. For the elderly dataset, the labeling is limited to the two eyes class, which is simply called eyes. This is due to using the two eyes box exclusively later in the methodology.
Data Augmentation After labeling the 1827 multi-age dataset images and the 656 elderly dataset ones, we integrate different data augmentation techniques using the Roboflow data augmentation tool in order to increase the size of our dataset  [25, 26]. The techniques applied are:  For the elderly dataset, the labeling is limited to the two eyes class, which is simply called eyes. This is due to using the two eyes box exclusively later in the methodology.
Data Augmentation After labeling the 1827 multi-age dataset images and the 656 elderly dataset ones, we integrate different data augmentation techniques using the Roboflow data augmentation tool in order to increase the size of our dataset  [25, 26]. The techniques applied are: • Horizontal flipping; • Rotations; • Cropping; • Saturation; • Exposure; and • Blur. Consequently, we had a total of 4255 images for the multi-age dataset and 1,791 images for the elderly dataset. Some of the augmented photos are presented in Figure  4. The rotations displayed in Figure  4  are done in order to allow the model to learn the different possible face positions of the subjects. This way, it can detect the subject's eyes whether he or she is lying down or whether the thermal camera is attached to the ceiling.
 Blur. Consequently, we had a total of 4255 images for the multi-age dataset and 1,791 images for the elderly dataset. Some of the augmented photos are presented in Figure  4. The rotations displayed in Figure  4  are done in order to allow the model to learn the different possible face positions of the subjects. This way, it can detect the subject's eyes whether he or she is lying down or whether the thermal camera is attached to the ceiling.
YOLO: Algorithm and Versions "You only look once" (YOLO) is a unified single convolutional neural network (CNN) model proposed by Redmon et al. for object detection in images  [27]. In this work, YOLO's open-source CNN based software is chosen due to its high detection accuracy and satisfactory computational complexity. The YOLO model is trained on the COCO dataset to detect multiple bounding boxes with their class probabilities  [27]. It divides the image into an S × S grid, wherein each grid cell predicts  bounding boxes along with their confidence scores. Equation  (1)  shows how the confidence score for each bounding box is calculated. Confidence Score Object = Pr Object * IoU prediction truth , ( ) 1 where Pr Object shows the probability that the cell contains an object and  is the intersection of the union between the detected box and the ground truth (Figure  5 ).
If the cell does not contain an object, then the Pr(Object) should be a zero leading to zero confidence score. On the contrary, the aim is to have the IoU prediction truth equal to the Confidence Score Object. The YOLO model is trained on the COCO dataset to detect multiple bounding boxes with their class probabilities  [27]. It divides the image into an S × S grid, wherein each grid cell predicts B bounding boxes along with their confidence scores. Equation (  1 ) shows how the confidence score for each bounding box is calculated. Confidence Score Object = Pr(Object) * IoU truth prediction , where Pr(Object) shows the probability that the cell contains an object and IoU truth prediction is the intersection of the union between the detected box and the ground truth (Figure  5 ). If the cell does not contain an object, then the Pr(Object) should be a zero leading to zero confidence score.
Moreover, YOLOv6 is of two sizes, nano or small (to date); larger sizes are still in development. The choice of the model size is a tradeoff between accuracy and computational power. Figure  7  shows how larger models have greater mean average precision (mAP) scores when trained and tested on the COCO dataset  [29]. Besides the confidence score above, each grid cell containing an object predicts C conditional class probabilities, Pr Class i Object. This metric shows the probability that the detected object belongs to class i. During testing of the model, a class-specific confidence score for each bounding box is calculated, as shown in Equation (  2 Figure  6  shows the workflow of the YOLO model, in which each image contains S × S × B bounding boxes. Each box has the following predictions: the center coordinates (x,y), weight, height, 1, confidence score, and C, conditional class probabilities  [27].
There are seven main YOLO versions: the first YOLO version, YOLOv2, YOLOv3, YOLOv4, YOLOv5, YOLOv6 and YOLOv7. Both YOLOv5 and YOLOv6 have models of different sizes, whereas YOLOv5 comes with five different model sizes from Nano to Xlarge (YOLOv5n, YOLOv5s,i YOLOv5m, YOLOv5l, and YOLOv5xl), as shown in Figure  7. Moreover, YOLOv6 is of two sizes, nano or small (to date); larger sizes are still in development. The choice of the model size is a tradeoff between accuracy and computational power. Figure  7  shows how larger models have greater mean average precision (mAP) scores when trained and tested on the COCO dataset  [29]. In this study, we train the three most recent YOLO versions, 5, 6, and 7. The YOLOv5 model architecture shown in Figure  8  comprises three independent parts: the backbone, neck and head.
YOLOv6 offers a hardwarefriendly design and high performance dedicated to industrial applications. Changes are made in the backbone and neck of YOLOv5, in order to meet these criteria. There exist, as of now, the YOLOv6-nano and YOLOv6-small model sizes, while other sizes are still in development  [32]. YOLOv7, unlike YOLOv5, does not use the ImageNet pre-trained backbones. However, the models are trained on the COCO dataset entirely. There are two major changes in the YOLOv7 architecture: The first is the presence of a computational block in its backbone named E-ELAN (extended efficient layer aggregation network), which allows the framework to learn better. Secondly, the YOLOv7 introduces new BoF (bag of freebies) methods that enhance performance without increasing training costs  [33]. where the data loader of YOLOv5, for example, applies three types of data enhancement: color space adjustment, scaling and mosaic enhancement  [30].
YOLOv6 offers a hardwarefriendly design and high performance dedicated to industrial applications. Changes are made in the backbone and neck of YOLOv5, in order to meet these criteria. There exist, as of now, the YOLOv6-nano and YOLOv6-small model sizes, while other sizes are still in development  [32]. YOLOv7, unlike YOLOv5, does not use the ImageNet pre-trained backbones. However, the models are trained on the COCO dataset entirely. There are two major changes in the YOLOv7 architecture: The first is the presence of a computational block in its backbone named E-ELAN (extended efficient layer aggregation network), which allows the framework to learn better. Secondly, the YOLOv7 introduces new BoF (bag of freebies) methods that enhance performance without increasing training costs  [33].
Training Settings The training settings of the augmented multi-age dataset are detailed in Figure  9 , where seven different models were established. friendly design and high performance dedicated to industrial applications. Changes are made in the backbone and neck of YOLOv5, in order to meet these criteria. There exist, as of now, the YOLOv6-nano and YOLOv6-small model sizes, while other sizes are still in development  [32]. YOLOv7, unlike YOLOv5, does not use the ImageNet pre-trained backbones. However, the models are trained on the COCO dataset entirely. There are two major changes in the YOLOv7 architecture: The first is the presence of a computational block in its backbone named E-ELAN (extended efficient layer aggregation network), which allows the framework to learn better. Secondly, the YOLOv7 introduces new BoF (bag of freebies) methods that enhance performance without increasing training costs  [33].
Training Settings The training settings of the augmented multi-age dataset are detailed in Figure  9 , where seven different models were established. For the elderly dataset, we have used both the original and augmented versions separately to train different models of YOLOv7 and compare among them. The training consisted of two stages: the first started from the original YOLOv7 weights; however, the second started from the weights of YOLOv7 model trained on the augmented multi-age For the elderly dataset, we have used both the original and augmented versions separately to train different models of YOLOv7 and compare among them. The training consisted of two stages: the first started from the original YOLOv7 weights; however, the second started from the weights of YOLOv7 model trained on the augmented multi-age dataset (i.
e. , transfer learning). A general structure of transfer learning is represented in Figure  10 , wherein the knowledge is transferred from a model trained on the large dataset to a new model  [34]. Figure  11  shows the training settings for the elderly dataset. Figure  12  shows how the trained models process the input image. First, the input image splits into 416 × 416 grids. Second, bounding boxes are predicted by each grid, along with their confidence scores and class probabilities. Finally, classes are detected on the output by tracing bounding boxes on the originally input image. dataset (i. e. , transfer learning). A general structure of transfer learning is represented in Figure  10 , wherein the knowledge is transferred from a model trained on the large dataset to a new model  [34].
Blindfold Testing The blindfold testing approach is adopted to examine the eye detection model further. This approach is basically training the model on a number of faces and testing it on new faces. In this way, the model is tested for its ability to work in a practical scenario where it will be subjected to varied images. In order to do so, the TFW elderly dataset is used for training (504 images), and the IRDatabase and TUFTS elderly datasets are used for testing (152 images). The previous data augmentation techniques are applied to the training dataset, where its size increased by three times. Additionally, the same training settings used before (130 epochs, a batch size of 32 and an image size of 416) are applied for the blindfold model. The training started from the weights of YOLOv7 model trained on the augmented multi-age dataset.
Detection of Inner Canthus Temperature After detecting the two eyes in a bounding box, this region is scanned for the highest temperature. The highest temperature corresponds to the inner canthus of the eye. As mentioned in the introduction, the inner canthus is the optimal region to detect the body's temperature using an infrared thermometer  [9]. The process of detecting the temperature is done using a code generated in MATLAB R2021a, wherein the temperature data is scanned for the highest temperature, starting from the location of the upper left edge of the bounding box to its lower right edge. The eye region, the inner canthus point, and the temperature are then presented on the image processed. In order to further validate the detected inner canthus point, we followed the following procedure: 1.
Trained Models Results The training and analysis of results of YOLO models are performed on Google Colab Python-based virtual machine using a TensorFlow library for machine learning and artificial intelligence  [36]. The results of testing the trained models are displayed in Table  2. The metrics chosen to evaluate the different trained models are precision (P), recall (R), and mean average precision (mAP). Precision shows the ability of the model to detect only correct predictions (true positives). A prediction is said to be a true positive (TP) if the IoU of the bounding box is greater or equal to 0. 5. Otherwise, if the IoU is less than 0. 5 or a duplicated bounding box exists for the same object, then the prediction is denoted as a false positive (FP). On the other hand, a false negative (FN) occurs when there exists a ground truth bounding box but the model did not predict it.
An example of eye detection using YOLOv7 elderly model, which is a result of transferring the augmented multi-age dataset model, is presented in Figure  13. Sensors 2022, 22, x FOR PEER REVIEW 13 of 20 mAP= ∑ AveP(q) Q q=1 Q ( 11 ) where Q signifies the number of queries in the set and AveP(q) signifies the average precision for a given query (q). An example of eye detection using YOLOv7 elderly model, which is a result of transferring the augmented multi-age dataset model, is presented in Figure  13. In order to test the inference of the trained models, we have recorded a video of 13 s of a volunteer looking at the camera and turning slightly to the left. Table  3  shows the size, inference time and frame processed per second (FPS) of the different training models. FPS is the number of image frames per second processed by the model.
On the other hand, comparing the scenes between the different models' detections allows us to infer their performance. For example, Figure  14  shows how YOLOv5n model falsely detects number 0 as one eye. However, such false detection is not found in YOLOv5s. The YOLOv7 model trained on the augmented multi-age dataset is tested for distance effect. A volunteer was asked to stand close to the camera one time, and to move further the other time. The testing performance can be found in Figure  15. The YOLOv7 model trained on the augmented multi-age dataset is tested for distance effect. A volunteer was asked to stand close to the camera one time, and to move further the other time. The testing performance can be found in Figure  15. The YOLOv7 model trained on the augmented multi-age dataset is tested for distance effect.