dataesr/scientific-paragraphs-categorization huggingface dataset:
Results In this section, we present the results of a series of experiments where we compare three classical filters: bandpass, wavelet, and Savitzky-Golay, with two LSTM-based deep-filter approaches-MTO and MTM. We use fivefold subject-independent crossvalidation on the MMSE-HR, VIPL-HR, and COHFACE databases. To evaluate the filtering methods rigorously, we use the mean absolute error (MAE) and the Pearson correlation coefficient (r) to describe the quality of the heart rate estimations, and also the signal-to-noise ratio (SNR), and the template match correlation (TMC), to describe the signal quality. Metrics are calculated with a 15-s sliding window with a 0. 5-s step. SNR, TMC, and r are to be maximized, while MAE has to be minimized For ease of data visualization, the MAE metric is presented in the figures with the vertical axis inverted; this is done so that the best performances for the four metrics are always at the top of the graphs.
Intra-dataset Figure  2  presents the results associated with the heart rate measurement metrics in the MMSE-HR, VIPL-HR, and COHFACE databases. On the left are the MAE values, and on the right r, on the upper part are the results of the k-fold cross-validation, and on the lower part is the final evaluation. Figure  3  depicts the metrics related to the signal quality, in the left part the SNR coefficient and the right part the TMC coefficient. Table  2  contains the rPPG-SNR average of the signals acquired in each method, where the same 2,256 subjects have a total rPPG signal duration of 1,114. 5 min for the six  sets. As the second part of this experiment, six state-of-the-art methods for rPPG signal estimation: PVM, POS, PbV, G-R, Chrom, and Green, were chosen to be applied to the VIPL-HR images.
Discussion Based on the tables and figures presented in "Results" section, this section analyzes in detail the behavior of the LSTM network in each of the proposed experiments. In the intra-dataset experiment, Figs. 2  and 3  show that MTO and MTM present good results in the MMSE-HR database compared with classical filters. The BP and WV filters have results very close to those obtained by LSTM-based filter approaches in MAE and r. We believe this is due to the good quality of the videos and scenarios within the database, verifiable in the minimal difference in MAE and r from the raw rPPG signal and all the filtering methods analyzed. In the VIPL-HR database, the MTO and MTM approaches always have the best results, being the MTM approach the best one. The  In the COHFACE dataset, the MTM approach is the LSTM-based filter that gives the best results, with the only exception of r in the final evaluation, where the value r = 0.
19  is very close to the best result r = 0. 22. The poor values shown in the raw signal rPPG in MAE and r can be associated with the compression of the videos present in this specific database. It is important to note that for the three databases, the best SNR and TMC values are those acquired by the two approaches of the LSTM network. However, MTM increases the quality of the signals to a greater extent. In Tables  3  and 4 , it can be seen that the LSTM-based filter always gives the best results compared with the filtering methods used in the literature. In MMSE, VIPL, and COHFACE databases, SNR and TMC metrics show significant improvements in signal quality when using the MTM approach. On the other hand, the MTO approach is only slightly higher than MTM in the MAE and r metrics. Finally, Fig.
4  shows that the filter is stable and able to work with rPPG signals from different algorithms. As a conclusion of this experiment, we could say that in an intra-dataset scenario, both sequenceto-sequence approaches outperform the results provided by the filters found in the  literature; however, MTM turns out to be better than MTO, especially when it comes to improving the quality of the signal. The cross-dataset protocol is particularly interesting because based on Fig. 5 , there is a dependence between the signals used during training and testing. This behavior may be because each dataset has rPPG signals of a specific quality. For example, the MMSE database has an rPPG-SNR average of 7. 65 ± 4 dB, indicating that the signals are mostly of good quality, VIPL-HR has a larger amount of data, and its rPPG-SNR average is 1.
04 ± 4 dB, COHFACE on the other hand, has an rPPG-SNR average of -0. 96 ± 4 dB. This signal quality difference between the three datasets is also visible in Fig. 9   the training in MMSE-HR but testing in COHFACE, we can see that LSTM-MTO has the best results for the MAE and r metrics, but some values are lower in SNR and TMC. LSTM-MTM, on the other hand, has a contrary behavior, it improves the other filters in SNR and TMC, and although it improves the result of r, it does not manage to overcome the BP and WV filters in MAE. Therefore, neither of the two proposed methods is sufficiently robust to over-perform the four metrics. LSTM-DF-MTM shows to be more sensitive to the noise of the training signals, this is visible when we test in VIPL and MMSE having trained in COHFACE, LSTM-DF-MTO on the other hand, is more stable in the same conditions.
Therefore, the cross-dataset scenario shows that for the LSTM-based filter, there is a dependence on training and test data related to the quality of rPPG signals, reflected in the rPPG-SNR value of the databases. We corroborated this hypothesis in the rPPG-SNR experiment. Regarding the experiment amount of training data, in Fig  6 , the many-to-one approach shows that when decreasing the number of data from 100% to 50% and 25%, MAE and r remain stable, when decreasing from 10% and 5%, MAE is stable but the value of r starts to decrease, finally when having 1% of data MAE and r decrease. Concerning signal quality metrics, the behavior is similar; MTO still has higher values than the classic filters in all configurations, although in the 5% and 1% of data, SNR and TMC show lower values than those present between 100% and 10% of data.
In HR measurement, the MTM filter shows a higher sensitivity to the amount of training data than the MTO. Even though the MAE value remains relatively constant, r starts to decrease from taking percentages equal to and less than 25%. In the case of signal quality metrics, the many-to-many approach shows a higher performance than manyto-one regardless of the amount of training data, and it is always higher than the conventional filters. Both sequence-to-sequence approaches can be trained with a set similar to the VIPL005 (45 min, 90 signals) and start to obtain better results than conventional filters in a test set with similar rPPG-SNR values. In the HR measurement metrics presented in the experiment rPPG-SNR dependence (Fig. 7 ), the MTO filter does present dependence on the rPPG-SNR during training; MAE values improve from VIPLAQ0 to VIPLAQ3, and the results decrease only a little in VIPLAQ4, r presents a similar behavior by having better results in the training sets with low values of rPPG-SNR than in the sets with the higher ones.
Conclusions In this article, we analyzed the performance of an LSTM network for rPPG filtering using multiple protocols. Two sequence-to-sequence filter approaches were evaluated: many-to-one and many-to-many. We used three public databases in different experiments from which we can draw the following conclusions: the experiment amount of training data showed that a relatively low number of signals is needed to train the LSTM network efficiently. It was shown that even a dataset of approximately 90 signals totaling 45 min in total could be sufficient as a training set. In an intra-dataset scenario, both MTO and MTM over-perform the conventional filters, but it is the MTM approach that gives the best results. We presented a comparison between the six state-of-the-art methods for rPPG estimation: PVM, POS, PbV, G-R, Chrom, and Green after using the LSTM-based filter.
The results suggest stable values better than conventional filters, where using the LSTMbased filter on rPPG signals acquired by the PbV method gave the best performance. The cross-dataset and rPPG-SNR dependence protocols are perhaps the most interesting experiments as they reflect a clear dependence of the LSTM-based filter on the rPPG-SNR average of the training and testing sets. Concerning this aspect, experiments showed that it is recommended that the rPPG-SNR average of the training set has to be as close as possible to those of the test set. This ensures the LSTM-based filter overperforms classical filters even in a cross-dataset scenario. The experiments let us appreciate how an LSTM-based filter is a better alternative than the classical filters, which improves not only the heart rate measurement but also the quality of the signal.
MMSE-HR The Multimodal Spontaneous Emotion Corpus-Heart Rate database (MMSE-HR)  [49]  includes videos with many facial movements and expressions. The database has been built for further investigation in emotion recognition. Forty subjects are recorded performing 102 tasks, the resolution of each video is 1040 x 1392 pixels with a frame rate of 25 fps, the BIOPAC 150 data acquisition system was used to obtain the blood pressure ground truth at 1 kHz. The duration of each video is between 30 and 60 s.
VIPL-HR The research in this paper uses the VIPL-HR database collected by the Institute of Computing Technology Chinese Academy of Sciences  [22, 50]. The database contains 107 subjects recorded by three different instruments in nine scenarios: stable, motion, talking, dark, bright, long distance, exercise, phone stable, and phone motion. Although this database contains 752 near-infrared videos, we only consider the 2378 visible light videos, the resolutions of the videos are between 960 x 720 and 1920 x 1080, at 25 and 30 fps, respectively. The ground truth photoplethysmography signals were recorded using the CONTEC CMS60C BVP sensor at 60 Hz.
Ground truth selection In the databases used, due to the movement of the subjects, or failures in the acquisition devices, some ground truth signals present anomalies. These inconsistencies (gaps and false peaks) usually happen at the beginning and end of the acquisitions and less frequently during the measurement. However, due to the nature of the LSTMbased networks, it is not suggested to perform the training procedure with ground truth signals that present these types of problems. Therefore, a ground truth selection step is necessary. For this purpose, ground truth signals are checked individually to take only the continuous segment with a reliable ground truth morphology. In most signals, it was sufficient to remove the first and last seconds, while a considerable part of the signal was removed in a small group.
In order to maintain the reproducibility of the studies carried out in this article, attached to this document are the supplementary files with the information of the signals that were cropped in each database (Additional files 1, 2, 3). Once only the reliable ground truth signals are selected; Table  7  contains the parameters related to rPPG signals acquired by the PVM method (PVM-rPPG)  [13]  from the three databases: MMSE-HR, VIPL-HR, and COHFACE. rPPG-SNR is the average of the SNR coefficient of rPPG signals. Figure  9  depicts examples of rPPG signals from the three databases used. Note how the quality of the acquired rPPG signals varies for each database. This is due to the nature of the scenarios where the videos were acquired for each database, i. e. , luminance, movement, and other factors present during the acquisition.
It is essential to mention that the VIPL-HR database was designed to estimate rPPG signals and contains a large amount of data during various scenarios. Due to this, its average rPPG-SNR is 1. 07 dB. MMSE-HR and COHFACE, on the other hand, have a considerably smaller amount of data and differ from each other, being MMSE-HR the database with the best quality in its rPPG signals with an average rPPG-SNR of 7. 65 dB, and COHFACE with an average rPPG-SNR of -0. 96 dB, being the most.
Protocols In the following, we will describe the experiments carried out to validate the use of an LSTM network for filtering rPPG signals. We conducted a classical intra-database evaluation followed by a cross-dataset evaluation. Additionally, we present experiments to study in more detail the advantages and limitations of using an LSTM-based network for rPPG filtering. First, we demonstrate that it is not necessary to use a large amount of data to train the network successfully and that there is a clear dependence of the rPPG-SNR average in the rPPG signals during training. It is important to mention that there is no overlap between training and testing data in any of the experiments presented. Figure  10  shows the distribution made in the databases used to study intra-and crossdataset evaluations.
A lists the databases used: COHFACE, MMSE-HR, and VIPL-HR. B contains the rPPG signals acquired by: PVM  [13] , POS  [17] , PbV  [16] , G-R  [17] , Chrom  [18] , and Green  [3]. C presents the intra-dataset and cross-dataset evaluations with metrics measurement. Figure  11  resumes the PVM-VIPL signal generated in Figure  10 , to be used in the study of the influence of the amount of data and noise on the rPPG signals during the training of the LSTM network, the protocol related to the amount of training is later called amount of training data (top panel), and the protocol related to noise is called rPPG-SNR dependence (bottom panel).
Intra-dataset In this standard evaluation protocol, we divided each dataset into 80% training and 20% testing to perform a final evaluation, within the 80% we developed k-fold cross-validation (CV for short) of fivefold, as shown in Fig. 12. In the first part of this experiment, we evaluated the performance of the LSTM-MTO and LSTM-MTM for the PVM-MMSE, PVM-VIPL, and PVM-COHFACE sets. In the second part, to analyze the filter robustness with rPPG signals acquired by different state-of-the-art methods, we chose the PVM-VIPL, POS-VIPL, PbV-VIPL, GR-VIPL, Chrom-VIPL, and Green-VIPL sets.
Amount of training data In this experiment, we wanted to study the sensitivity of the LSTM-based filter to the amount of information given during training. A stratified train-test-split in PVM-VIPL was made, having the VIPLA and VIPLB sets with 80% and 20% of signals, respectively. Consequently, we used the VIPLB set as a testing set and VIPLA as the training one. This way, we take the set VIPLA and perform a stratified division at 100%, 50%, 25%, 10%, 5% and 1% of the data, calling these training sub-sets VIPLAx with x=[100,050,025,010,005,001]. Note that the set VIPLA100 is indeed the same VIPLA, but this notation is used only to appreciate the sub-sets characteristics easily. The stratified division guarantee sub-sets with a balanced rPPG-SNR. This last detail is important as we will see in rPPG-SNR dependence, that having training and testing sets with unbalanced average rPPG-SNR may change the performance of the network.
rPPG-SNR dependence In this experiment, we wanted to simulate a scenario where the training and testing sets of the LSTM-based filter present a different signal quality given by the rPPG-SNR values. This scenario may happen when rPPG signals are estimated from different approaches, as we did in cross-dataset, and in the intra-dataset experiment with PVM-VIPL, POS-VIPL, PbV-VIPL, GR-VIPL, Chrom-VIPL, and Green-VIPL. In order to assign a label representing the level of the rPPG-SNR, the SNR average of each subject ( SNR i with i ∈ [1, N ] ) was measured and assigned a label e i : 0, 1, 2, or 3, where 0 indicates a low value of rPPG-SNR and 3 a high one. We relate the value of rPPG-SNR to the quality of the signals, and to define the rPPG-SNR thresholds for the labels, the maximum and minimum SNR values were taken from all subjects, and this range was divided among h-length 4 sub-ranges.
Model To perform the rPPG signal filtering, we use the long-short-term deep-filter proposed in  [47]. The workflow of the procedure is presented in Fig. 13. As a first step, we perform face tracking allowing us to take into account the pixels of the skin of the face only (ROI selection) followed by a spatial averaging process. Then, we use the RGB channels, and according to an rPPG estimation method, a color channel combination is performed. Afterward, the denoising of the rPPG signal is performed, and finally, the physiological parameter estimation on the filtered rPPG signal is computed through a fast Fouriertransform (FFT)-based analysis. In the rest of this section, we will explain the LSTM-DF filter with its two approaches, MTO and MTM, followed by the procedure for organizing the databases so that they can be used to train the network.
Long short-term deep-filter We need to extract the raw rPPG signal from the videos in the databases. Therefore, having one video with T frames, we compute a spatial ROI selection and tracking on the face of the subject to perform a spatial averaging between the pixels related to the skin. The result is a triplet of RGB values for each frame r j , g j , and b j , where j ∈ [1, T ]. Con- sequently, the vectors r , g , and b represent the RGB values for the full video. This way, being ω a color channel combination function applied by an rPPG estimation method, the rPPG vector y is presented in Eq. 3: With the signal to be filtered, we use an LSTM network constituted by an input, a memory block, and an output. The memory block has the following parts: one input gate that learns which information is stored in the memory block, a forget gate that learns how much information is forgotten or retained from the memory block, and finally, an output gate that takes care of learning when the collected information can be used.
rPPG training dataset building To create the training set for the LSTM network, we consider the rPPG signal y i related to the video i with length T i , where i ∈ [1, N ] and N is the number of videos. Then, for the N videos, we use an L-length sliding window with a step of one frame to create the training matrices Y and G (vector G for MTO). In MTM and MTO, Y is composed of rPPG signals with a fixed length L. G represents the ground truth. In MTM, G is a matrix of PPG signals of same size than Y. G is a vector in MTO approach. Y and G are given as the training set for the LSTM-based filter. The whole procedure mentioned is presented in Algorithms 1 and 2. (5) ŷ G -L-length ground truth matrix [j] = L l=1,j-l+1>0 ỹ(j -l + 1) [l]. 1 c ← 1 /* counter */ 2 for i ← 1 : N do 3 for t ← 1 : T i -L do 4 Y[c] ← y i [t : t + L]; 5 G[c] ← g i [t : t + L]; 6 c ← c + 1; 7 end 8 end With the input data and the neural network defined, all that remains is to know the architecture of the two approaches, MTO and MTM.
Intra-dataset, final evaluation results for rPPG signals estimated from VIPL-HR dataset by PVM, POS, PbV, G-R (GR), Chrom and Green methods performance of the LSTM-based filter in this particular dataset is due to a large number of signals available, allowing the network to learn in a greater variety of signals than the MMSE-HR and COHFACE databases, where there is less variety of the signals.
SPR assay Surface plasmon resonance (SPR) assay was performed to quantify the binding avidity of the recombinant mos-tri-RBD to the receptor hACE2 using BIAcore 8K (GE Healthcare) with NTA chips. Firstly, the His-tagged hACE2 protein was immobilized onto the chip surface. Then, the purified mos-tri-RBD protein sample was serially diluted in HBS-T buffer (HBS buffer and 0. 05% Tween20). The diluted samples were injected at a flow rate of 30 µL/min for 120 s, and then the HBS-T buffer was flowed over the chip surface to facilitate dissociation of the bound protein for an additional 120s. Subsequently, the sensor chip surface was regenerated by injecting 350 mM EDTA solution with a flow rate of 30 µL/min for 120s. The SPR signal response was monitored as a function of time. BIAcoreTM Insight Evaluation software was used to analyze the experimental data.
The dotted curves in the figure represent the direct connection between the C-terminus of the former RBD and the Nterminus of the latter RBD. The schematic structure of mos-tri-RBD was drawn by Chimera software (Pettersen et al. , 2004) based on the PDB file with accession number 6zgi. (B) SDS-PAGE analysis of the recombinant mos-tri-RBD. (C) Concentration-dependent binding ability of mos-tri-.
2), Plasmid-adjuvant 2020 This paper DNA reagent Omicron SARS-CoV- software, (BA. 3), 2-mos-tri-UCSF Chimera team https://www. cg algorithm Alpha, Beta, RBD Chimera at University of l. ucsf. edu/chim peptide, Delta, Gamma, Recombinant This paper California era/ recombinant software, Lambda and mos-tri-RBD BIAcore GE Healthcare protein algorithm Mu pseudo-protein Insight type (species) or cell line (CHO) Designation viruses (mammalian Evaluation CHO-K1 cell expressed) cell-Software Source or reference ATCC Identifiers Cat#CCL-61, Additional information resource software, line GraphPad GraphPad RRID:CVCL_0 https://www. gr peptide, algorithm Recombinant Prism Sino Biological Software 214 Cat#40592-aphpad. com/ strain, strain recombinant Live SARS-monomeric version 8 National Inc. , China V08B background protein CoV-2 his-tagged Institute for (SARS-CoV-2 cell line (Homo- prototype Huh-7 cells RBD of the Viral Disease JCRB Cat#JCRB040 virus) sapiens) virus (QD-01 prototype Control and 3; RRID: strain) SARS-CoV-2 Prevention, CVCL_0336 strain China CDC (Baculovirus- strain, strain cell line Live SARS-Vero cells insect cell- National National background (Chlorocebus CoV-2 expressed) Institute for Institute for (SARS-CoV-2 virus) sabaeus) peptide, protein recombinant Omicron virus (NPRC 2.
Study Design and Ethical Approval The Cat Prospective Aging and Welfare Study (CatPAWS) commenced in 2016, with the key aim of monitoring the health of pet cats during the aging process, facilitated by the establishment of the Feline Healthy Aging Clinic (FHAC) at the University of Liverpool, UK. Cats were enrolled at the FHAC between 7 and 10 years of age with the aim to monitor regularly thereafter using various clinical assessments (see below), data presented here are from their enrolment appointment only. Ethical approval for this study was granted by the University of Liverpool Veterinary Research Ethics committee (VREC491abcd) and the Royal Canin Ethical committee. At enrolment, all owners were provided with a detailed information sheet explaining the purpose of the study and gave written consent for their cat to be enrolled, which included permission to obtain their cat's medical records from its registered veterinary practice.
Data Collection and Storage Owner Questionnaire Data Before attending their first appointment, owners completed the enrolment questionnaire (see Supplementary Material). In section one of the questionnaire, signalment data were captured including age, sex, neuter status, color, and breed. In section two, information about environment and lifestyle (including housing and physical resources) were recorded, whilst the third and fourth sections covered food and water resources, as well as eating and drinking behaviors. The fifth section covered their cat's health, previous and current illnesses, and any observed physical or activity changes from adult (3-6 y) to mature (7-10 y) life stages; the sixth section covered their cat's current behavior and any changes, whilst the seventh section covered details of the primary caregiver, the household and their relationship with the cat.