dataesr/scientific-paragraphs-categorization huggingface dataset:
Statistical Analysis Statistical analyses and data representation were performed using GraphPad Prism 9, software version 9. 4. 1 (GraphPad). The normal distribution was assessed by applying the Shapiro-Wilk normality test. For multiple comparisons, unpaired one-way ANOVA followed by the Holm-Sidak post hoc test (when selecting a set of means to compare) or Dunnett's post hoc test (when comparing every mean to a control mean) was performed. Unless otherwise indicated, data in graphs show the mean ± SD. p value of < 0. 05 were deemed significant (*, p < 0. 05; **, p < 0. 001; ***, p < 0. 0001).
Acknowledgments: The authors thank  Angela Vallone ,  Katharina Blatter , and  Yuxiang Ren  for their excellent technical assistance and the staff of the  ETH Rodent Center HCI  for animal husbandry. Moreover, the authors thank  Dario Neri ,  Abdullah Elsayed ,  Louis Plüss , and  Frederik Peissert  of  Philochem AG, Switzerland  for their introduction to phage display, precious advice, and data discussion.
In fact, a variety of optimization tools has been employed in neural networks to learn from past experiences and use that prior training to identify new patterns and classify new sentimental data. We have recently witnessed a progress in the use of deep learning (DL) for neural networks leading to classifcation  [7] [8] [9] [10]. Facial expression recognition in videos is an active area of research in computer vision. Accordingly, researchers in this feld are interested in developing techniques to interpret facial expressions, code them, and extract their features in order to have a better prediction of emotions. Capitalizing on the remarkable success of deep learning, various architecture types are harnessed to attain superior performance. Te remainder of this paper is organized as follows: Section 2 reviews and discusses the previous approaches used for expression recognition.
Previous Work Sinha and Aneesh  [9]  embedded the handcrafted features in the training process of the network in order to reduce the diference between the features learned by deep networks and the handcrafted ones. Teir work was based on the HoloNet with feature loss (HNwFL) for feature learning and the fusion network for recognition. In HNwFL, the handcrafted feature information was integrated into the HoloNet and the new feature loss was tested on CK+, JAFFE, and FER2013 datasets. When compared with other works, this network obtained the best accuracy with a 97. 35% on the CK+ dataset. In fact, the suggested network provided much better accuracy than the network which did not include feature loss as well as the original handcrafted feature. In the study in  [11] , the initial work of both computer vision and image processing has been developed in order to facilitate the teaching of young autistic children recognizing the human facial expression.
Facial expression recognition work was proposed by Haque et al. using a deep convolutional neural network. In order to experiment and train the deep convolutional neural network model, Kaggle's FER2013 dataset was used. Teir work resulted in 63. 11% accuracy without overftting the model. In fact, the bright images achieved a better accuracy than the dark ones. Tus, the dataset was modifed across four groups with diferent lighting conditions, and each set is again trained with the same model. Mollahosseini et al. [12]  advanced deep neural network architecture to address the FER problem via wellknown face datasets. Seven publically available databases, mainly MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER2013, were used in this experiment. Te proposed network consisted of two convolutional layers.
Each layer was followed by max pooling and then by four inception layers. Te architecture network took facial images as an input and classifed them into six emotional expressions (anger, disgust, fear, happiness, sadness, and surprise). Its results were comparable to or more efcient than traditional convolutional neural networks in both accuracy and training time. Tümen et al. [13]  aimed to build a convolutional neural network (CNN) based on a facial expression recognition (FER) system so as to classify expressions presented in the FER2013 database. Te presented CNN achieved a 57. 1% accuracy rate on the FER2013 database  [13]. It also provided a good result while detecting emotional expressions without any preprocessing. A high rate was particularly achieved in three classes of happiness, surprise, and disgust.
Xie et al. [14]  introduced a novel approach for FER named deep attentive multipath convolutional neural networks (DAM-CNNs). Te proposed model contained three modules including the VGG-Face, the Salient Expressional Region Descriptor (SERD), and the Multipath Variation-Suppressing Network (MPVS-Net). Te VGG-Face (visual geometry group) extracted a feature. Te SERD automatically located expression-related regions in the target image. Te MPVS-Net module separated expressional information from irrelevant variations. By jointly combining SERD and MPVS-Net, the DAM-CNN was able to highlight relevant facial traits and yield a robust image representation for emotion recognition. AlMarri  [15].
Background Te current paper is based on the studies of Github et al. 's  [16, 17]  technique which was selected according to diferent tests presented in the "Study of previous works" section. Tis method was modifed to boost the performance of its neural network. Github et al. 's  [16, 17]  approach is presented through the neural network architecture summarized in Figure  1. Initially, a lot of images were captured with a camera. Te face extraction module then used trained Haar cascade/deep neural networks (DNNs). Te classifcation algorithm was trained on the FER2013 dataset. Finally, a model was generated to classify emotions through diferent facial expressions such as angry, disgusted, fearful, happy, sad, surprised, and neutral. 3. 1. Proposed Method. Te facial expression recognition was difcult to realize due to the slight diference between several emotions, which required an efcient and precise algorithm to be trained.
Te imbalanced distributions of emotion classes gave rise to low accuracy. Terefore, two fundamental rules, notably the data-centric machine learning (ML) and the model-centric approach  [18] , were used in deep learning algorithms. Te data-centric ML aimed at improving the quality of the used dataset by (i) Balancing classes: the number of items should be the same in each class (ii) Data augmentation: adding augmented images to some classes (iii) Increasing diversity: images should be from shops (high quality) and consumer-taken (low quality) (iv) Removing classes Te model-centric approach included two techniques: (i) Early stopping method using Keras (ii) Hyperparameter tuning Two enhancement strategies were adopted to overcome the shortcomings of the Kant et al. method and improve its performance.
Te frst strategy accentuated the RelGAN (relative attributes generative adversarial networks) approach  [19, 20] , while the second introduced the early stopping technique. Tese two strategies were explored in the functional block diagram of our implementation, as shown in Figure  2. First, the anavid dataset  [21]  was collected from both private and public datasets and its preparation was optimized in three versions. Ten, various data augmentation techniques were exercised on the anavid dataset. Moreover, the RelGAN method was performed on the same dataset. After creating our own dataset, the resulted images were divided into train, test, and validation sets as per its usage. Afterwards, memory optimization was integrated into the algorithm. Subsequently, early stopping was applied to the neural network architecture.
Data Augmentation. Data augmentation is a widely employed method in computer vision, which entails applying diverse transformations to the initial training data, thereby generating additional augmented samples. By adopting this approach, the size and diversity of the training dataset can be expanded, leading to enhanced performance and robustness of computer vision models. Both input images and the labels for those images can be enhanced with data using certain techniques. In computer vision, the following are some typical data augmentation techniques: Tese are just a few instances of computer vision techniques for data augmentation. Te selection and combination of the augmentation methods depend on the particular task, dataset, and the desired characteristics that the model should learn. Data augmentation is a useful technique for enhancing computer vision models' performance and generalization, particularly when there is a lack of training data.
Generative Adversarial Network (GAN). Te generative adversarial network (GAN), defned as a powerful unsupervised generative model, was widely applied in order to generate photorealistic images and a variety of other image types. It is, thus, benefcial to classify emotion recognition and train data augmentation. An analysis study of the GAN method was performed leading to a whole GAN family identifcation including CycleGAN, DCGAN, MoCoGAN, and RelGAN  [22] [23] [24]  which are representative methods in a multidomain image-to-image translation. Te RelGAN technique was selected, and its code was modifed. Te RelGAN technique consists of generating the same face with other expressions and looks ("Pale_Skin," "Smiling," "Eyeglasses," and "Gray_Hair"). Tis method is capable of modifying images by changing particular attributes of interest  [19, 20].
Four attributes were applied to public datasets: (i) Pale skin (ii) Smiling (iii) Eyeglasses (iv) Gray_Hair Figure  3  illustrates the original image and its four efects ("pale skin," "Smiling," "Eyeglasses," and "Gray_hair"). Tests revealed the efciency of the RelGAN technique on a public dataset because the public images were taken from a close distance and the facial features were therefore clear. Using this technology, a DL algorithm could generate more facial expressions. Examples of the RelGAN technology are presented in Figure  3.
Datasets. A series of experiments was performed over diferent datasets: JAFFE  [27] : Te Japanese Female Facial Expression (JAFFE) database contains 213 images of posed expressions from 10 Japanese female subjects. Each subject represents 7 diferent emotional facial expressions (6 basic facial expressions + 1 neutral). Te database is challenging since it consists of a little example images per subject/expression. Tis dataset is asked to do acted facial expressions related to Eckman's emotions, including the following facial expressions: "happiness," "anger," "sadness," "surprise," "disgust," "fear," and "neutral. " Te resolution of original facial images is 256 × 256 pixels with tif format. Several dataset images of each expression are assembled by Miyuki Kamachi, Michael Lyons, and Jiro Gyoba, at Kyushu University, Japan.
Each image is annotated with average semantic ratings on nouns describing the posed expression by 60 Japanese viewers. Facial expression recognition (FER) 2013  [28] : Te FER2013 database was presented during the ICML 2013. Te dataset was labeled and created by the Google search API. It  [29]  is used in 32% research studies and consists of 35,887 images. In fact, FER2013 includes 28,709 training sets, 3,589 validation images, and 3,589 test sets. In order to label all the 36k photos, the dataset uses Ekman's emotions which are composed by seven emotional expressions (neutral, fear, disgust, sadness, happiness, surprise, and anger). Each image is in grayscale with a resolution of 48 × 48 pixels. Te images are smaller than other datasets. FER has more variation in the frames, including facial occlusion (mostly with a hand), partial faces, low-contrast images, and eyeglasses.
KDEF  [30] : Karolinska Directed Emotional Faces (KDEF) comprised images from 70 individuals: 35 men and 35 women, all between 20 and 30 years old, where each image belonged to one of the following classes (fear, anger, disgust, happiness, neutrality, sadness, and surprise) and each expression is photographed from fve diferent angles (full left profle, full right profle, half left profle, right half profle, and straight profle). Te KDEF dataset represents color photographs with a 562 × 762 image format. Te KDEF dataset  [31]  consisted of 4,900 images which were made at the Karolinska Institute, Stockholm. It is widely used in the feld of facial expression recognition. Te extended Cohn-Kanade (CK+)  [32]  is a popular facial expression recognition dataset and is commonly used in several works.
Te CK+ dataset contains 593 image sequences of persons with diferent age (18 to 50 years old), gender, and heritage. Te dataset contains images with a resolution of 640 × 490. Among all, 327 sequences are labeled with an emotion in seven basic expression labels: anger, contempt, disgust, fear, happiness, sadness, and surprise. (MMA Facial Expression Database MMAFEDB)  [32]  is collected from diferent emotion and expression images with a resolution of 48 × 48 pixels. It is divided into 3 sections as testing, training, and validating. Each section includes seven Te Yale face database  [33]  contains 165 grayscale images of 15 individuals with diferent facial expressions including center-light, w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink. Te GENKI database  [34]  was collected by the Machine Perception Laboratory (MPLab), University of California, San Diego.
Te MPLab GENKI-4K is a database subset containing 4,000 real-life faces annotated in two classes of images "smiling" and "nonsmiling," downloaded from publicly available Internet repositories. Tese images involve a wide range of backgrounds, geographical locations, illumination conditions, personal identity, and ethnicity. Tis dataset has large variations in pose, age, and gender. It consists of 1,940 grayscale face images with varying facial expressions. Te GENKI-SZSL subset includes 3,500 images collected from the Internet. Tey are classifed according to face location and size. Te current release of the GENKI dataset is GENKI-R2009a, a version which consists of 7,172 images that combine to form the following subsets: (1) GENKI-4K: 4,000 images representing expression and head-pose labels (2) GENKI-SZSL: 3,500 images introducing face position and size labels A training set consisting of 2,958 data samples is selected from the whole GENKI dataset.
Correa et al. [39]  designed a neural network based on an artifcial intelligent system for emotion recognition. Tree promising neural network architectures were trained and subjected to various facial expressions' classifcation tasks. Ten, the best performing network was further improved. For this purpose, diferent approaches were experimented and evaluated. Te fnal model was portrayed and applied in a real video stream application that could instantaneously return the user emotion. In this paper, deeply trained models for emotion detection are presented through the use of the FER emotion databases. Te Github et al. method  [16, 17]  achieved the best results compared to the Correa et al. [39]  and Li et al. [38]  methods. Te simulation has the most prominent values in diferent classes (angry, happy, sad, and neutral).
Model Comparison (with and without Data Augmentation). Te small dataset afects the trained model. Tey do not generate reliable data from the test and validation image. Tus, these generated models sufer from the overftting issue which can be solved through various proposed methods. According to the study in  [40] , the principle of this approach is to add regularization to the weights. Another technique is a dropout technique  [41]. Tis principle consists of dropping certain connections in layers or removing a neuron from layers during training. Another popular method is batch normalization  [42]. It is applied to any neural network layer. Some works explore data augmentation techniques to resolve the overftting problem. In deep convolutional neural networks, the limited numbers of pictures contribute to the overftting phenomena.
During training, the number of samples in each class is too small and the rate of accuracy is wrong. For this reason, a diferent kind of data augmentation is applied to the dataset. Various data augmentation techniques such as rotating, noise adding, fipping, zoom, cropping, and brightness are performed. As a result, the loss classifcation is reduced, and the neural network learns better and generates the best model. Tis strategy is an efective way to improve the accuracy of image classifers. As it can be noticed in Table  3 , data augmentation is an efective method for image classifcation. It can be observed that the balanced dataset model based on data augmentation achieves a high true positive value of four emotion categories compared to unbalanced dataset models. Te most facial landmarks are the mouth, the corners of the eyes, the lobes of the ears, the chin, and the tip of the nose.
Tis helps to distinguish emotion class for pictures captured in the front of the camera. In the case of blurry images or pictures captured by the position away from the camera, the algorithm provides incorrect classifcation (see Table  4 ). Tis model can be improved by adding a bunch of datasets and collecting enough data for training using the RelGAN technique. In the following section, future experimentations may be fulflled using various datasets. Te early stopping method is realized in the implementation phase. Te hyperparameters of early stopping are adjusted to further improve the model. Te hyperparameters are summarized in Table  5. In this section, specifc parameters are described, including mode, monitor, patience, verbose, min_delta, and restore_best_weights. "mode": mode is set to min.
We seek the minimum for validation loss and the maximum for validation accuracy. "monitor": Quantity to be monitored. "patience": Te number of epochs without decreasing the monitor after the training stops. For example, if patience is set to 3 without decreasing the loss for three consecutive epochs, the training stops. Te "verbose" can be tuned to 1. Once the training is stopped, the epoch number is printed. "min_delta": min_delta is fxed to 0. 00001. Terefore, an absolute change of less than min_delta cannot be considered as an improvement. "restore_best_weights": Te restored best weight is fxed to true positives to make sure the fnal model we get is the best. Supervised machine learning (ML) algorithms require a lot of data to be able to generalize models. Te principle of data augmentation is to generate more images from the original dataset and create more variations of image appearance such as backgrounds and diferent contexts.
Terefore, during training, more image variations are obtained to boost the performance of the recognition model. In order to increase both data size and training data diversity, various transformations are used. As a result, the dataset anavid  [21]  is modifed in three diferent sets and each of these datasets is again trained. Table  6  contains these datasets with distribution frames of each class. A more detailed analysis of our training data provides the frst element of understanding. When we do not have sufcient data, it is impossible to carry out the initial application efciently. In fact, the number of images for each class is not equivalent, so unbalanced classes are obtained, frequented classes are acquired, and rare classes are procured in case we have little data. Tis lack of data blocks our network in two diferent ways: (i) By not providing it with sufcient information to allow it to learn to diferentiate certain classes (ii) By leading it to specialize in learning data (overftting) and therefore be unable to provide a correct answer for the test data For this reason, the neural network can face the overftting problem.
In brief, neural networks require a lot of learning data and balanced classes. Te model which is generated by training the abovementioned three datasets is tested on a test dataset that includes 2,000 images for each class (happy, neutral, angry, and sad). Te result is illustrated in diferent confusion matrices (see  Table 7). It is interesting to look at class-by-class results through a confusion matrix. It indicates the proportion of data correctly predicted for each class through a diagonal column. Te rest of the data is assigned to other classes (see Table  8 ). Tis shows a considerable improvement in the confusion matrix as it helps to generalize the model and boost its performance. Early stopping is utilized to avoid overftting in our experiments. Troughout the simulation, the designed networks are trained with an iterative technique which allows the model to better ft the training dataset.
Conclusion Facial emotions are important factors in human communication. Te feeling and the expression of emotions are the basic skills of social interaction. Tis work aims to study human emotion. FER can also be combined with early stopping and data augmentation using ordinary transformations and GAN technologies in order to boost the performance of our DL algorithm. Moreover, diferent experiments are applied to analyse the validity and the applicability of this technique with other methods in the feld of emotion recognition. In future works, we will carry out new experiments to further ameliorate this research paper. In addition, we will further enhance the performance of the Jetson card by studying the structural characteristics of embedded systems and improving their functionality.
(27) P * 0 = K r G F 0 -2 F -T 0 + w i n i -n 0 i F -T 0 , (28) ∇ ⋅ P * = 0 or ∇ 0 ⋅ P * 0 = 0, the time evolution laws in (20) for the two protein fractions GPCRs and MRPs involved in the ligand binding processes, as follows: With the aid of the software COMSOL Multiphysics®[119], numerical solutions of Eq. (  29 ) have been carried out by considering the model parameters reported in Table  1. In particular, a circular domain Ω = {(x, y) ∈ R 2 ∶ x 2 + y 2 ≤ R 2 } , with R = 5 m , and a time span t ∈ 0, t max } , with t max = 1 h , have been considered in the analyses, in anal- ogy with numerical set up referred to  Carotenuto et al. (2020). The simulation, performed using the classical coefficient form PDEs, provide homogeneous initial conditions for the protein fractions (x, y, 0) = 0 and (x, y, 0) = 0.
e. , after 0 h, 12  Biodegradation and bacterial function DOC concentration was measured with a Shimadzu TOC-L for all long-term incubation samples and the short-term incubation samples at the subarctic site, and with a Shimadzu TOC-V CPH for the short-term incubation samples at the alpine site. DOC degradation kinetics were modeled using a reactivity continuum model, with a Gamma distribution as the initial distribution of reactivities, according to Koehler et al. 35. Model parameters were estimated using the nonlinear regression package nlme in R 36. Oxygen was measured with a Microx system (PreSens) to determine bacterial respiration and ensure oxic conditions along the experiment. Bacterial respiration rates are obtained as described in Supplementary Information 4. Bacterial carbon production was determined via incorporation of 3 H-leucine according to Smith & Azam  37.
Mass spectra were collected between 150 and 2000 m/z over 500 scans and internally calibrated using a list of 51 expected compounds. Peaks were identi ed using the Bruker Daltonics data analysis software. The detection limit was calculated according to Riedel and Dittmar  39 (2014). Molecular formulae were considered over the ranges C 1-40 H 1-82 O 1-40 N 0-4 S 0-1 P 0-1 under the conditions O ≤ C; 0. 3C ≤ H ≤ 2C+2; O > (2P + S). Regarding heteroatoms, only molecules with these combinations: N*S*P, N2*S, N3*S, N4*S, N2*P, N*S2, N3*P, N*S2, N2*S2, N3*S2, N4*S2, S2*P were considered. Unassigned masses were not further included in the data treatment. Sample SAA_S1_A_T72 was lost and replicates GKS_S1_A_T0 and GKS_S2_A_T72 were removed as outliers based on multivariate ordination (Supplementary information 3.