dataesr/scientific-paragraphs-categorization huggingface dataset:
Comparison between pre-surge and post-surge observation Two separate training scenarios were tested. In the first, we computed the PCA axes and embeddings on the 2015 data set. Then we applied this data encoding to the 2018 images. We call this scenario Single, because only one single calculation of the cAE network or of the main PCA axes was performed. In a second scenario, we recalculated the embedding or PCA axis decompositions on the images of the second period: we call this mode Dual because the calculations are performed twice: once for 2015, the second for 2018. The visualization results obtained are shown in Fig. 3. Globally, we note a majority of the two complementary colors green and magenta, especially on the CAE visualisation. We can verify that the colored channels corresponding to the second and third axes of CAE are highly correlated (0.
Observation of the whole dynamics When the scenario is performed on the entire image, and over a much longer acquisition period, then the conclusions become very different from the previous ones. The color compositions found for the glacier image are shown in Fig. 4. If it is difficult to make a quantitative evaluation, it appears that the CAE type visualization brings out as much information as the PCA, with a good spatial homogeneity of the zones, even though no spatial information was used in the learning architecture. This time, we can verify that the 3 axes of the CAE are equally well decorrelated, and therefore provide non-redundant information. In conclusion on this dataset, it turns out that on very short time series, both types of visualization are sensitive to differences between the observed periods, such as glacier crevasses.
CAE seems to obtain better contrasts and better segmented areas corresponding to different generic profiles, which tends to confirm our previous results obtained on agricultural plots. The richness and complexity of the observed temporal profiles justify the use of deep learning algorithm to extract all the information as well as possible, and not necessarily in a linear way. On shorter events such as 10 dates, CAE struggles to find three degrees of freedom. Therefore, a CAE could be used to establish a criterion to determine the number of degrees of freedom necessary to describe the diversity of temporal profiles. For example, the computation of correlations between the different components of the output of an autoencoding could be a valuable a posteriori tool to determine the optimal size of the data compression space.
Distance to water Distance to the nearest permanent water source (coastal rivers and lagoons, Fig. 1 , Table  2 )  (Cawthra et al. , 2020)  was calculated in ArcMap version 10. 5 (ESRI, 2017) using the Near tool. We determined the probability of an animal occurring at a certain distance from water from geospatially referenced game census data for Kruger National Park between 2008 and 2017 (South African National Parks, unpublished data). The probability that a herbivore would occur at a certain distance from a water point was calculated using an exponential regression equation (r 2 = 99. 6, Fig. 2 ) in CurveExpert  (Hyams, 2020). Regardless of species or functional group, species tend to aggregate nearer to water when it is available, and for this reason we are not differentiating between functional groups in this paper.
Statistical analysis To determine the probability of occurrence of herbivore functional groups we ran general linear models (GLMs) for each of the five functional groups. Models included herbivore biomass  (Brooke et al. , 2021)  as the dependent variable and probability of occurrence in relation to water, vegetation type, vegetation density, and proportion human risk (either low or high risk; both were not included in the same model). We also included two-way interactions between all possible combinations of these variables. Categorical variables (vegetation type and vegetation risk) were included as factors in the GLMs. Continuous variables (probability of occurrence in relation to water and herbivore biomass) were scaled between 1 and 0. For the GLMs, we employed a model averaging approach using the best fitting models (those with a akaike information criterion [AICc] < 2) using the dredge function in the MuMIn package  (Bartoń, 2020).
Acknowledgements. We thank  Nelson Mandela University  and the  African Centre for Coastal Paleoscience  for funding this study. South African National Parks  for supplying distribution data on mammals in Kruger National Park. CWM recognises the support of a grant from the  National Science Foundation  ( BCS-0524087 ,  BCS-1138073  and  BCS-1460376 ),  Hyde Family Foundations , the  Institute of Human Origins (IHO) at Arizona State University , and the  John Templeton Foundation  to the  Institute of Human Origins at Arizona State University. The opinions expressed in this publication are those of the author(s) and do not necessarily reflect the views of any of these funding organisations.
The bright radio galaxy PKS 1934-63 was used as a flux and bandpass calibrator (observed for 10 minutes); the nearby quasar PKS 0237-233 was observed as a gain calibrator for 3. 5 minutes after each 20-minute observation of the LADUMA field. The total on-source integration time was 7. 3 hr. We reduced the data across the full L band, excluding regions affected by radio frequency interference (RFI), after splitting into 25 MHz spectral windows (SPWs). For the source discussed in this paper, the relevant SPW spans 1086-1111 MHz. Bandpass, flux, and phase calibration were performed using the PROCESS- MEERKAT pipeline, 52 which is written in Python, uses a purpose-built CASA  (McMullin et al. 2007 ) Singularity container, and employs MPICASA (a parallelized form of CASA). Flux calibration used the  Reynolds (1994)  model for the spectrum of PKS 1934-63, which is ultimately tied to northern hemisphere calibrators with flux uncertainties of ∼5% at ∼1 GHz (e.
20-275518. 1. We also thank the anonymous referee for comments that have helped improve the paper. The MeerKAT telescope is operated by the South African Radio Astronomy Observatory (SARAO; www. sarao. ac. za), which is a facility of the National Research Foundation (NRF), an agency of the Department of Science and Innovation. The authors thank the members of the SARAO engineering, commissioning, and science data processing teams for building and operating an absolutely superb facility. The MeerKAT data presented in this paper were processed using the ilifu cloud computing facility (www. ilifu. ac. za), which is operated by a consortium that includes the University of Cape Town (UCT), the University of the Western Cape, the University of Stellenbosch, Sol Plaatje University, the Cape Peninsula University of Technology and the South African Radio.
52 https://idia-pipelines. github. io/docs/processMeerKAT 53 https://docs. ilifu. ac. za/#/about/what_is corrected to the heliocentric reference frame and rebinned by a factor of 2. At the observed frequency, each rebinned (52. 2 kHz) channel corresponds to 14. 3 km s -1 in rest-frame velocity. A single-Gaussian fit to the line spectrum yields (with fit uncertainties noted as errors) a central frequency of ν = 1095. 07 ± 0. 09 MHz, a rest-frame line width Δv = 459 ± 59 km s -1 (FWHM, which corresponds to an observed frequency width of 1. 68 MHz), a peak flux density of 1. 42 ± 0. 16 [±0. 07] mJy, 54 and a total spectral line flux F line = 0. 69 ± 0. 02 [±0. 03] Jy km s -1 that agrees well with a direct integral of the spectrum (0. 69 ± 0. 02 [±0. 03] Jy km s -1 ). We note that the 1665 MHz OH line may contribute to the measured line flux and width, although this fainter feature is not yet detected in this single-track observation nor currently preferred over a single-Gaussian fit.
Astronomy Observatory. The ilifu facility is supported by contributions from the  Inter-University Institute  for Data Intensive Astronomy (IDIA, which is a partnership between the  UCT , the  University of Pretoria  and the  University of the Western Cape) , the  Computational Biology division at UCT , and the  Data Intensive Research Initiative of South Africa (DIRISA). Data processing used pipelines that were developed at IDIA and are available at https://idia-pipelines. github. io. M. G. acknowledges support from IDIA and was partially supported by the  Australian Government  through the  Australian Research Council's Discovery Projects funding scheme  ( DP210102103 ). A. K. M. and A. J. B. acknowledge support from  NSF  grant  AST-1814421. A. K. M. also thanks the  LSSTC Data Science Fellowship Program , which is funded by the  LSST Corporation ,  NSF  grant  OAC-1829740 , the  Brinson Foundation , and the  Moore Foundation ; his participation in the program has been helpful for this work.
H. R. and Je. D. acknowledge support from  NSF  grant  AST-1814648 ; M. B. and J. O. acknowledge support from  NSF  grant  AST-1814682 ; R. B. acknowledges support from an  STFC   Ernest Rutherford Fellowship  (grant No. ST/T003596/1 ); Ja. D. and H. P. acknowledge the financial assistance of SARAO; K. M. H. acknowledges funding from the  State Agency for Research of the Spanish Ministry of Science, Innovation and Universities through the "Center of Excellence Severo Ochoa"  awarded to the  Instituto de Astrofísica de Andalucía  ( SEV-2017-0709 ); from grant  RTI2018-096228-B-C31  ( Ministry of Science, Innovation and Universities/State Agency for Research/ European Regional Development Funds, European Union ); and from the coordination of the participation in SKA-SPAIN, funded by the  Ministry of Science and innovation (MICIN) ; and Z.
Recently, Deep Neural Networks (DNNs)   have become a central subject of discussion in computer vision for a broad range of applications, including image classification and face recognition. Compared to existing conventional machine learning methods, deep learning algorithms have shown prominent performance with high accuracy and speed. However, they always require a large amount of data to achieve adequate robustness. Furthermore, additional samples are timeconsuming and expensive to collect. In this paper, we propose an approach that combines generative methods and basic manipulations for image data augmentations and the FaceNet model with Support Vector Machine (SVM) for face recognition. To do so, the images were first preprocessed by a Deep Convolutional Generative Adversarial Net (DCGAN) to generate samples having realistic properties inseparable from those of the original datasets.
Second, basic manipulations were applied on the images produced by DCGAN in order to increase the amount of training data. Finally, FaceNet was employed as a face recognition model. FaceNet detects faces using MTCNN, 128-D face embedding is computed to quantify each face, and an SVM was used on top of the embeddings for classification. Experiments carried out on the LFW and VGG image databases and ChokePoint video database demonstrate that the combination of basic and generative methods for augmentation boosted face recognition performance, leading to better recognition results.
Introduction Despite the exceptional efficiency of 2D and 3D recognition, face recognition based on DNNs has faced several challenges, such as the difficulty of collecting enough training images, because DNNs often require a large amount of data for effective learning. Generally, a large volume of data is useful to achieve high recognition accuracy. Because DNNs have powerful learning ability, they need various views of the face for each subject. However, obtaining such a dataset for a one class is not only impractical, it is very time-consuming. An insufficient amount of samples leads to over-parameterization and over-fitting issues, resulting in an obvious decline in the effectiveness of the learning outcomes. Moreover, it is often necessary to train samples of faces in different conditions of illumination, facial expressions, poses, and occlusion.
To deal with the issue of scarcity or insufficiency of samples, an efficient solution is to use data augmentation techniques. The main purpose of data augmentation is to expand the diversity and size of the training database and expose the model to various aspects of data in order to guarantee that the model is evaluated on images that are not seen twice during training, achieving greater robustness, higher accuracy, and stable classification performance. However, there is a gap in the literature concerning research on the methodological and theoretical augmentation techniques for face recognition. Furthermore, there is a lack of studies on how to increase the size of small datasets using augmentation. As a result, in the current work we focus on filling this gap. We propose a new data augmentation method and compare the effect of using various data augmentation techniques on face recognition accuracy.
Specifically, we consider geometric transformations, image brightness changes, and application of different filter operations, as well as using DCGAN to generate new images from the original dataset to augment the amount of training samples. A number of studies using deep learning methods have claimed high performance in a significant number of tasks. These include image classification  [1] , natural language processing  [2] , and text classification  [3]. These models use the Softmax function in the classification layer. However, there have been studies  [4, 5]  conducted that consider SVM as an alternative to the Softmax function for classification. These studies have asserted that the use of SVM in an artificial neural network (ANN) instead of the Softmax function may improve recognition accuracy.
In our work, the augmented training data are sent to FaceNet to extract the embedding and then classified using an SVM. The present work is an extension of our preliminary work published at the ISVC conference in 2020  [6]. The main contributions of this paper can be summarized as follows: • We propose a novel data augmentation technique in which DCGAN and basic manipulations are combined. We use the Wasserstein loss to replace the standard DCGAN cross-entropy loss to solve the problem of DCGAN training instability. We show that our model improves face recognition performance by considering the LFW dataset  [7] , VGGFace2 dataset  [8] , and ChokePoint video database  [9].
Li et al. [19]  introduced a data augmentation technique, dubbed Moment Exchange (MoEx), that encourages the model to use the moment information of latent features. More precisely, the moments of the learned features of one training image are exchanged for those of another. This approach effectively improves classification accuracy and robustness across several datasets, model architectures, and prediction tasks. Despite the easy implementation of these methods, they remain the subject of research due to the increased training time and higher memory use they require. Generative models are able to generate new training data, resulting in better performance by classification models. Generative approaches include methods such as Neural Style Transfer (NST) and Generative Adversarial Networks (GAN).
NST  [20]  synthesizes a new image by manipulating the sequential representations across a CNN such that the style of one image can be transferred onto another while maintaining its original content. While this facilitates the transfer of textures, color temperatures, and lighting conditions, it can lead to bias in the data. Furthermore, NST algorithms  [21]  are slow. GANs create samples similar to images from the original dataset. Data augmentation based on GANs can enhance model performance even though the generated images do not appear hyperrealistic. However, at present the training of GANs faces many challenges, such as their instability and need for massive amounts of training data. Bowles et al. [22]  used GANs as a way to unlock additional information from a dataset. Yi et al. [23]  used a conditional generative adversarial network (cGAN) to expand the training dataset by generating emotional face images.
We compare the use of only basic image manipulations methods, only generative methods, and the combination of these two techniques. As a generative approach, we used a DCGAN to generate synthetic image data. This is a generative approach that shows remarkable performance in generating different face attributes such as glasses, different colors of hair, and smiles. The main idea behind our proposed data augmentation method is to integrate strategies. We applied traditional manipulations (filtering, brightness changes, and geometric transformations) to images produced by the generative approach (DCGAN) in order to simulate the changes that can occur with a face. We then added these images to the original training dataset as augmented data. Sometimes, only a small amount of samples is required to train a CNN; however, with more images, the CNN has better performance.
In 2016, Khoi et al. [30]  proposed to evaluate two variants of LBP, namely, Rotation-Invariant Local Binary Pattern (RILBP) and Pyramid of Local Binary Pattern (PLBP), for face retrieval. They split the facial image into small regions using the Grid LBP technique then constructed a histogram of spatially enhanced features based on the LBP feature vectors. Their system was robust against increases in the size of the dataset, with no sudden fall in mean average precision (MAP). A local appearancebased method called LBP network (LBPNet) has been proposed in  [31]  which effectively contributes to the extraction of hierarchical data representations. Laure et al. [32]  used robust LBP to extract facial features to address the problem of large variations in lighting, expressions, and poses. KNN was applied for classification.
[50]  proposed to use a deep network to recover the canonical frontal view. Each face was categorized as corresponding to a known identity by training a CNN. PCA was used with SVM on the network output for face verification. Taigman et al. [51]  proposed to train a multi-class network on about four thousand identities in order to recognize faces. They employed a Siamese network to optimize the L1 distance between two face features. Their high accuracy obtained on LFW dataset  [7]  was the result of an ensemble of three networks using various color channels and alignments. The VGGNet-16 network presented by Simonyan et al. [52]  achieved an accuracy of 98. 95% using 2. 6 million samples. Compared to DeepFace  [51] , which employs a single large CNN to learn features, Sun et al. [53]  built their DeepID model for face representation to learn features from multiple CNNs using the network fusion technique.
DeepID extracts features from images with various facial poses. Then, Sun et al. [54]  developed DeepID2, which is an extension of DeepID. They employed a set of 25 nets, each taking a particular face patch as input. Then, 50 responses were combined to assess their performance under the LFW dataset  [7]. A Joint Bayesian model  [55]  that usefully corresponds to a linear transformation in the embedding space has been proposed as well. To train the networks, the verification and classification losses are combined. The verification loss has the same concept as the triplet loss  [44] , as it aims to decrease the L2-distance between pairs of faces belonging to the same identity and enlarge the distance between faces of various identities. The verification loss is similar to that applied in Wang et al.
Data Augmentation Using DCGAN Combined with Basic Manipulations After applying face alignment and cropping, the extracted faces were passed through a DCGAN to synthesize new images. Basic manipulations are characterized by their simplicity and ease of implementation, however, they cannot reveal realistic face variations. While the generative approach can generate significant realistic face variations, it uses additional resources. To alleviate this issue, we trained the DCGAN  [59]  using the images from each class of the original dataset in order to learn the features of the faces and then artificially synthesize more facial images. We exploited the capacity of the DCGAN generator to synthesize more faces similar to the original faces in the training dataset, the idea being to simultaneously train two adversarial neural networks: the discriminator tries to discern whether the sample comes from the actual data distribution, while the generator aims to trick the discriminator by generating better samples.
In the DCGAN training process, the discriminator aims to differentiate real samples from G(z), i. e. , log(D(x)) + log(1 -D(z)) should be maximized. Here, G(z) represents the generator, which takes as input a random noise z sampled from a uniform distribution, while D(. ) represents the discriminator, which takes x as input, such as images, from the selected database or the output of G(z). At the same time, the generator aims to trick the discriminator by minimizing log(1 -D(z)). A stable point in training is achieved when, after multiple steps, the discriminator is unable to differentiate between x and G(z). The generator takes a random noise vector as input sampled from a uniform distribution between (-1, 1), followed by a fully connected layer containing 8192 neurons and resized to the dimension of 4 × 4 × 1024, then four transposed convolutional layers with a stride of 2 and padding, which results in channel reduction and an upsampling of the features by factor of two.
Results and Discussions In this paper, we propose to combine DCGANs and basic image manipulations as data augmentation techniques for our face recognition approach based on FaceNet + SVM. We evaluated the proposed method on two face image datasets, the LFW dataset  [7]  and VGGFace2 dataset  [8] , as well as on a video face dataset, the ChokePoint dataset  [9]. We proposed to add 100, 250 and 500 generated images per one class. We compared our proposed approach with three popular face recognition algorithms: Principal Component Analysis (PCA), Tensor Robust Principal Component Analysis (TRPCA)  [38] , and Local Binary Pattern Histograms (LBPH). PCA is often employed for dimensionality reduction. It keeps only the values which contribute the most to variance in order to compress the dataset.
It decomposes the covariance matrix to obtain the principal components (i. e. , the eigenvectors) of the data and their corresponding eigenvalues. TRPCA  [38]  plays a critical role in handling high multi-dimensional datasets, aiming to recover the low-ranked and sparse components both accurately and efficiently. The LBPH method is based on the LBP, which is considered a texture description method. Using a face image, the histogram features are extracted from the occurrences of the LBP codes for texture categorization. Then, classification is carried out by finding the similarity between histograms. Additionally, we compared our approach with the work of Pei et al. [63] , which was based on standard data augmentation techniques, including filtering, geometric transformations (rotation, translation,.
), and brightness changes along with a VGG-16 network for face classification. Furthermore, we compared our work with our own previous work  [6]  based on DCGANs for data augmentation and FaceNet + SVM for face recognition. In our experiments, we began by identifying faces in images, then moved on to identifying faces in videos. The LFW dataset is the standard dataset for face verification and recognition. This dataset contains 13,233 facial images of 5749 subjects. It includes several challenges, such as varying face poses, expressions, and illumination conditions as well as partial occlusion. In this dataset, only 1680 subjects out of a total of 5749 identities have more than one facial image. A subset of the database containing 3137 pictures obtained from 62 identities was employed in our experiments by choosing only those subjects with 20 or more images  [7].