dataesr/scientific-paragraphs-categorization huggingface dataset:
2. Observe that several spurious local maxima appear when reducing the sample size. We conclude that it is convenient to carry out L1-PCA using globally convergent algorithms such as that of  [16]  if the sample size is small. The bit flipping algorithm of  [17] , though suboptimal, achieves good convergence in all our experiments. 2) Comparison with LDA and L2-PCA: Since L1-PCA is related with L2-PCA (Section II-C), one may naturally wonder about their comparative behavior. To this end, Fig. 3  shows the scatter plot of the T = 200 data points used in the previous experiment, where data from class 1 and class 2 appear as red crosses and blue circles, respectively. The red line marked 'L1' is the direction of the optimal L1-uLDA projection. Projecting the whole dataset onto this line gives the histogram plotted in Fig.
4 , clearly showing the clustered structure of the data. Similarly, the black line labelled as 'PCA' in Fig. 3  is the direction of maximum Red line: direction of the optimal projection according to the proposed criterion (L1-uLDA). Green line: optimal Fisher's discriminant direction (LDA). Black line: direction of maximum variance (L2-PCA). variance of the data, constructed by classical L2-PCA, whereas the green line marked 'LDA' represents Fisher's discriminant direction for this dataset. Even though the three approaches present a similar behavior in this example, strictly speaking classical L2-norm based PCA is not a linear discrimination technique and, therefore, it may yield different results in certain cases. Supporting this claim, Fig. 5  shows the projection directions obtained after repeating the experiment under the same conditions except for the covariance matrices, which are set to V 1 = V 2 = diag(1, 3), so that the direction of maximum variance lies now along the x 2 -axis.
B. Experiments With Real Data The theoretical analysis of Section III assumes that the density is Gaussian for both classes. This section presents three additional experiments to test the behavior of the proposed method when that assumption is violated, as is often the case when analyzing real data. To this end, we use three classical datasets with a deliberately small number of samples so that we can also test the impact of short sample length. 1) Iris Dataset: Originally used by Fisher when introducing LDA  [35] , this is perhaps the most famous dataset in the pattern recognition literature. It contains the sepal and petal length and width of 150 iris flowers from three different species (setosa, versicolor and virginica). One of them (setosa) is linearly separable from the other two; the latter are not linearly separable from each other.
The dataset contains 50 samples from each of the three species of iris. It can be downloaded from the UCI Machine Learning Repository  [36] ,  [37]. The database can be written as a matrix X ∈ R 4×150 , corresponding to 4 measurements from 150 flowers. After centering and sphering the data, we calculated the unit-norm L1-uLDA projection vector a L1 ∈ R 4 of the data matrix. Fig. 8  plots the probability density function, obtained by a kernel density estimator method, of the projection of the data onto the direction of a L1. The distribution is clearly bimodal, which suggests that the sample is not homogeneous but arises from at least two different populations  [22]. In fact, the mode on the left exclusively corresponded to the setosa individuals, while the other mode represented the mixture of the other two classes.
Next, we separate the projected data into two groups using as threshold the local minimum between the two peaks in the Fig. 9. Experiment on the Iris dataset. Histogram of the projection of the data containing only versicolor and virginica individuals (corresponding to the right-hand mode of Fig. 8 ) onto the L1-uLDA optimal direction. bimodal distribution. It is thus obtained one cluster that only contains iris setosa and another cluster that only contains versicolor and virginica. Then we repeat the procedure (centering, sphering and L1-PCA) on the cluster with the versicolor and virginica individuals. A bimodal distribution is again apparent as a result (see Fig. 9 ). Dividing these data once more into two groups, using zero as threshold for deciding to which class each flower is allocated, only three flowers are misclassified: one virginica is misclassified as a versicolor and two versicolor are misclassified as virginica.
This 98% accuracy is achieved by L1-uLDA in unsupervised operation, without knowledge of the sample labels. 2) Wisconsin Diagnostic Breast Cancer Dataset: This dataset is used for breast cancer diagnosis. It contains 569 instances of 30 real-valued features from images of a fine needle aspirate of a breast mass, that describe the characteristics of the cell nuclei present in the image. Of these 569 instances, 357 Fig. 10. Scatter plot of the three first principal components of the Wisconsin Diagnostic Breast Cancer Dataset  [38]. The plane represents the decision boundary that partitions the space into positive and negative projections onto the L1-uLDA optimal direction, which is plotted as a black line. correspond to 'benign' (non-cancerous) cases while 212 are 'malignant' (cancerous). The dataset is downloaded from the UCI Machine Learning Repository  [36] ,  [38].
This experiment uses the AT&T face dataset, which contains 10 facial images of each of 40 different subjects (400 images in total). For a given subject, the facial images differ in lighting, pose, expression (open / closed eyes, smiling / not smiling) and details (glasses / no glasses). Each image is 92 × 112 pixels, with 256 grey levels per pixel. Fig. 11  shows some of the images of the database. Fig. 12  shows the recognition rate with 10-fold crossvalidation versus the dimension of the image projection subspace, where we use 90% of the images for training and 10% for validation, with a slightly superior performance of the L2-PCA based method. We repeat the experiment but, to generate data far from the clusters, 10 images of the training dataset are severely distorted by salt-and-pepper noise in this second case.
The result, shown in Fig. 13 , illustrates again the robustness of L1-principal subspaces against outliers. L1-uLDA reaches a recognition performance up to the mark of LDA's but with no need for training data. recognition. In these last experiments, we test the capabilities of L1-uLDA for dimensionality reduction when applied to the output of a ResNet network  [41] , which has been implemented in the dlib C++ library  [42] , with 29 convolutional layers. This CNN transforms the image of a human face into a 128 dimensional vector, where images of the same person are mapped close to each other and images from different people become separated far apart after the transformation. The experiments are conducted on images from the database "Faces in the wild"  [43] , which comprises a total of 13 233 labelled images of faces from 5 749 persons, albeit only the 3 023 photographs of the 65 people with more than 20 images in the dataset are used in our experiments.
review of this manuscript and approving it for publication was  Prof. Yuantao Gu. This work was supported by the  Spanish Ministry of Economy and Competitiveness  under project  TEC2017-82807-P. The work of  R. Martín-Clemente  was supported by professorship at the  I3S Laboratory  in July 2018 funded by an  IFA  scholarship from  Université Côte d'Azur. (-Clemente. ).
These aspects cannot be understood by looking at raw gaze data, which is almost always corrupted with noise and outliers, or even by studying mean gaze error values. With respect to gaze error analysis, several questions arise: These include: (1) how can gaze errors caused by one error source be distinguished from those caused by another? (2)  How can the presence of different error sources in a certain gaze dataset be detected without prior knowledge? (3) Is it possible to predict the level of gaze errors that have been caused by different error sources? (4) Can suitable features be extracted from gaze datasets to identify the different error sources. These questions form the main topics that are addressed in this paper. This paper focuses on defining methods for detailed analysis of eye tracking data obtained from a generic commercial eye tracker for the detection, identification, and prediction of gaze errors.
The aim of this study was to observe gaze error patterns produced by three error sources that commonly affect eye trackers in consumer platforms like desktop and tablets. These include head movement, user distance, and platform orientation. It was found that there currently exists no publicly available gaze dataset that contains raw gaze and ground truth (gaze target location) data along with signatures of these error sources. Therefore, a new eye tracking dataset was built by collecting gaze data from 20 participants using a commercial eye tracker on both a desktop and a tablet platform. This dataset was published in an open repository for use by the gaze research community (link to the dataset webpage is in Section 2. 5. 5). During data collection, variations in head pose, user distance, and platform orientations were introduced sequentially and in a calibrated manner so that the gaze data contains the influence of one known condition (or error source) at a time.
Reference data, which does not have the influence of any of these conditions, was collected as well. For the detection of gaze error patterns caused by the different error sources, the collected data was used for training machine learning (ML) algorithms by creating training features from the datasets. As the operating conditions change, the feature variables in the training data get affected and as a result the ML models learn to differentiate between error patterns caused by different sources. In this way, anomalous gaze data may be distinguished using classifiers, which were trained using both affected and reference gaze datasets. Finally, regression algorithms were built to model and predict gaze errors that may be produced by the above three error sources.
Gaze patterns were used with clustering and classification algorithms to predict the user's intention to perform a set of tasks in  [6]. A special de-noising and segmentation algorithm based on naive segmented linear regression and hidden Markov models was developed in  [7]  to classify gaze features into fixations, smooth pursuits, saccades, and post-saccadic oscillations using good-quality as well as noisy data. Automated classification of fixations and saccades was achieved using random forest classifiers on high-quality as well as noisy gaze datasets in  [8]. It may be observed that gaze or eye movement pattern modelling have mostly been applied towards either cognitive studies, i. e. , to interpret viewing patterns or distinguish between oculomotor event types, e. g. , saccades, fixations, and smooth pursuits.
( 3 ) The ground truth (θgt) gaze angles for the AOI locations with coordinates (AOI_X, AOI_Y) were: ,. The gaze yaw and pitch angles were derived as follows: Gaze pitch(θpitch) = tan -1 (GazeY/Z), Gaze yaw (θyaw) = tan -1 (GazeX/Z). ( ) 5 The ground truth pitch and yaw angular values for each AOI dot with screen coordinates (AOI_X, AOI_Y) were given by: AOI pitch = tan -1 (AOIy/Z), AOI yaw = tan -1 (AOIx/Z). ( 6 ) These gaze variables along with their statistics were later used to construct the feature vectors for the learning algorithms. The plots of the gaze frontal and rotational angles are shown below in Figure  4. Throughout this paper, the terminologies gaze angle, gaze yaw, and gaze pitch angle will be used to indicate the variables defined by Equations (  3 )-(  6   The common problems associated with analyzing the collected gaze dataset were: (a) The presence of outliers, (b) missing or null values, and (c) unequal data row lengths.
(d) Inter-quartile range(IQR): The concepts of using the Z-score and inter-quartile range (IQR) to study outliers were discussed in our previous study  [1]. A data point is denoted as an outlier if the value for the point is 1. 5⋅IQR above the third quartile or below the first quartile of the data. The results from the three outlier removal methods are shown in Figure  5a-c  and it is seen from the figures that median filtering (with a kernel size =41, which is the mean number of data points around each AOI) worked best for all of the datasets. 2 2 _ ( _ ) ( _ ) OSD GT AOI X AOI Y    1 ( ) tan ( _ / ) GT gt OSD GT Z    (a) (b) (c).
Exploratory data analysis and visualizations Since the eye tracking datasets were collected under unconstrained conditions and the nature of such data or its distributions were unknown, data exploration was essential to observe any underlying patterns before proceeding to the machine learning step. This was done after the data cleaning steps described above as the presence of outliers and null values made it impossible to study any meaningful patterns prior to data preprocessing. After null and outlier removal, gaze angular errors for the frontal and rotational components were computed in absolute units (degrees) as the angular deviation between ground truth and estimated gaze angular values using Equations (  3 )-(  6 ), as: (Gaze frontal angular error)i = (θgaze)i -(θgt)i, ( 8 ) (Gaze yaw angle error)i = (θyaw)i -(AOI_Yaw)i, (9) (Gaze pitch angle error)i = (θgaze )i -(AOI_Pitch)i.
Additionally, the error levels due to various head poses were quite high compared to when the head pose was neutral (UD60 values in Table  1 ). All values in the tables units of degrees of angular resolution. Table  2  shows the statistical properties of the gaze errors for different experiments on the tablet platform, including variations in the user distance and tablet pose (here the roll, pitch, and yaw represent tablet poses). The magnitudes of errors due to tablet pose changes were high and the highest error was caused due to platform roll variations. It was also seen that the error characteristics of tablet data are quite different than those from the desktop platform. Compared to desktop data, the error magnitudes were lower for the tablet for all user distances. However, the magnitude of errors produced due to different platform poses was higher compared to errors induced due to the head pose.
The one-dimensional distributions of the angular error values for different gaze datasets were studied using the kernel density estimate (KDE)  [16]. Since the exact distributions of the gaze errors were unknown, the kernel density estimation was useful since it is a non-parametric way to approximate the probability density function of the data, compared to parametric estimation, where a fixed functional form and its parameters are required to fit the data. For data with samples x(i), using a kernel function (K) and bandwidth h, the probability density at a point x is: KDE=. ( 11 ) The Gaussian kernel K is given by:. Using the data collected from the desktop and tablet experiments, the KDE plots of gaze errors for three different user distances were plotted (bandwidth = 0. 2) in Figures  6  and 7  above.
One significant aspect noted is that the gaze errors from the tablet are much lower than the errors from the desktop for the same user distances. This could indicate that the distinguishing aspect between the two platforms, which is the display size and resolution, could be a factor determining the error levels for an eye tracker. However, further comparison of the data from the two platforms was not done in this study. Other takeaways from this section include the identification of robust outlier removal methods for gaze data and observations of the gaze error distributions, which were heavily affected by operating conditions but were not observable in the raw gaze data plots. Additionally, studies on the correlation of different eye tracking datasets (Figure  9 ) revealed an important aspect: Under different operating conditions, an eye tracker's data may behave in totally independent ways, which are not related to the tracker's data characteristics under stable conditions.
Objectives and Task Definition for Classification of Desktop and Tablet Data In this section, the main goal was to identify a certain error source or operating condition solely from the output data from an eye tracker, which was influenced by the condition. For this purpose, multiple classifier models were trained using data collected from the different eye tracking experiments, which were seen to produce different error patterns. The objective was to see if machine learning models  [17, 18]  could learn to distinguish between these gaze error patterns as they appear among a mix of data captured under different operating conditions. The following classification tasks were performed using the desktop datasets: (1) Classification of errors for different user distances (i. e. , between four classes of data from a user distance of 50-, 60-, 70-, 80-cm datasets), (2) classification of errors for different head poses (i.
e. , between four classes: neutral pose, roll (20 degrees), pitch (20 degrees), and yaw (20 degrees) datasets), and (3) classification between the head pose and user distance errors patterns (i. e. , between four user distance classes, and three head pose classes, with a total of seven classes). With data from the tablet experiments, the following classification tasks were implemented: (1) Classification of errors for different user distances (i. e. , between four classes of user distances data), (2) error classification for different tablet orientation poses (i. e. , between four classes of data from the neutral tablet pose, roll (20 degrees), pitch (20 degrees), and yaw (20 degrees) datasets), and (3) classification between the tablet orientation and user distance error patterns (i. e. , between four user distance classes, and three tablet pose classes, with a total of seven classes).
Before implementing the classification algorithms on both the desktop or tablet data, the user distance, head-pose, and tablet orientation datasets were augmented to increase the number of training samples  [19, 20]. After this, training features were constructed and formatted before being input into the models. Same data augmentation strategies were used for both the desktop and tablet data. For the training and testing, the gaze angle, yaw, and pitch feature datasets were created and used. The classification results from the desktop data are provided in Section 3. 3 while that from the the tablet datasets are in Section 3. 4.
Data Augmentation Strategies With our dataset size (20 persons x 4 operating conditions each for the user distance and head pose), in order to use a sufficient number of features for classification without facing the overfitting problem, augmentation of the dataset was essential. In this study, 10-fold augmentation strategies were used on the gaze angle, yaw, and pitch error datasets estimated from raw data using Equations (  8 )-  (10). The methods used for data augmentation were as follows: 1) The addition of Gaussian noise: Gaze error magnitudes at all data points were perturbed with Gaussian noise with 0 mean and 0. 2 sigma  [21] ; (2) addition of jitter: Human eye jitter was modelled as pink noise  [22, 23] , in which the power spectral density is inversely proportional to the frequency of the signal, given by PSD= 1/f α.
The jitter signal was simulated by first generating white noise with the mean, sigma (0, 0. 2), and successively applying a pink noise filter with the parameters α = 0. 8 at a frequency of 2Hz and added to the error datasets; (3) interpolation: Linear interpolation on the input data points was used to produce variants of the original gaze error samples  [24] ;(4) convolution: The error signals were convolved with a raised cosine kernel with a window size of N = 30 with the form of Equation (  13 ) below to produce a smoothed variant of the original data  [25] :. ( 13 ) The convolution operation was given by: , (5) time shifting: The gaze data were shifted by 10 samples to estimate new variants in error values from the shifted datasets  [26] ;  (6)  combinations: Combinations of the addition of different noise patterns to interpolated signal were used to augment the dataset; and (7) flipping: Horizontal and vertical flipping of error magnitudes at the different AOIs were used to augment the dataset as well as to remove any viewer bias towards the top, bottom, and side locations of the screen.
In horizontal flipping, the error magnitudes of the top AOI-1 to AOI 5 (Figure  2b ) were replaced by the values of 2 ( ) 1/ 2 1 cos( ) 1 n w n N           ( * )( ) ( ) ( ) f g t f t g        the bottom AOIs (AOI-11 to AOI 15). In vertical flipping, the error magnitudes of the left AOIs (AOI No. 1,6,11) were swapped with the right AOIs, (No. 5,10,15). Figure  10  shows how the samples from a dataset were modified by these augmentation methods. Table  3  shows how training samples were created from the collected dataset. Figure  10. Shows a sample from a dataset after applying each augmentation strategy. The blue lines represent samples from the original data, while the red lines are from the augmented dataset after applying each strategy.
(15)  The above feature set was calculated from the gaze angle as well as the yaw and pitch angle data for each sample. To train the machine learning models, the features from all datasets were standardized so that they had a zero mean and unit variance and were shuffled randomly before splitting the datasets into the train and test sections. The proportion of train vs. test samples was varied between 0. 4 to 0. 25. Table  3  below describes the contents of all the training and test datasets used in this study. For visualization of the high-dimensional feature set, the t-distributed stochastic neighbor embedding (t-SNE) method  [31]  was used, which maps data points xi in the high-dimensional feature space R D (D is the dimension of the feature set, D=20 here) to points yi in a lower d-dimensional space (R d , here d= 2) by finding similarities (pji) between the data and learning the corresponding low dimensional mapping points y1, …, yN (with yi ϵR d that reflects these similarities as best as possible  [32] [33] [34].
The pairwise similarity (pji) between points xi and xj is: pji =. ( 16 ) To observe the training data in the feature space, the t-SNE algorithm (with n_components = 2, perplexity = 80) was applied on the feature sets computed from the user distance, head pose, and platform pose datasets, as plotted in Figure  11 (a) and Figure  11 (b) for the desktop and tablet data, respectively. On the datasets from both the desktop and tablet, the t-SNE points look scattered with exp( /2 exp( /2 i j i i k i k i x x x x              no structure. For the head pose data and platform pose data and mixed datasets, the t-SNE plots show several clusters, but within them, the class labels were found to be highly mixed. This reflects the high degree of complexity of the datasets used in this study since the different classes within them do not form any observable clusters; they are neither symmetrically distributed nor have any clear separation between them.
This is close to real world gaze datasets in which normal gaze data is most often mixed with anomalous data due to unpredictable operating conditions. For feature selection, the relative importance of features (Equation (  15 )) were estimated using a random forest classifier method  [35] [36] [37]. This model comprises of a number of decision trees in which it can be computed how much each feature reduces the weighted impurity  [38]  in the tree. The total impurity decrease due to each feature was averaged to rank the feature's importance. Features computed on desktop and tablet datasets were fed to a random tree classifier model with hyperparameters (n_estimators = 200, max_depth = 8). The rankings of features for the desktop and tablet datasets are shown in Figure  12a, b. In all the datasets, the mean, standard deviation, IQR, and confidence intervals (feature numbers 16-20) emerged as the most significant features.