dataesr/scientific-paragraphs-categorization huggingface dataset:
Classification Models: k-NN, SVM, and ANN After data augmentation and exploration of the feature set and labelling, the datasets were used with three different machine learning models from the Python Scikit-Learn libraries, run on a Windows 7 computer with core i7 2. 6 GHz processor. A brief description of the models is given below. (a) K-nearest neighbors (k-NN): In this, the underlying assumption is that samples of the same class will be the nearest neighbors of each other, i. e. , the distance between them will be small since they are related  [39, 40]. While using this method, the training samples form vectors in a multidimensional feature space, each with its class label. In the training phase of the algorithm, the feature vectors are stored along with class labels of the training samples.
In the classification phase, the distance between an unlabelled input vector and the training dataset is calculated to get the k (a user defined hyperparameter number) nearest points of the input sample. Then, the input sample is categorized into the class of the majority in the k nearest points. Thus, if the training data is {(x1,,y1), (x2,,y2),…, (xN,yN)} and x is the feature vector of an input sample, the KNN finds the class of the input sample x using a distance metric  [41, 42] , which in this study is the Euclidean distance given by: Dist (xi,xj) =. ( 17 ) (b) Support vector machines (SVMs): It is a supervised learning method  [43, 44]  that works on the principle of transforming the input feature space by a nonlinear transformation to a high-dimensional feature space, and searching for an optimal separating hyperplane for the input classes in this new high-dimensional space  [45, 46].
KNN, SVM, and MLP-Based Classification Results on Desktop Data In the sub-sections below, the results from training the three ML models with the dataset described in Table  3  are presented, along with statistics of the learning process. The models used here were optimized through experimentation to select the set of hyperparameters that best improve their cross-validation scores. In some cases, the results show underfitting, especially in seven class classification problems, which are inherently complex. For this kind of problem with little separation between classes, model underfitting is a commonly observed issue. The KNN, SVM, and MLP models were used for the classification of gaze datasets corresponding to four different classes of user distances (50, 60, 70, 80 cm) and the results are given in Table  4.
Results, Task II: Classification of Desktop Data for Different Head Poses In this task, the gaze datasets collected from the desktop setup for different user head poses were used. The head pose datasets include the gaze error values for the head roll (20 degrees), pitch (20 degrees), yaw (20 degrees), and neutral (roll, pitch, yaw = 0 degrees). The KNN, SVM, and MLP models were used and the classification accuracies for the different classifiers are tabulated in Table  5. Using the grid search, for KNN, the best number of neighbors was found to be 3 and a reduced feature set was used. For SVM, a C value of 10 and gamma of 1. 25 was used with the reduced feature set. For the MLP model, a three-layer network was used, and its architecture was tuned by varying the number of units in each layer between 50 and 100 and varying the regularization parameter values between 0.
Results, Task III: Classification on Merged User Distance and Head Pose Datasets from Desktop In this task, the head-pose and user distance datasets were merged, and classifiers were applied to do a seven-class classification on this mixed dataset. The classification results are presented in Table  4  and the confusion matrix for the MLP model is shown in Figure  13. Since the head pose data was collected at a user distance of 60 cm, the neutral head pose data and the UD60 data are the same, and therefore only one of these two datasets were used to avoid a class imbalance in the mixed dataset. As above, the three classifiers were trained and tested on this dataset. For the KNN, the best number of neighbors was chosen to be 3 as shown in Figure  14a  below, which shows the dependence of the train, test, and cross-validation error as a function of the number of neighbors used in the model.
For the SVM model, the parameters were the same for the above two tasks. For the MLP, 2 hidden layers with 100 units each and a regularization value of 0. 001 were used with the full set of features. Table  5  shows the performance of the three classifiers on different datasets, with true and false detection rates and precision values. It is observed that for desktop data, the KNN and MLP classification performances are close, with KNN performing well for all the datasets.
KNN, SVM, and MLP-Based Classification Results on Tablet Data Here the results from using the tablet data with various classification models are presented. These include datasets for different user-tablet distances (50, 60, 70, 80 cm) and tablet poses of the neutral, platform roll, pitch, and yaw of 20 degrees. A mixed dataset was created by merging the datasets for the user distance and platform pose, and classifier models were trained to do seven class classifications, i. e. , distinguish between four user distance and three tablet pose classes. 3. 6. 1. Results, Task IV: Classification of Tablet Data for Different User Distances For the user distance dataset from the tablet platform, the KNN classifier with three neighbors was used. The SVM was used with the RBF kernel, C = 10 and gamma of 1, and the MLP classifier was used with 3 layers, 200 units each, and with a regularization value of 0.
0001. The best performance was by MLP and the classification results are shown in Table  6. For the tablet pose dataset, the KNN classifier with three neighbors was used. The SVM was used with RBF kernel, C = 5, and gamma of 0. 5. The MLP model was used with 2 layers, 100 units each, and a regularization value of 0. 0001. The best classification results were by KNN and are shwon in Table  6.
Results, Task VI: Classification of Tablet Data for Mixed User Distance and Tablet Pose Datasets For the mixed user distance and platform pose datasets, the KNN classifier with three neighbors was used. The SVM was used with RBF kernel, C = 5, and gamma of 0. 5 and the MLP classifier was used with 2 layers, 100 units each, and with a regularization value of 0. 0001. Both SVM and MLP performed well and the classification results are shown in Table  6.
Discussion The results demonstrate that ML models can distinguish between gaze data collected under normal and varying operating conditions. Thus, when using these models, anomalies present in gaze datasets may be detected. This is similar to the anomaly detection approaches  [54] [55] [56]  used in fields, such as cyber intrusion and video surveillance, where training sets of normal and nominal examples are used to design a decision rule such that occurrences of erroneous data samples are detected. For the classification tasks, gaze datasets were constructed such that they contained signatures of a single or multiple error sources. It was found through the decision tree-based feature selection technique that statistical attributes, such as gaze error confidence levels and interquartile ranges, are significant parameters that can be used to distinguish gaze error sources.
In the classification tasks, the KNNs and MLP classifiers performed the best for all datasets, although the MLP models take more time to converge. Additionally, it is seen from Tables  4  and 6  that using the gaze yaw and pitch features result in better detection accuracies than the frontal gaze datasets. From Tables  5  and 7 , it is seen that the rate of false detections is also low for all the models. All the datasets, especially the ones created with seven classes, were found to be quite complex as no class separation could be detected from the t-SNE results. Despite this, the classification models achieved a cross-validation score of 85%-90% on most datasets. This shows the feasibility of the ML models in detecting gaze error sources even from complex gaze datasets where more than one source is present.
Modelling and Prediction of Gaze Errors Using the collected desktop and tablet datasets, regression models  [57, 58]  were trained to predict gaze estimation errors using the gaze angle, yaw, and pitch angle values as input features. Two different models were built, which may be used to predict the gaze errors of an eye tracker under two different operating conditions. These include the head-pose error model, which can be used to predict gaze errors occurring due to various user head poses. The other is the platform pose error model, for predicting gaze error corresponding to different poses of the eye tracker. The gaze errors were predicted as a function of a user's gaze angle and gaze yaw/pitch angles (Equation (  28 )). The results from the various gaze error models are presented in sub-sections 4.
( ) 28 For the regression tasks, the input features were standardized, such that their distributions had a mean value of 0 and standard deviation of 1. Six different regression models for predicting the gaze errors were trained on the features created using Equation (  28 ) on the head pose (desktop) and platform pose (tablet) datasets. For each model, the fit estimates in RMSE values are summarized in Table  8. The coefficients of the best performing models are listed in Table  9.
Head Pose Error Model The plots for the actual gaze error values due to the head pose (obtained from desktop setup) and the error values predicted after training by the different models are shown in Figure  15a-d. Table  8  shows that ElasticNet (with penalty parameter = 0. 5) has the lowest prediction errors for all head pose datasets. The regularization used for the Ridge and Lasso models is 0. 001. The MLP model is used with 1 hidden layer with 100 units, ReLU activation, and a regularization value of 0. 001.
Establishment of the Error Models As described above, a regression model with three predictor variables can be expressed as: Y = B0 + B1*X1 + B2*X2 + B3*X3, ( 29 ) where B0 is the intercept; B1, B2, and B3 are coefficients; and X1, X2, and X3 are input features (gaze, yaw, pitch angles). Since the ElasticNet model had the lowest RMSE for predicting the head and tablet pose errors, the B0, B1, B2, and B3 parameters of this model were computed for the head pose and tablet pose datasets and are presented in Table  9. With these parameters, the head and platform pose error models may be constructed for error prediction using gaze angular variables as the input.
MLGaze: An Open Machine Learning Repository for Gaze Error Pattern Recognition In this study, the major aim was to present the research direction on the pattern recognition of gaze errors and to show, as a proof of concept, that machine learning algorithms can be used to distinguish gaze error patterns created due to different operating conditions of an eye tracker. To test and replicate the principles, the full set of gaze data collected for this study was provided in the Mendeley data repository as mentioned in Section 2. 5. 5 above. For any eye tracker other than the Eyetribe, the gaze error characteristics will be different based on the tracking principle and tracker hardware. Therefore, for other eye trackers, the classifier models described in this study have to be re-trained using data from the respective eye trackers if their characteristics vary widely from Eyetribe (which can be determined by comparing the respective eye tracker's data with the data provided in the Mendeley data repository).
To facilitate this and make it possible for researchers to replicate the principles of this study on their own eye tracker's data, the codes for implementing the full data processing pipeline as described in this paper and in Figure  1    There are eight python scripts in the repository, and these are fully documented. The codes in data_augmentation. py and outlier_removal. py may be used for data augmentation and outlier removal as described in Sections 3. 2 and 2. 4, respectively. Code create_features. py is for generating training features, whose output may be used with the codes train_SVM. py, train_KNN. py, and train_MLP. py to train KNN, SVM, and MLP models, respectively, as described in Sections 3. 4 and 3. 5. Code regression_models. py is for running all the regression models described in Section 4.
Conclusions and Future Work In contemporary eye gaze research, there is a scarcity of analytical for studying the variability of gaze accuracy in eye tracking data acquired under unconstrained conditions. There is also a lack of datasets that provide gaze and ground truth information for different operating conditions, and gaze datasets labelled with different experimental scenarios are absent. Therefore, for this study, a new and diverse gaze dataset was collected and labelled with different operating conditions. The dataset named NUIG_EyeGaze01(Labelled eye gaze dataset) comprises of fixation data from 20 participants captured from different user platforms and operating conditions described in this paper. This dataset was then analyzed to identify the presence and to model the impacts of the above conditions on the gaze error levels.
Several new approaches with respect to gaze error pattern analysis were implemented in this study. This includes the use of multiple strategies for outlier removal, and it was found that the median filtering approach, among others, was successful in de-noising gaze data from all the collected datasets. Next, various gaze data augmentation methods were applied, which helped to increase the size of the collected gaze datasets by an order of magnitude and introduce a lot of variabilities within the datasets. The choice of gaze error features and use of a random forest-based feature selection method helped to reveal insights about the gaze error characteristics, e. g. , that the interquartile range and confidence intervals are significant indicators for distinguishing gaze error patterns. The t-SNE algorithm is a relatively new concept in machine learning, which was used in this study to visualize the distribution of classes within the gaze datasets, when gaze data influenced by different error sources were mixed together.
This technique demonstrated the complexities of the gaze datasets used in this study, which are close to what might be expected in gaze data from unconstrained practical applications. Finally, the use of machine learning for classification and prediction of gaze error patterns in artificially created heterogeneous gaze datasets as done in this study is a new concept, which has been sparsely explored before. The results from the training of the ML models reveal that these could be highly robust in identifying gaze error patterns even in complex gaze datasets. It was seen that classifier models can successfully distinguish between eye tracking data collected under normal conditions and data collected under varying operating conditions, such as a high degree of head pose variations, different user distances, and platform poses.
With these classifiers, familiar error patterns in eye gaze datasets may be detected, and gaze error patterns that do not match the normal gaze behavior may be identified and recognized as new error types. The various concepts developed in this study, including the application of classifier and regression models to gaze error data, may be used to build improved eye tracking systems and algorithms and also to obtain better results from them. The main benefit of training the classifier models is that they can distinguish anomalous gaze data present in realistic and complex gaze datasets as used in this study and possibly recognize what caused them (e. g. , impact of user distance or head pose). This can help eye gaze application developers, researchers, or engineers to deploy appropriate prevention methods, compensation strategies, or setup improvements to reduce the impacts of the error sources affecting their data.
The regression models may be used predict how an eye tracker might behave under practical operating conditions like a high degree of head pose or platform pose variations and forecast the possible levels of error caused by different error sources. This can help eye gaze researchers or engineers to quantitatively specify the limits of their systems while operating under these challenging conditions and also develop suitable error correction methods. Additionally, the outlier detection techniques described in this study could be useful for applications to any gaze dataset prior to analyzing them, to observe underlying data patterns. Overall, the gaze data analysis pipeline as implemented in this study could be of use to any eye gaze researcher or engineer who wishes to gain deeper insights into their collected gaze data and have an estimate about the impacts of various non-ideal operating conditions on their eye tracking system.
Therefore, the collection of relevant gaze datasets from complex operational scenarios using desktop, handheld, head mounted, or automotive setups to understand a diverse range of gaze error patterns could be another potential future study in this domain. features were included in the training set, so the total training data size was (20 × 3 × 10) = 600 samples per class. As an example, for the classification of the user distance data from the desktop or tablet, there were four classes and all three categories of gaze data were collected for each of them. So, the total number of training samples was 600 × 4 = 2400 samples for the 4-class classification problem (this is shown in Table  3  above). This feature size was deemed to be useful while training and is supported by the learning curves for the different ML models whose plots are shown in Figure  B1a-d.
for Train, Test Original Feature Set (20 Features) Augmentation Strategies Samples Per Person Samples and Classes features) classes Mean, SD, IQR, 0. 95 interval bounds for each sample. (6) Tablet pose, 2400 samples, 4 classes (7) Tablet mixed: 4200 samples, 7 classes (1) Gaze error values (1) Desktop at 15 AOIs, Mean, (1) Gaussian user dist, SD, IQR, 0. 95 noise 2400 Total participants: 20 12-16 participants for training, 8-5 for test Data labelled and randomly shuffled interval bounds of gaze error (2) Yaw error values at 15 AOIs, Mean, SD, IQR, 0. 95 interval bounds of yaw error (3) Pitch error values at 15 AOIs, Mean, SD, IQR, 0. 95 (2) Jitter or pink noise (3) Horizontal data flipping (4) Vertical data flipping (5) Magnitude warping (6) Time warp (7) Interpolation 10 x gaze error 10 x yaw error 10 x pitch error Merged dataset 30 samples samples, 4 classes (2) Head pose, 2400 samples, 4 classes (3) Desktop mixed: 4200 samples, 7 classes interval limits of and (5) Tablet user pitch error combinations dist, 2400 Reduced feature set (5 samples, 4.
user Distance For Head Pose On Mixed Datasets Dataset k-NN SVM MLP k-NN SVM MLP k-NN SVM MLP Only gaze angle features 89% 69% 86% 91% 83% 90% 87% 74% 83% Only gaze yaw features 83% 64% 84% 88% 78% 86% 81% 66% 80% Only gaze pitch features 96% 89% 73% 95% 96% 83% 92% 91% 79% Merged (gaze, yaw, pitch) feature dataset 84% 67% 81% 89% 80% 84% 85% 70% 77%.
INTRODUCTION The rise of MOOCs has widened the educational landscape with new opportunities. One original promise of MOOCs was to provide high quality, free educational resources around the world, especially to those learners lacking ready, affordable access to higher education  [6]. However, many studies have reported that most MOOC learners are already educated and from affluent countries  [11]. Most of these studies have focused on global MOOC providers (such as edX, FutureLearn or Coursera), where Anglo-American higher education universities teach courses primarily in English. However, very few studies have delved into differences with local or regional MOOC providers, that center their attention on a local or regional population. Many institutions and national MOOC initiatives are using the open source software Open edX, which provides an easy way to publish their courses.
As of 2018, there are over 800 organizations, institutions, and governments which have been running instances of Open edX  [3]. There are numerous studies that have discussed the impact of language and culture in learning  [8] , and previous researchers have linked the country of origin of MOOC participants to different behavioral patterns in the course  [10]  or to social identity threat in less developed countries  [9]. Previous work that compared Arab learners in both Edraak (an Arabic MOOC provider) and edX found that learner populations in Edraak had a wider range of education levels and a more even gender, and the courses showed more favorable completion trends  [12]. This previous work suggested that regional MOOC providers might be better positioned to fulfil their learners' needs as they offer courses in their local language, and taught by instructors of similar culture and background.
METHODOLOGY 2. 1 Multiplatform MOOC Analytics We describe the process that was followed to conduct this research now. First, the project lead launched an initial call looking for partners with access to large MOOC datasets from different platforms with the objective of running a comparative study on global and regional trends. Once the partnership was settled, we followed the next steps to conduct the research: (1) Partners shape their data into the same common format. (2) The project lead generates a Jupyter notebook that is expecting the common data format established in the previous step. This script outputs aggregate data from different institutions that is merged together for the joint analysis. (3) We conduct the joint data analysis of all providers together and iterate over these three steps as required.
Benchmarking is likely a useful outcome from this researchwhere providers and course teams can compare their demographics against these published datasets with the normalised set of figures. However, it is anticipated that this research will start to unlock new understanding on regional and global online learning. We can see from this study there is an impact of the locality of the platform. Platforms have very different catchment areas for their courses, with varying levels of concentration. This concentration of home country participation ranges from as high as 98% for XuetangX, to 30% for the global providers. Exploring the reasons for this would help understand when providers differ between a   local or global focus or either want to shift from being local to becoming more international or to hone in on a specific region or demographic in reach or appeal.
Gender balance is one indicator of how each platform has managed to attract different audiences. Overall, participation by gender is imbalanced, with average of 63% of learners identifying as male across all platforms. On some platforms, notably openSAP, openHPI and ChineseMOOC, this imbalance is significantly larger with an average 79% male learners. FutureLearn ranks as the platform with the largest percentage of learners who identify as female and OpenWHO, UPVx and Edraak have notably better gender balanced demographics. The analysis does not intentionally exclude learners who identify as fluid or nonbinary -these data are not available in sufficient quantities from enough providers. Further studies would benefit from providers being able to collect a wider dataset in this area and for caution in analysis when using traditional binary classification.