dataesr/scientific-paragraphs-categorization huggingface dataset:
Second, we switch the datasets and train on Galileo to predict dropouts on Curtin. We denote these experiments as Curtin on Galileo and Galileo on Curtin respectively. Third, we use the MIX dataset, in which the training and test sets include examples from both systems. Finally, we determine the importance scores for our features from the Boosted Decision Trees to identify the predictive power of each feature for the detection of dropouts. Universidad Galileo. As shown in Fig. 2 (a) and (b), for MOOCs of Universidad Galileo, we see that not always increasing the number of considered interactions and days guarantee higher accuracy. Firstly, there is a set of MOOCs plotted in green, for which the accuracy increases over time or, after an initial growth, stabilizes. The second group is plotted in red and consists of MOOCs for which the accuracy trend is less steady.
For Curtin on Galileo the accuracy remains almost unaltered. Generally, we obtain better results with the absolute number of interactions approach. The difference in the accuracy is particularly marked for the MIX dataset. Findings. For self-paced MOOCs, such as those from Curtin University, a small number of initial interactions contains already valuable information for a correct classification of the users. Particularly, given the high details of the logs, the number of interactions with each tool is a strong indicator whether users will drop out or not. Due to users' enrollment actions, the first 7 days from users initial interaction reveals to be a less effective approach for self-paced MOOCs. For fixed schedule MOOCs, such as those from Universidad Galileo, both metrics are less accurate.
9, which steadily increases when more days are considered. The multi-system experiments, using Boosted Decision Trees, also benefit of the removal of Course Enrollment actions. Galileo on Curtin and Curtin on Galileo have values for the accuracy higher than the ones in the absolute interaction experiments. For Curtin on Galileo the accuracy increases when more days are considered, while for Galileo on Curtin it lowers slightly for 6 and 7 days. This may be caused by an initial phase with a low number of interactions in Universidad Galileo's MOOCs (see Fig. 3(a) ), which introduces noise for the classifier. However, the accuracy increases for the prediction experiment using the MIX dataset. Already the first day of interactions is sufficient for an accuracy of 0. 7.
the (average) session characteristics, such as Session Length, Timespan Clicks and Active Time, have higher scores. This suggests that users' behavior during a session relates stronger to whether users are Completers or Dropouts, than the number of sessions they have. For the First 7 Days Experiment, the rankings of Curtin on Galileo and MIX are similar to those obtained in the Absolute Experiment, with Timespan Clicks, Requests and Active Time always being the features with highest weights. For Galileo on Curtin, Session Length and Requests are almost always the highest scoring features. This mixed ranking could be due to the smaller dimension of Universidad Galileo's dataset, in respect to Curtin University's one. However, this aspect does not seem to be relevant for the Absolute Experiment.
" Existing works usually rely on expensive and specialized eye-tracking hardware (e. g. a Tobii eye tracker) to obtain gaze data, which is not available to the average MOOC learner. It is therefore still an open question whether eye-tracking based mind-wandering detection can be performed in a scalable manner. Our goal in this paper is to develop a fully automatic method for detecting mind-wandering and loss of focus in near real-time using only low-end webcams ubiquitously found on laptop computers. To this end, we conducted a laboratory study with 13 participants, collecting a dataset of gaze features (i. e. features extracted from gaze data) and self-reported mind-wandering. To motivate this approach, refer to Fig. 1  which visualizes the gaze of two of our study participants through heatmaps.
The MOOC video shown has several relevant visual areas, including the lecture slides, the subtitles, and the speaker's face. In the depicted scene, a changing set of examples is shown on the slides which are important to grasp the lecture content. The participant who reported mindwandering in the 30 s interval intently gazed on a spot on the speaker's face, ignoring the slides and the shown examples, while the second participant who reported no mind-wandering focused on all relevant areas of the video. Our proposed approach employs supervised machine learning to automatically learn such mind-wandering patterns based on gaze features. Our contributions in this work are as follows: 1. We create an elaborate gold dataset to foster eye-tracking based mindwandering research, featuring 13 participants watching two MOOC videos each in a controlled laboratory setting, reporting feedback on mind-wandering in brief intervals.
We counter the effects of this imbalance by applying the oversampling method Synthetic Minority Over-sampling Technique (SMOTE)  [5]. We have two requirements for our choice of classifiers as follows: 1. The selected models trained with our data can be used effectively to infer mind-wandering in data of unseen participants. 2. The selected models trained with our data can be used in real-time mindwandering detection. For the first requirement, we consider the bias-variance trade-off of machine learning models and the data size in our experiments. We select Logistic Regression, Linear SVM and Naive Bayes classifiers in our experiments as they have a low variance on small datasets like ours. These classifiers are also suitable for our second requirement. Since the trained models are small and require few inference steps, they can easily be integrated into Web applications within MOOC platforms.
Retrospective Analysis Following the Empirical Error Minimization approach, we selected the Logistic Regression classification algorithm which gave lowest empirical risk on average for the 12 weeks, 29. 84%, based on our training data using 10-fold CV. The bag of classifiers C also contained SVMs with linear and Gaussian kernels, a decision tree, a K-neighbours classifier and a Random Forest. The retrospective analysis carried out on CS1 using 2015/2016's data shows we can successfully gather student data about their learning progress and leverage that information for predictions, reaching a usable accuracy. We then progressed to run pseudo real-time predictions every week on the incoming 2016/2017's dataset.
g. diameter of the peer assessment graph, coverage percentage of corrected students plus their immediate neighbors, average distance between inferred and corrected students); (2) to choose the set of corrected students through topological strategies. The available datasets were used to generate also graphs with higher diameter than the original ones. As hypothesized, OA is less effective on higher diameter topologies. While this represents a general result regarding Bayesian models, the response to RQ2 indicates how this can be avoided in the specific case of peer assessment, if the students to be corrected are chosen by considering the network topology. The results provide an educational-specific operational strategy for using OA in a concrete setting, and for designing a suitable peer assessment network for each single session.
Methodology To show (RQ1) if the propagation of information through the BN drops in quality the more it moves far from the set of corrected nodes, we need networks with higher diameter (the maximum of minimal distances between any two nodes). In our datasets we have two groups of real assessments (datasets I and M, respectively with 2 and 6 peer-assessment sessions) where each student graded the 3 next peers in order from the group (modulo the size of the group). This produced "ring-shaped" networks with a diameter proportional to the number of nodes and inversely proportional to the number of corrected peers (Fig. 1 , left). To get even higher diameters we can either cut the ring (Fig. 1 , bottom, "broken" network) or reduce the number of assessments by each student (Fig. 1 , right, "2-peers" network), or both (Fig.
1 , bottom right). The figure shows also the teacher grades for each student (node color, darker = better) and the grades given by each peer (edge color, darker = better). When a new teacher grade is asserted for a student C variable (new evidence), two information propagations happen in two directions: from the corrected student towards her "judging" peers, and towards her "judges" peers. Because the information flows both directions the edges are considered as undirected (propagation has same weight in both directions). Table  1  shows for each assessment in datasets I and M the diameter for the graphs depending on the transformations: 3 peers or 2 peers, and ring shaped or broken. The aim of the experiments is to show that the distance traversed by the information in the model has an adverse effect.
Methodology To answer the research questions, the authors decided to follow the guidelines for systematic literature review in software engineering  [4]. In fact, before conducting the review, the authors developed a review protocol to reduce researcher bias and to keep a clear scope of the study. The search was performed in iterations in five main academic electronic databases in Technology Enhanced Learning (TEL): ACM DL, IEEE Xplore, SpringerLink, Science Direct, and Wiley, and two additional databases, SAGE and ERIC. The second cycle, included an independent search in the top ten educational and technology journals listed in the Google metrics sub-category: Educational Technology. The third and final cycle included search in the reference section for each selected paper in order to find additional relevant papers (i.
Review Methodology The review methodology followed to carry out this SLR is described in Kitchenham and Charters (2007)  [13]  which has been already employed in previous surveys in technology-enhanced learning. This methodology structures the SLR in three phases (i. e. , planning, conducting, reporting) providing guidelines for key issues such as the definition of the search strategy and study selection. In our case, the selected databases were ACM Digital Library, IEEE Xplore Digital Library, Science Direct, Scopus and Springer Link, since the authors consider that these databases are the most relevant databases in the topic field. Moreover, other articles not stored in these databases but cited in the retrieved publications have been also included in the review. The search includes journal publications, conference proceedings, books, book chapters, technical reports and thesis, trying to avoid possible bias.
An Empirical Study The study is based on recording and analyzing the digital traces (playing analytics) during a game session with Tamagocours. Tamagocours is a game implemented as an online and asynchronous teaching program for approximatively 200-300 students each year. The game aims at teaching pre-service teachers the rules (i. e. copyright) that comply with the policies for the use of digital educational resources. Tamagocours is an online, multiplayer and collaborative Tamagotchi. Each team of 2 to 4 online players 'feeds' a character (Tamagocours) with digital educational resources. The challenge consists of selecting appropriate resources for feeding a character and to make it healthy. Each player can see his teammates choosing the resources from a large database and the format with which these resources are used.
The data collected are split in two sets. The first set encompasses data from levels 1 to 3, relatively easy to win. The difficulty increases for levels 4 and 5, a second set of data. We perform a principal component analysis (PCA) and an ascending hierarchical classification (AHC) to identify classes of players' strategies. AHC extracts some classes where players' strategies are similar and where classes are far apart. Then, for each class, the most significant variables (assessed by Student's t-test) give indications to build meaning of classes. This analysis sequence is managed with the two datasets, first for levels 1 to 3 and second for levels 4 to 5.
Co-Location Datasets Used Co-location in the arena of our research can be roughly interpreted as the location of two or more individuals in the same physical place and at the same time. Individual incidence of a co-located pair cannot be interpreted as the individuals as having a relationship and does not infer personal contact between 2 people. Quannan Li  [7]  having mined subject GPS logs, used an hierarchical clustering algorithm to develop a trajectory model that determines a semantic meaning to stay-points (points where time is spent) and inferred similarity between subjects based on this. In that work, the results could be used as the basis of a recommender system, but in our work we will interpret the context of the co-location. In the context of a University campus, meetings can be either formal or informal i.
ReaderBench implements various metrics and categories of textual complexity indices that can be used to leverage the automated classification of datasets in multiple languages, such as English  [2] , French  [3] , Romanian  [4]  and Dutch  [5]. Surface Indices. These are the simplest measures that consider only the form of the text. This category includes indices such as sentence length, word length, the number of unique words used, and word entropy. All these indices rely on the assumption that more complex texts contain more information and, inherently, more diverse concepts. Word Complexity Indices. This category of indices focuses on the complexity of words, but goes way beyond their form. Thus, the complexity of a word is estimated by the number of syllables and how different the flectional form is from its lemma or stem, considering that adding suffixes and prefixes increases the difficulty of using a given word.
Introduction and Background Learning Analytics (LA) is a field that analyzes learners and their contexts mainly utilizing the data coming from digital realms to understand computer-mediated contexts. It has been argued, that multimodal data collection and analysis techniques that go beyond the digital environments can bring novel methods to understand when students solve problems, interact with peers and act in both -digital and physical worlds  [1]. In order to analyze learning as a whole and its context, there is additional data needed. This data can be coming from learning scenarios  [2, 3]  coupled with documenting their enactment  [4]. We argue that real-time human semantic labeling can be used to illuminate different areas of teaching and learning process, enrich LA data and be combined into Multimodal LA (MMLA) datasets.
To our knowledge, lesson observation tools provide observational data that are not compliant with LA datasets  [4]. The proposed solution is the classroom observation application that is able to aggregate semantic annotations and gather context-aware, human-labeled systematic observational data. This approach takes into account pedagogical underpinnings and collects data that is aligned with specific pedagogical intentions and foci. In this paper we present Observata prototype. The prototype has been validated with design-based research that involved semi-structured focus group interview during a design session with stakeholders. In order to understand teaching and learning processes, context of the learning experience is highly relevant; for this purpose, data coming from LMS is not enough.
Moreover, collection and analysis of only digital traces is not sufficient  [1]  and inclusion of qualitative data into the equation might be beneficial  [5]. MMLA data containing observations can offer insights into this issue and help enrich LA with contextual aspects. Aligning Learning Design and LA helps understanding learning behavior and creating actionable feedback loop  [6, 7]. Also, linking the generic pedagogical scenarios with contextualized learning scenarios and LA adds more to the evidence  [8]. Combining observational data into MMLA datasets has also been proven useful by some studies  [9]. As the classroom life is very busy and there can be around 1000 thousand learning interactions (or activities) taking place within a single day, data collection becomes difficult  [10, 11].
At the same time, observational data are especially informative  [12]. The approaches used in observations may be qualitative or quantitative; quantitative data is criticized for being taken out of context and failing to show the "story of the classroom life"  [10]. Systematic classroom observations that are aimed at capturing learning activities need a defined Unit of Analysis. It has been suggested that such unit of analysis is an (Learning) Event  [4]  (observable events). Previous study on observations shows that it is possible to annotate lesson events with xAPI statements  [13]. Semantically annotated xAPI statements can then be combined with MMLA datasets. In the next chapter we present observation application Observata that has been validated by scenario-based participatory design-session with participation of 6 persons representing different stakeholder groups, including in-service teachers, their mentors and teacher educators.
Technological Background NoteMyProgress 1  is composed of two parts: (1) a Google Chrome plugin developed in JavaScript; and (2) a dashboard developed in Ruby on Rails. The plugin is the component responsible for integrating the learning platform with the dashboard. Additionally, it collects information on the learning activities that the learner performs from the course URLs that they visit on the learning platform. The dashboard implements the modules that interpret the information collected from the user according to the original learning platform, and stores them in the database defined in PostgreSQL. Once the information is incorporated into the data model, the dashboard generates interactive and personalized displays to help the learner monitor their learning process. In the bibliography we have identified three tools aimed at supporting learner selfregulation in MOOC environments.
However, this is over a single course with 108 students, and so suffers from the same generalizability issues that we encountered earlier. In a novel paper  [9]  Zorrilla et al. attempt to solve a similar problem by training many ready-made models. They then train a meta-classifier to take a new dataset and classify it based on which model is most similar to it. We believe that our method improves on this, as it does not require extracting meta-features from sets of data.
From the perspective of OR, the ethical processes of academic research seem a straitjacket which prevents them from applying their methods. The universally accepted principle in academic research that consent should be gathered from users before their data is collected, and that this is only valid for specified uses of the data, is particularly remote from OR practice. 3 Is There a Learning Analytics Waiver? The ethics of LA is in a tangle. Let us take as an example the Open University (OU), because of the praiseworthy clarity of its policies on ethics and LA. The OU FAQs on LA inform students that "it is not possible, at present, to have your data excluded" because the OU wishes to use the dataset as a whole  [14]. On the other hand the OU's Ethics Principles for Research Involving Human Participants state that "Except in exceptional circumstances, where the nature of the research design requires it, no research shall be conducted without the opt-in valid consent of participants.
CollAnnotator Tool and Study Settings Given the popularity of the Community of Inquiry model  [6] , a dedicated platform for supporting content annotation based on CoI would prove useful to the researchers. We therefore developed such a content analysis tool, called CollAnnotator, which provides the following functionalities: rich annotation support, possibility to attach more than one code to a message, support for multiple coders and suggestive comparisons between them, support for the negotiation phase, detailed statistics and reports of the coding results, all in an intuitive and easy to use interface. Furthermore, CollAnnotator is adapted to our goal of using CoI for investigating the online community formed in our social learning environment, eMUSE  [5] ; it directly retrieves student content (blog posts and tweets) from eMUSE database and generates reports and statistics specific to our instructional scenario.
All students' contributions on these tools were retrieved and stored in a local database by means of the eMUSE platform  [5]. In particular, every team had a blog where each member could contribute and report the progress of the project, share interesting resources, describe problems encountered and potential solutions, ask questions to peers, provide feedback etc. A total of 479 contributions were recorded on students' blogs: 399 posts and 80 comments. These were subsequently analyzed using CollAnnotator, according to the procedure described in the next subsection.
Opeka for Teachers Opeka is an online tool with background questions and four questionnaires with different themes. Those themes are Digital operating environment, Organizational culture, Pedagogical activities and Competences. After completing the background questions other questionnaires, including mainly numerical questions can be finished in free order. When at least one of the questionnaires has been finished, the respondent gets his or her own results and feedback compared with other respondents. Opeka has been in use from year 2012 and the reports from every year after that can be compared easily as the data is saved in one single database.
2. Populate the game grid with the seed (see Fig. 1 ). First, a random cell is selected as the starting point for generating the word. Then, each move is randomly generated, with the conditions that the cells are neighbors and an already filled-in cell cannot be occupied. If the algorithm reaches a dead end, where at least one cell cannot be filled-in, the grid is rolled-back to the previous step. 3. Find and filter semantically similar concepts. For this step, we use a dedicated web service offered by ReaderBench, a framework designed for advanced text analysis  [7, 8] , to find semantically similar concepts. This service takes as input: a) the seed (the start word), b) the used language (English or French), c) the employed method (e. g. , semantic distances in lexical databases -WordNet and WOLF (Wordnet Libre du Français) -, Latent Semantic Analysis -LSA and Latent Dirichlet Allocation -LDA) and d) the corresponding corpus (e.
Solution's Architecture NAPP's architecture is based mainly on three technologies: Ionic Framework, PouchDB and CouchDB (see Fig. 2 ). On the client side (student's side), the multiple NAPP mobile applications, built with Ionic Framework, are able to keep their local PouchDB database upto-date even when the users are offline. On the server side (NAPE-TP's side), NAPP web application provides access to the information on the CouchDB server that is synchronized with all PouchDB instances.