dataesr/scientific-paragraphs-categorization huggingface dataset:
: Adoption and use of web 2. 0 in scholarly communications. Philos Trans A Math Phys Eng Sci. 2010a; 368(1926): 4039-4056. PubMed Abstract | Publisher Full Text Procter RN, Williams R, Stewart J, et al. : If you build it, will they come? How researchers perceive and use Web 2. 0. Technical report, Research Network Information, 2010b. Reference Source Public Knowledge Project: OJS stats. 2016; Date accessed: 2017-06-12. Reference Source Pullum GK: Stalking the perfect journal. Nat Lang Linguist Th. 1984; 2(2): 261-267, ISSN: 0167-806X. Reference Source Putman TE, Burgstaller-Muehlbacher S, Waagmeester A, et al. : Centralizing content and distributing labor: a community model for curating the very long tail of microbial genomes. Database (Oxford). 2016; 2016: pii: baw028. PubMed Abstract | Publisher Full Text | Free Full Text Raaper R: Academic perceptions of higher education assessment processes in neoliberal academia.
2017. Reference Source Thung F, Bissyande TF, Lo D, et al. : Network structure of social coding in GitHub. In 17th European Conference on Software Maintenance and Reengineering (CSMR), IEEE, 2013; 323-326. Publisher Full Text Tomkins A, Zhang M, Heavlin WD: Reviewer bias in single-versus double-blind peer review. Proceedings of the National Academy of Sciences. 2017; 201707323. Publisher Full Text Torpey K: Astroblocks puts proofs of scientific discoveries on the bitcoin blockchain. Inside Bitcoins. 2015; Date accessed: 2017-06-12. Reference Source Tregenza T: Gender bias in the refereeing process? Trends Ecol Evol. 2002; 17(8): 349-350. Publisher Full Text Ubois J: Online reputation systems. In: Dyson E, editor, Release 1. 0, EDventure Holdings Inc. , New York, NY, 2003; 21: 1-35.
The Student Model For this study, we used an Additive Factor Model (AFM), introduced by Cen et al. [20] , to model students' knowledge. The model uses logistic regression to predict the probability of a student i completing a step j correctly as a linear function of student parameters (the student's proficiency θ i ), skill parameters β k and the learning rates of skills γ k , as shown in Eq. (1). AFM takes into account the frequency of prior practice and exposure to skills but not the correctness of responses since it assumes all students accumulate knowledge in the same way. In this paper we implemented the AFM model following the approach of Chi et al. [21]  who modeled students working on physics problems using a dialogue-based tutor. ln p ij 1 -p ij = 𝜃 i + ∑ k 𝛽 k Q kj + ∑ k Q kj (γ k N ik ) (1) The dataset was collected by training 291 students on Rimac over a period of 4 years  (2011) (2012) (2013) (2014) (2015).
During students' interactions with Rimac, they answered reflection questions on physics problems about dynamics, such as: "In our first question we will focus on the horizontal motion of the arrow. Let`s imagine a scenario in which an archer is standing at the edge of a high cliff. He shoots an arrow perfectly horizontally with an initial velocity of 60 m/s off this cliff. During the arrow's flight, how does its horizontal velocity change (increases, decreases, remains the same, etc. )? Remember that you can ignore air resistance". Students worked on reflection questions about three physics problems that explored motion laws and addressed 88 knowledge components (KCs). The dataset contained in total 15,644 student responses. Each student response answers a question posed by the tutor and was classified as correct or incorrect.
TURET2. 0 was developed in Python with the Django+HTML5 Web framework, to display the user interface and evaluation results. Also, the open-source relational database management system MySQL was used to store the results of each evaluation of students. Finally, Freeling tool was installed as a server, such that the lemmatization process was performed under a service scheme, i. e. , when a student requests an evaluation in the tutor, the system requests the lemmatization service of Freeling.
One plausible explanation for the latter could be the timing of the prompts. The specific prompts were towards the end of the MOOC, when the design groups were too advanced in their documents to be able to incorporate new ideas. We plan to investigate the feedback uptake by the design groups in the future. There is a growing interest towards MOOCs with complex social structures, where participants benefit from small group collaboration, as well as larger scale communities of interest. This contribution presents one such example and shows the contextualisation of data within the nested social structure. The authors hope that this contribution exemplifies forthcoming MOOCs with innovative social and pedagogical scenarios. In conclusion, this multi-level analysis has opened a few new directions for further investigations and interventions.
[35]  of 55 dashboards looked at the context in which dashboards had been deployed, their purpose, the displayed indicators, the technologies used, the maturity of the evaluation and open issues. The scope of all these reviews included learning analytics dashboards, regardless of their target users. Focusing on the challenges identified by Ferguson  [13] , we narrow down our scope to LA dashboards aimed at learners in order to focus on their perspective. A closely related work to this paper was published by Bodily and Verbert  [4]. They provided a systematic review that focused exclusively on student-facing LA systems, including dashboards, educational recommender systems, EDM systems, ITS and automated feedback systems. The systems were analysed based on functionality, data sources, design analysis, perceived effects on learners and actual effects.
Although other works looked into the learning theory foundations of gamebased learning  [46] , one major limitation of previous dashboard reviews is that none investigate the connection to learning sciences. Moreover,  [4, 35]  provide recommendations for the design of learner dashboards, but none suggest the use of educational concepts as a basis for the design or evaluation of the dashboards. Through this systematic literature review we aim to bridge this gap by investigating the relation between educational concepts and the design of learning dashboards. Dashboard design was previously examined by looking at the type of data displayed on the dashboard and the type of charts or visualisation that were used. However, in this study, we will specifically focus on how the data presented on the dashboard is contextualised and framed to ease the sense-making for the learners.
Methodology Prior to the systematic review, we conducted an informative literature search in order to get an overall picture of the field. We ran the systematic literature review following the PRISMA statement  [31]  and we selected the following databases which contain research in the field of Technology Enhanced Learning: ACM Digital Library, IEEEXplore, SpringerLink, Science Direct, Wiley Online Library, Web of Science and EBSCOhost. Additionally, we included Google Scholar to cover any other sources, limiting the number of retrieved results to 200. We searched the selected databases using the following search query: "learning analytics" AND (visualization OR visualisation OR dashboard OR widget). The first term narrows down the search field to learning analytics, while the second part of the query is meant to cover different terminologies used for this type of intervention, addressing one of the limitations identified in  [35].
The dataset also included teachers' typed observations regarding children's performance, behavior and participation in the learning process; per researchers' instructions, these observations was noted by teachers at the end of each session in a specific notesarea for typing within the Kinems software. Furthermore, at the end of the programme, semi-structured interviews were conducted with all participating teachers. As shown in Table  2 , questions focused on teachers' perceptions of students' improvement through their participation in the programme and the value of the Kinems games for embodied learning for students with motor impairments and educational needs. In what ways did the games support children's motor needs and learning needs? 4 Were the games hard, easy, and usable for the children and the teacher/therapist? 5 Please describe the general performance of children across sessions (motivation, completion time, body and hand movement etc.
Methodology In our review, we have followed the methodological guidelines proposed by Kitchenham and Charters  [22]. We queried four academic databases for works in WPLA: Science Direct 1  , IEEE Xplore 2  , Springer Link 3  and ACM Digital Library 4. Additionally, we used Google Scholar 5  to find grey literature and other references we might have overlooked. We also searched manually in specific literature sources in the area, namely the Journal of Learning Analytics 6  and a recent workshop on WPLA  [27]. Our review focuses on LA studies devoted to support WPL and professional development. Given the recent emergence of the term and the fragmentation of this research community, other overlapping terms were also added to the query we used on these literature databases: 'educational data mining' (very related to LA and with a slightly longer history), 'adaptive learning systems' and 'intelligent tutoring systems' (to catch earlier works which have many commonalities with what we now denominate LA).
However, we did not include terms related to the transition between vocational schools or higher university and workplace learning, nor the use of LA to assess higher education activities outside the classroom. The query string we used to query those databases was: ("Learning Analytics" OR "Educational Data Mining" OR "educational datamining" OR "adaptive learning systems" OR "intelligent tutoring systems") AND ("workplace" OR "professional development") The query was launched on September 2016. The references were obtained from the four databases were collected, as well as the 100 first results (out of 7520) from Google Scholar. Furthermore, we added resources known to us from previous work on the area of WPLA and we obtained a total amount of 1320 references. It should be noticed that there maybe variations in the way each search engine applies the query (e.
g. , papers whose low quality prevents understanding and assessing the contribution). After this filtering, 30 papers were left to be reviewed in detail, forming the dataset for the rest of the analysis in the following sections. These 30 papers included 7 journal publications, 19 conference papers and 4 book chapters. The descriptions of these papers are summarized in Table  1. Note that the reduction from 1320 initial results to 30 reviewed papers is mainly due to four aspects. First, we launched the same query in five different search engines so many of the results obtained were duplicated. Second, the search engines of some academic databases do not allow to search only the terms included in the title, keywords and abstract; hence, in many reference the terms that are relevant for us were cited but were not too relevant for the paper.
Research Methods and Evaluation The methodological approaches followed in the 30 papers under review can be broadly classified in four categories. The largest set of papers  (13)  follows the traditional methodological approach of presenting and evaluating a proposal. Another significant cluster (8) spans several research iterations, combining topdown and bottom-up approaches, which allow them to carry out exploratory and evaluative work. There are also papers (4) that explore certain aspects in a bottom-up fashion, inferring theory or trends from available datasets. Finally, 5 papers are exclusively theoretical proposals that draw from previous literature. Concerning their evaluation, 6 of the analyzed papers do not portray an evaluation. In other examples  (9) , the purpose of the evaluation is merely to provide a proof of concept, or to illustrate the potential of the proposal.
Many of the papers analyzed were published under different keywords, some of which we collected when querying research databases (e. g. , 'adaptive learning systems' or 'teaching analytics'). Nonetheless, there may be other terms that we did not consider and can provide further insight on this and other related fields. The provision and adoption of WPLA solutions are higher in education and healthcare sectors. In both cases, the professionals involved share some routines that contribute to the applicability of WPLA (e. g. , need for being up to date, need for reflection processes). On the contrary, other sectors (e. g. , construction) could be more challenging in order to receive LA support due to the lack of trackable evidence in their current activity. Additionally very few existing proposals are targeted at, and evaluated in, multiple domains.
Therefore, we decided to improve the baseline to compare with in order to obtain richer conclusions about the impact of using a criteria-based approach for grouping. Hence, in the control group, we segregated the students with zero page views by grouping them together prior to the creation of the random teams. The algorithm selected for implementing the homogeneous grouping was kmeans clustering because it is a well known, effective technique that works with big datasets  [31]. We combined it with a balancing algorithm to obtain clusters with exactly the same number of members (same size k-means variation 1  ). To carry out the experiment the following steps were followed: 1. Finding out the statistical distribution of the selected variables (page views, submitted tasks and forum messages).
For example -let's suppose we have the sequential behavioral pattern of: Task Question Asking(person 2) Uncertainty(person1) that maximizes the utility of curiosity of person 1. On perceiving that person 2 has asked a task-related question, and person 1 is passive in subsequent time steps, the social influence knowledge database can be consulted and the specific causal influence rule of: Uncertainty (other) Uncertainty (own) can be picked by a pedagogical agent to verbalize an expression of uncertainty about some aspect of the task that was related to the question asked by person 2, along with (maybe) asking person 1's opinion about the same. This is likely to capture person 1's attention, who might express uncertainty about similar aspects of the task. Such shared uncertainty might make person 1 eager to reduce their knowledge gap by engaging in joint exploration, in turn maximizing their curiosity.
We explore in this article the interactions between learners and the remote apparatus to study the potential correlations between their performance score at the final assessment test, and both quantitative indicators and sequential action patterns. Our objective is to identify behavioural patterns for a practical session that lead to better learning outcomes, in order to predict learners' performance and to automatically guide students who might need more support to complete their tasks. The next section presents the computational settings (i. e. , our learning environment, with a focus on its tracking framework), and exposes the experimentation protocol together with the resulting dataset. While a first analysis exposed in Sect. 3 covers engagement indicators such as the number of actions achieved by a student, or the time between two actions, Sect.
Measure of Academic Performance We defined in this study the assessment score (AS) as a continuous variable between 0 and 20 that denotes the score learners got when they took the test at the end of the course. The distribution of AS in the experiment presents qualitative cutpoints that make clearly appear three categories of AS (AScat): low (named L; number of students (N) within this category = 22), medium (M, with N = 27) and high (H, with N = 36). In the next two sections, the dataset resulting from the experimentation is analyzed against the AS and/or the categories of AS. The following section defines some quantitative indicators as independent variables and investigates their correlation with the two above mentioned dependent variables, before we go into deeper pattern mining analysis in Sect.
Study of Quantitative Indicators Starting from the records of the dataset, we first studied the four following quantitative indicators: (1) the number of commands submitted by a learner (#submissions); (2) the percent of commands executed successfully (%success); (3) the average time spent between two submissions of commands of the same working session (ΔTime); and (4) the number of commands submitted by a learner referring to help seeking (#help). The first three indicators can be found in other research works  [4, 12]  and allow quantifying learners' production. The last indicator identifies help access. While it can be difficult to compute in other contexts (i. e. , when help resources reside outside the learning environment), remote or virtual labs often come with their own assistance material, whose access can be easily tracked  [13].
Very closed to our works,  [20]  suggests a topical N-gram Model applied to two Coursera MOOCs to extract common session topics (e. g. , "Browse Course", "Assignment and Forum"), to cluster learners according to these topics, and eventually to study the difference of apparition of the topics between highand low-grade students. Still on the dataset of Coursera MOOCs,  [21]  studied patterns of actions at a higher level of abstraction to distinguish between highand low-achieving users. The authors proposed a taxonomy of exclusive MOOC behaviors (i. e. , viewer or collector, solver, all-rounder, and bystander) based on the observation of the number of assignments and lectures users completed, and explored their distribution through different dimensions such as engagement, time of interaction, or grades.
This would give MOOCs' providers actionable information, allowing them to adapt their courses accordingly. Additionally, comparing features that best distinguish completers and dropouts across different systems will yield new insights into general behavioral patterns that dictate individual outcomes of MOOCs for online learners. Specifically, the identification of such features, common to MOOCs across different systems, can reveal useful information to devise new strategies to mitigate the high dropout rates and keep users engaged and motivated when participating in MOOCs. Approach. First, we conduct and evaluate prediction experiments to detect at-risk users in early stages of MOOCs from two different systems. Second, we train a model based on features present in all our datasets, to identify the best predictors of dropouts across different systems.
Their results suggested that using a lower amount of past information could yield results comparable to the ones from a full window size. Balakrishnan and Coetzee  [1]  used Hidden Markov Models (HMM) to predict if users will drop out in the following week. The dataset consisted of a MOOC from Berkeley University, offered on edX. Their results can be used to suggest changes in the engagement style to those students who are more likely to drop out in the close future. Vitiello et al. [19]  attempted dropout predictions over a set of 5 MOOCs. Their results indicated that certain combinations of features could significantly improve prediction scores. In Sinharay  [18] , the author presented a detailed review of different data mining techniques and compared their performance with real-data examples.
Particularly, the author predicted dropouts on a dataset including students from various high schools in Florida. The obtained results indicated that methods such as Random Forests and Boosting can improve performance in regard to linear and logistic regression approaches. The work presented in this paper further extends the state-of-the-art by analyzing MOOCs from two different sources: Universidad Galileo and Curtin University. We initially analyze and perform a dropout prediction experiment on each of these individually. Then, we excerpt a multi-systems model for MOOC evaluation and classification of users likely to drop out. In order to do so, we rank our features according to their importance and compare the obtained results.
Dataset Table  1  shows the characteristics of the MOOCs from Universidad Galileo and Curtin University. Logs of Curtin University include interactions of each enrolled person, while those of Universidad Galileo only report interactions of active users (or learners). In our setting, interactions coincide with clicks of users in the MOOCs' environment. In particular, a total of 3, 157 active users in our datasets are from Universidad Galileo, and 35, 473 enrolled users are from Curtin University. We will use the more general term users to refer to these groups for each system. The average number of interactions for MOOCs of Curtin University is significantly lower than the ones of Universidad Galileo (see Fig. 1 ). We can see that Completers interact more often with the MOOCs in both systems, while the percentage of Dropouts is higher for the datasets from Curtin University.
The ones from Curtin University, however, are intended for a more general audience, not necessarily with experience on the topic of the course. Feature Comparison. Additional differences include the tools that each system deployed/implemented and the level of granularity of the logged interactions for We use these interactions to identify the total amount of enrolled users, but we do not consider such interactions when constructing the features. Multisystem Dataset. We create three additional datasets; the first one combines users of all MOOCs of Universidad Galileo, the second one includes users of all MOOCs of Curtin University and the third one combines users of all MOOCs from both systems. We reference them as Galileo, Curtin and MIX respectively. within the first group consist of time-related information, obtainable for both systems.
Class Balancing. For both systems, the number of Dropouts is significantly higher than the number of Completers. We addressed this class imbalance problem by oversampling  [9, 12]. This means that new samples are randomly picked and added to the class with fewer examples until its dimension equals to the one of the larger class. Training. Once classes are balanced, we split the examples into a training set, used to train the classifier, and a test set, which we use for evaluation. We use a ratio of 80:20 between training and test datasets, using a Stratified Shuffle Split with 10 folds. With this approach, each fold will also be balanced in the number of examples from each class. Furthermore, the shuffle assures that each fold will consist of different examples. Evaluation. Finally, we use accuracy to evaluate the prediction error of the experiments.
(c) and (d)). For the individual prediction experiments we use Support Vector Machines (SVM), which try to find the optimal hyperplane (in higher dimension spaces) to separate data points. For the system-to-system and multisystem experiments, we predict dropouts using Boosted Decision Trees  [7]. This ensemble classifier combines the outputs from a set of single decision tree in a sequential way. For each learned model, the examples are re-weighted; the misclassified ones receive a higher weight, while the correctly classified ones get a lower weight. This way, the next decision tree will focus more on the misclassified examples. Overall, we propose three experiments. First, we conduct two system-to-system dropout prediction experiments. We use the Curtin dataset for training and the Galileo dataset to test our classifier.