dataesr/scientific-paragraphs-categorization huggingface dataset:
EAST As mentioned in the state of the art, the EAST network performs well at extracting text boxes from an image. In its original use, the network is trained to detect text in natural scenes, which means this dataset does not fit in our context. Nonetheless, the segmentation remains robust and accurate in diverse contexts. For this reason, adapting it to our data helps to generalize the results obtained by the correction performed by the user. In order to ease the task of the network, we chose to provide keep the histogram-based segmentation method to extract paragraph and then use the deep learning algorithm to detect word bounding box. This aims to attenuate the differences between ground truth data which can be significant due to the different writing. Thus, to gather a sufficient amount of data to train the model for our context, we used the best results of the corrected histogram-based word level bounding-box.
After training the model on this personal dataset, it can correctly detect the text inside the main features of the paragraphs given to it. To finally provide extracted samples in a logically correct order, the detected bounding box set needs to be ordered based on the text layout. A naive sorting algorithm could be confused by uneven bounding boxes because of large capital letters or even badly cropped paragraphs. This problem is addressed by the algorithm 1. The latter's goal is to provide a top to bottom and left to right sort based on important changes within the bounding box height position distribution to detect line breaks. Once this information has been acquired and a proper vertical sort performed, it only remains to sort horizontally the bounding box. Algorithm 1: Bounding Box sorting algorithm Input: EAST detected bounding boxes of size N Output: Sorted bounding boxes Sort Input content based on vertical coordinates Distances: Array which contains the differences of the height between each successive bounding boxes By putting end to end the different tools described in this section, a process granting the ability to automate text-area detection is created in order to provide a text-focused image to OCR algorithms.
Transcription dataset builder tool Once the text segmentation is computed, it remains to acquire, in an accurate manner, the text data via an OCR process. This is done based on deep learning algorithms, using a gated convolutional neural network as described by  de Sousa Neto et al. [2020]. Therefore having ground truth data through which the network can trained was necessary. Contributors were asked to manually annotate original segmented paper words. To reduce these interventions, the building of a tool that automatically presents all paper fragments to the user has helped saving time. The GUI, shown in figure  15 , developed for this project aims to prevent time-consuming operation with the help of keyboard shortcuts that help the user to navigate in the segmented data and provide transcription.
Handwritten text recognition After transcribing 8,741 words, we created training, validation and test sets, which will then be used to train the network. The distribution between the different sets is the following : Train = 5,664 words, Validation = 1,535 words, Test = 1,542 words. Once the model is trained on the data created, it can be used to extract the text from the images given in input. Then the whole process can be set up. It will take a full page as input and segment it into words on the basis of the same preprocess as the dataset creation part. By doing so, the images are then in the exact same format as the ones used for training. The words extracted are then concatenated following the text structure to have the different paragraphs well separated.
Experimental results The number of detected words in every paragraph is then used to estimate the accuracy gain between both segmentation methods. The need for a comparison basis is, however, unavoidable concerning the metric meaning. The lack of perfectly accurate ground truth built through manual work in this project made us use the data partially corrected by contributors. This can be seen in the figure  17  that presents algorithms accuracy compared to the original one. The error rate between ground truth and method results may be explained in different ways. First of all, the histogram-based method is the basis of the training data that contributors have sorted out to remove false positive from the dataset, which causes this method to have more detected words than the final dataset. The results provided by the EAST detector come from the signatures false positive detection as well as words bounding box inaccuracy.
Indeed, the detector main issue is in handwritten text width inaccuracy that causes words to be mixed. To have a global idea of how the whole process performs, the mean character error rate (CER) and the mean word error rate (WER) were used. To calculate these metrics the Levenshtein distance was evaluated between ground truth and the predictions on 11 pages that were not in the dataset. We also compared these results to the ones obtained using Tesseract. The results shown in figure  18  reveal that the solution presented gives better results on the main paragraphs than on the full text. These paragraphs are easier to segment since no additional information has been added to them over time, therefore their shape is similar for all of them that make it easier to analyse. This points out that the layout analysis still needs to be improved so that all the additional data about one person remains together and is not mixed with the main paragraph.
Finally, this method might ease the adaptation of the current pipeline on other types of data and thus make it more flexible. The development of a fully modular software could also enhance the processing pipeline and make it more robust for new data shapes integration. This would ease the use by other teams and make it more accessible. Then, as described in 2. 1, the goal would be to get a fully-featured pipeline through which the user would almost only have to provide scanned images to acquire data in a common database format. These features might be developed based on NLP algorithms that could provide an accurate and deep analysis of text information that may help to create queryable data. This could then be used directly by final users through standard software. The work presented in this paper aimed to propose a complete software pipeline through which users will be able to save time-based on state-of-the-art and production-ready algorithms.
The choice of machine learning algorithms for both segmentation and recognition processes brings a general accuracy and robustness to the process which, however, requires ground truth datasets. Therefore, the creation of the latter, in an accurate and typical manner, needs manual intervention which has been strongly reduced by the help of temporary tools based on simple heuristics and traditional image processing strategy. The support of a basic user interface helped, moreover, the user throughout the validation of the pre-annotated data. These additional tools mainly saved time and allowed the creation of a sufficiently exhaustive dataset by small teams in a limited period. This makes it more accessible and less expensive to set up separately from the project size and its number of contributors.
Dataset We utilised a digital exam from a computer science course at Stockholm University carried in 2020 as the dataset for the experiment. The quiz consists of 16 questions, each of which is labelled with a programming-related topic by the course teacher. , such as class, abstraction, parallelism, operators, increment and iteration. Each question gives a score in a binary fashion. Overall, the dataset contains 121 students' quiz performances. Students' private information like personal IDs and names are removed to completely anonymise the data. Consent for doing this research was received from all students participating through an online survey in the beginning of the course.
Participants -Data collection -Materials Seventeen adolescents with diagnosed autism (age 11-19 years) participated; seven of them had some musical knowledge, only two played one or more musical instruments, and all were sufficiently familiar with the use of mobile touch screen devices. Additional details of the user sample are provided in Table  1. All participants performed the test in a natural setting (ten at school, five at home, and two in a lab where they carry out afternoon study sessions) and he/she could relax by doing something else for a while. Data collected during tests were mainly observational, focusing on user behavior while performing the proposed tasks (relevant to the specific module) and interacting with the UI elements. We also collected automatic data recorded through the app such as trial errors, successful tasks, task execution time and interactions with the UI components outside the virtual keyboard.
This data was collected from the universities data warehouse. Based on prior research  [21]  on the relation between SE and the years needed to obtain the bachelor diploma, the SE was discretized into three target classes: "no risk" for students with a SE of 75% or higher (622 data samples), "moderate risk" for students with a SE between 40 and 75% (464 samples), and "at risk" for students with a SE lower than 40% and dropouts (738 samples). Data of four academic years, 2015-2016 up to 2018-2019, was collected resulting in 2016 student samples. After deleting incomplete entries, 1824 samples remain. More information on the data collection and the data structure can be found in prior work  [14, 16]. As high correlations were found between the three high school course grades (math, chemistry, and physics), the three high school grade variables were combined into a two-dimensional variables using a Principal Component Analysis (PCA), before using them in the predictive model.
In the designed ER lecture, the teacher programs Thymio to illustrate the various concepts students are expected to acquire. Specifically, as the students are expected to understand what a regular polygon is, know its properties and how to construct it, the teacher illustrates the construction of complex polygons (e. g. hexagons, octagons. ) by programming Thymio and making it draw polygons on a sheet of paper. Additionally, as the students are also expected to recognise and name the various angles in parallel lines  6  , two Thymios are concurrently used to demonstrate the relationships between these angles. The material prepared for the lecture is available at 10. 5281/zenodo. 4649842. The lecture was designed to last 90 min, which equals the duration of the corresponding traditional theoretical lecture.
Allowing students to develop their own assessment activities is suggested as an innovative assessment practice enhancing tutor experience  [2]. More importantly, engaging students in peer assessment has a positive impact on students learning experience, and helps improvement of performance  [2]. Despite prior work in this field, the intrinsic mechanisms and temporal dynamics of peer interactions that drive peer assessment in hybrid classes remain understudied. A few online tools exist for supporting peer assessment  [35] , and support for transparent and meaningful peer assessment learning analytics is lacking  [9]. For instance, such learning analytics may allow for reliability check of assessment  [9]. This paper aims to provide insights on how students engage in peer assessment and address the following main questions: 1) How peer assessment interactions occur? 2) What are the consequent student roles regarding the peer assessment process? 3) How the student assessment roles evolve temporally? To this end, we applied a graphlet-based method, a meaningful and expressive network analysis approach, to a dataset collected across seven in-class courses integrating an online peer assessment platform called Sqily.
Data Analysis. We identified teachers' adaptation strategies (Q1) by adding up the answers regarding how they performed (with a VLE, other digital tools or non-digital tools) or not in 24 school tasks (Sect. 2 of the questionnaire). We regrouped these 24 tasks according to the pursued higher-level objectives (design, transmission, facilitation, verification, communication, and self-training). Then, we used a K-means classification method on the tasks answers grouped by objectives in order to identify groups of teachers that use VLEs, and/or other digital tools for equivalent purposes and to model technology integration (Q2). We determined 5 levels for each means in order to respect the TIM scale of technology integration  [30]. For description purposes, we calculated the average of the objective performed and the number of teachers in each level.
Methods For the current study, we recruited students, aged 18-19 years old, who joined the university in October 2020, to take part in an online survey. Ethical approval was obtained from the author's university ethics committee, and participation in the survey was voluntary. Prior to completing the survey, the respondents were provided with an online information sheet and a consent form. The survey was initially piloted with four students, aged 19, who were members of the OU student association, and minor changes took place. The survey was then administered to a random university-wide sample of students. The survey ran between 2-28 February 2021 and received 377 responses. The dataset was anonymised on the 1st of March 2021, prior to initiating the process of data analysis.
Data Collection Collected data included student responses in a 5-item Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree), with an extra 'not applicable' (N/A) option, exploring motivations for joining a distance learning university (RQ1). The set of statements were developed based on student open-ended responses to previous internal studies on their motivations. The selection of statements drew from the Self-Determination Theory (STD)  [14] , by which people choose to engage in activities because they are inherently interesting or enjoyable (intrinsic motivation) or because they will lead to separable outcomes (extrinsic motivation). For example, potential motivations for joining distance learning higher education at the OU included enjoying studying via online learning (intrinsic motivation), but also saving money on living expenses (extrinsic motivation).
The final list of motivation statements was reviewed and agreed upon with the university's strategy office. Moreover, a closed-ended question gathered information as to whether the Covid-19 pandemic has impacted their decision for joining. In order to identify any differences in motivations for joining distance learning among student groups (RQ2), student demographics and study information were retrieved from the university's database. Table  1  presents the demographics and study information for all the 377 survey respondents. These included gender; race summarised in Black, Asian and minority ethnic (BAME) and non-BAME students; declared disability; faculty; and previous qualifications (PEQ), such as A-levels (the traditional subject-based qualifications that are offered by schools and colleges in the UK and can lead to university or further studies).
To reduce the dimensionality of the extracted features and select the best ones, we used Recursive Feature Elimination using the estimator Decision Tree. We also set the number of desired best features at different thresholds: 1% (19 features) and 2. 5% (48 features). The 3D tensor was therefore aggregated in a matrix of size 1927, 19 (no. samples × no. attributes). The target class values and other metadata including the id of the session and the attempt duration (in seconds) were kept in a separate data frame. Consequently, a cluster of machine learning classifiers was selected including (1) Decision Trees (DT), Gradient Boosting (GB), k-Nearest Neighbours (KNN), Naïve Bayes (NB), Random Forests (RF), Support Vector Machines (SVM). The hyperparameters of the classifiers were tuned using Grid Search on a subset of the dataset to optimise the convergence time.
Since the target class (presence or absence of a mistake) exhibited a highly unbalanced distribution (9% mistake -91% not-mistake) we applied oversampling using the Synthetic Minority Oversampling Technique using 10-fold-cross validation. We made sure to oversample only the nine training folds and not the validation fold. The dataset was between training and testing using leave-one-session-out: we trained the models on 13 sessions and tested on the remaining one, iteratively. We then averaged the results obtained. Additionally, we applied feature scaling using min-max normalisation with a range of -1 and 1. The scaler was fitted on the training set and applied on both validation and test sets. All the classifiers mentioned treating the variables as independent and identically distributed. Although there are no constraints about the independence of the observations, for example, required by the logistic regression, all these models do not capture the sequential nature of the physiological signals.
As shown in Table  4 , the multivariate HMM outperforms by a big margin all other classifiers in terms of accuracy, precision, recall and F1-score, both for the 1% and 2. 5% best features thresholds. In our understanding, this proves the idea that Bayesian probabilistic models are better suited for the proposed supervised classification tasks as they better consider the temporality of the physiological signals. Among the remaining and more traditional classifiers, the best performing ones are RF and DT. When compared to HMM, they score higher in Precision but lower in Recall, resulting in lower F1 scores. Both DT and RF are more flexible in accommodating high dimensional datasets and more transparent in terms of feature importance. Concerning what is the best number of features for the classification task, the results suggest that it depends on the model.
This process was served as a way to reflect on the concept asked by the question. All activities on the platform were logged as transactions including question content, self-explanation content, timestamps, and correctness. The dataset in this study was collected from an introductory programming course offered in Spring 2018 at a four-year public university in the United States. There were 249 undergraduate students enrolled in the course. The platform was introduced to students as an optional learning tool for them to practice and monitor their learning progress. The students were told that their activities on the platform would not be graded, which primed them to take questions for their benefit but not the requirements of the course. Throughout the semester we collected in total 2228 transactions from 112 students in the course.
Data Collection Tools As mentioned above, we decided to collect student's individual reflective writing behaviours using Google Docs (http://docs. google. com). It is an online collaborative web-based platform for word processing. There are various platforms for reflective writing tasks such as Input Log. However, installation and activation are required and this might not be practical for real-time teaching and learning contexts where reflective writing happens at students' personal computers. Google Docs, on the other hand, can keep track of every change by chronologically storing versions of the file (called 'revisions') in the cloud database. Each revision has a unique and auto-incremental identification number. However, Google Docs occasionally merges revisions for space optimization purposes  1  which results in minor changes or some revisions lost.
Pre-processing Out of 42 students, 2 students were excluded because they did not submit the reflections via Google Docs. As a result, 40 students remained for processing. Another three students submitted the weekly reflections through multiple Google Docs, thus merging was performed. Additionally, the changes that did not belong to the students (e. g. , the reviewer accidentally edited the document) or the changes that were made after the submission date, were filtered out. In the end, the resulting dataset described approximately 600000 editing actions (revisions) in total.
Derived Features Two datasets were created to be investigated: the 'Activity' dataset composed of the actual changes that students have made and timestamps, 'Student' dataset contained information related to students, their SRL level and their grade. For each editing action described in the 'Activity' dataset, two features were added. By integrating timestamp and students' timezones, we inferred (1) DayOfWeek_local-day of the week in which the change happened at the student's local timezone. By considering the type of changes, starting index and ending index, (2) strCount-number of letters added or deleted was counted regardless of the change types. For individual students, seven features were derived: (1) TotalRev-number of total revisions, (2) FinalStringCount-number of strings in the final document, (3) AvgRevPerDay-the average number of revisions made per day, (4) AvgStrCountPerDay-the average number of strings added/deleted per day, (5) TotalActiveDay-number of days that students have made changes (possible 99 days), (  6 ) AvgStrCountPerWeek-the average number of strings added/deleted per week, and (7) TotalActiveWeek-number of weeks that students have made changes (possible 15 weeks).
Therefore, the technical base was chosen to be the Unity Engine (unity. com). Data storage was realized by secure communication via https to a MySQL database under the researchers' authority. Cards. The card and card deck metaphors were kept in the transformation process in a general approach of skeuominimalistic aesthetics  [23]. Thereby, the proportional shape, fonts and colored frame of the cards were kept. On the other hand, there was no attempt to simulate paper card depth with shadows (Fig. 2 ).
Methodology Data is the fuel of ML projects and is the start point of the methodology we propose. As shown in Fig. 1 , the first step in a learning analytics project is the collection of the different learning traces from the available data sources and cleaning it. This first phase allows us having the raw data ready to be used and analyzed in the next phase, which is feature extraction. The overall goal of the feature extraction is to prepare a new dataset composed of a set of computed features representing learning indicators. The third step is modeling using ML algorithms. Each model takes as input the set of features previously computed and gives as output the predicted class of each learner. The prediction results are then evaluated according to one general measure, which is the accuracy.
These five phases comprise a wide variety of learning interactions. However, due to the lack of data related to two-votes-based processes  [3] , little work has explored how to use these interactions to bring new knowledge about formative assessment. Hence, in this paper, we address the following research questions: Which meaningful information can be inferred from the analysis of data gathered from a tool implementing a two-votesbased process and used in authentic contexts? How can such information contribute to facilitate two-votes-based process orchestration? The three main contributions are the followings: -findings about formative assessment, based on a dataset gathered from the use of a formative assessment tool in authentic learning contexts in higher education; -recommendations to assist designers of formative assessment systems; -recommendations to assist teachers when orchestrating two-votes-based sequences.