ethanporcaro/eric-fulltext-meta huggingface dataset:
Conditional Subspace Clustering of Skill Mastery: Identifying Skills that Separate Students
Author: Nugent, Rebecca
Date: 2009-07-00
Type: Reports - Descriptive
Subjects: Data Analysis,Students,Skills,Cluster Grouping,Matrices,Computation,Automation,Identification,Profiles,Intelligent Tutoring Systems,Mathematics Instruction

In educational research, a fundamental goal is identifying which skills students have mastered, which skills they have not, and which skills they are in the process of mastering. As the number of examinees, items, and skills increases, the estimation of even simple cognitive diagnosis models becomes difficult. We adopt a faster, simpler approach: cluster a "capability matrix" estimating each student's individual skill knowledge to generate skill set profile clusters of students. We complement this approach with the introduction of an automatic subspace clustering method that first identifies skills on which students are well-separated prior to clustering smaller subspaces. This method also allows teachers to dictate the size and separation of the clusters, if need be, for practical reasons. We demonstrate the feasibility and scalability of our method on several simulated datasets and illustrate the difficulties inherent in real data using a subset of online mathematics tutor data. (Contains 3 figures and 2 tables.) [For the complete proceedings, "Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009)," see ED539041.]
Process Mining Online Assessment Data
Author: Pechenizkiy, Mykola
Date: 2009-07-00
Type: Reports - Evaluative
Subjects: Data Analysis,Methods,Computer Software,Computer Assisted Testing,Multiple Choice Tests,Case Studies,Foreign Countries,College Students,Test Format,Feedback (Response),Charts

Traditional data mining techniques have been extensively applied to find interesting patterns, build descriptive and predictive models from large volumes of data accumulated through the use of different information systems. The results of data mining can be used for getting a better understanding of the underlying educational processes, for generating recommendations and advice to students, for improving management of learning objects, etc. However, most of the traditional data mining techniques focus on data dependencies or simple patterns and do not provide a visual representation of the complete educational (assessment) process ready to be analyzed. To allow for these types of analysis (in which the process plays the central role), a new line of data-mining research, called &quot;process mining&quot;, has been initiated. Process mining focuses on the development of a set of intelligent tools and techniques aimed at extracting process-related knowledge from event logs recorded by an information system. In this paper we demonstrate the applicability of process mining, and the ProM framework in particular, to educational data mining context. We analyze assessment data from recently organized online multiple choice tests and demonstrate the use of process discovery, conformance checking and performance analysis techniques. (Contains 6 figures and 4 footnotes.) [Funding was provided by the Dutch Science Foundation (NWO). For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Student Consistency and Implications for Feedback in Online Assessment Systems
Author: Madhyastha, Tara M.
Date: 2009-07-00
Type: Reports - Evaluative
Subjects: Web Based Instruction,Computer Assisted Testing,Computer Software,Computer Science Education,Artificial Intelligence,Computer Graphics,Predictor Variables,Data,Information Retrieval,Data Analysis,Educational Indicators,Educational Assessment,Educational Strategies,Educational Policy,College Students,College Instruction,Scaffolding (Teaching Technique),Formative Evaluation,Laboratories,Cognitive Processes,Misconceptions,Feedback (Response)

Most of the emphasis on mining online assessment logs has been to identify content-specific errors. However, the pattern of general &quot;consistency&quot; is domain independent, strongly related to performance, and can itself be a target of educational data mining. We demonstrate that simple consistency indicators are related to student outcomes, and suggest how consistency might be used in an online assessment framework to provide scaffolding to help students in need. (Contains 2 figures and 2 tables.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Predicting Correctness of Problem Solving from Low-Level Log Data in Intelligent Tutoring Systems
Author: Cetintas, Suleyman
Date: 2009-07-00
Type: Reports - Research
Subjects: Programming,Evidence,Intelligent Tutoring Systems,Regression (Statistics),Prediction,Problem Solving,Artificial Intelligence,Elementary School Students,Mathematics Instruction,Grade 4,Special Needs Students,Federal Aid,Educational Experiments,Computer System Design,Computer Managed Instruction,Computer Software,Data,Information Retrieval,Data Analysis,Word Problems (Mathematics),Least Squares Statistics

This paper proposes a learning based method that can automatically determine how likely a student is to give a correct answer to a problem in an intelligent tutoring system. Only log files that record students' actions with the system are used to train the model, therefore the modeling process doesn't require expert knowledge for identifying domain specific skills that are needed to solve the problem or students' possible solution methods etc. The model utilizes a set of performance features, problem features, time and mouse movement features and is compared to i) a model that utilizes performance and problem features, ii) a model that uses performance, problem and time features. In order to address data sparseness problem, a robust Ridge Regression algorithm is designed to estimate model parameters. An extensive set of experiment results demonstrate the power of using multiple types of evidence as well as the robust Ridge Regression algorithm. (Contains 3 tables.) [For the complete proceedings, "Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009)," see ED539041.]
A User-Driven and Data-Driven Approach for Supporting Teachers in Reflection and Adaptation of Adaptive Tutorials
Author: Ben-Naim, Dror
Date: 2009-07-00
Type: Reports - Descriptive
Subjects: Foreign Countries,Intelligent Tutoring Systems,Computer System Design,Computer Managed Instruction,Integrated Learning Systems,Automation,Reflection,Reflective Teaching,Student Behavior,Expertise,Action Research,Comparative Analysis,Web Based Instruction,Data,Information Retrieval,Data Analysis,Educational Assessment,Educational Strategies,College Instruction

It has been recognized that in order to drive Intelligent Tutoring Systems (ITSs) into mainstream use by the teaching community, it is essential to support teachers through the entire ITS process: Design, Development, Deployment, Reflection and Adaptation. Although research has been done on supporting teachers through design to deployment of ITSs, there is surprisingly little discussion about support for teachers' Reflection--the ability to draw conclusions from ITS usage, and Adaptation--adapting the content to better meet the needs of students. We describe our work on developing analysis tools and methodologies that support reflection and adaptation by teachers. The work was done in the context of helping teachers understand student's behavior in Adaptive Tutorials by post-analysis of the system's data-logs. We used a hybrid solution--part of the data-mining effort is teacher driven and part is automated. We tested our approach by comparing the results of expert analysis of two Adaptive Tutorials with and without an automated Refinement Suggestion Tool, and found it to be a useful teacher's aid. By using this tool, teachers act as &quot;action researchers&quot;, confirming or disproving their hypotheses about the best way to use ITS technology. (Contains 3 figures.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Unsupervised MDP Value Selection for Automating ITS Capabilities
Author: Stamper, John
Date: 2009-07-00
Type: Reports - Research
Subjects: Data Analysis,Computer Assisted Instruction,Intelligent Tutoring Systems,Markov Processes,Logical Thinking,Automation,Comparative Analysis,College Instruction,College Students

We seek to simplify the creation of intelligent tutors by using student data acquired from standard computer aided instruction (CAI) in conjunction with educational data mining methods to automatically generate adaptive hints. In our previous work, we have automatically generated hints for logic tutoring by constructing a Markov Decision Process (MDP) that holds and rates historical student work for automatic selection of the best prior cases for hint generation. This method has promise for domain-independent use, but requires that correct solutions be assigned high positive values by the CAI or an expert. In this research we propose a novel method for assigning prior values to student work that depends only on frequency of occurrence for the component steps, and compare how these values impact automatic hint generation when compared to our MDP approach. Our results show that the utility metric outperforms a classic MDP solution in selecting hints in logic. We believe this method will be particularly useful for automatic hint generation for ill-defined domains. (Contains 2 figures and 4 tables.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Consistency of Students' Pace in Online Learning
Author: Hershkovitz, Arnon
Date: 2009-07-00
Type: Reports - Research
Subjects: Foreign Countries,Psychometrics,Comparative Analysis,Correlation,Electronic Learning,Online Courses,Test Preparation,Secondary School Students,Learning Strategies,Web Based Instruction,Data,Information Retrieval,Data Analysis,Educational Strategies,Pacing,Time Factors (Learning),Semitic Languages,College Entrance Examinations

The purpose of this study is to investigate the consistency of students' behavior regarding their pace of actions over sessions within an online course. Pace in a session is defined as the number of logged actions divided by session length (in minutes). Log files of 6,112 students were collected, and datasets were constructed for examining pace rank consistency in three main situations: day/night sessions, beginning/end (for both situations, sessions of the same learning mode were taken), and a comparison between sessions from different learning modes. For each dataset, students were ranked twice, according to their pace in the two sub-groups, and these ranks were correlated. Results obtained with this study's data suggest that pace is sometimes not consistent, hence might not be considered as a characterizing measure for the whole learning period. A discussion of this study and further research is provided. (Contains 1 figure and 5 tables.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
How Do Students Organize Personal Information Spaces?
Author: Hardof-Jaffe, Sharon
Date: 2009-07-00
Type: Reports - Research
Subjects: Data Analysis,Organization,Information Management,Undergraduate Students,Graduate Students,Multivariate Analysis,Filing,Classification,Web Sites,Foreign Countries

The purpose of this study is to empirically reveal strategies of students' organization of learning-related digital materials within an online personal information archive. Research population included 518 students who utilized the personal Web space allocated to them on the university servers for archiving information items, and data describing their directory hierarchies. Several variables for measuring folders size and depth were defined, and four of them were chosen as best representing different aspects of the user's archive structure. Then, as a result of cluster analysis of the students, four organization strategies emerged, refining the classical piling/filing classification: piling, one-folder filing, small-folders filing, and big-folder filing. Also, associations were found between the organization strategies and archive size, students' studies degree. A discussion of this study and further research is provided. (Contains 4 tables.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Determining the Significance of Item Order in Randomized Problem Sets
Author: Pardos, Zachary A.
Date: 2009-07-00
Type: Reports - Research
Subjects: Data Analysis,Bayesian Statistics,Statistical Analysis,Problem Sets,Intelligent Tutoring Systems,Questioning Techniques,Reliability,Validity,Accuracy,Simulation,Mathematics Instruction,Secondary School Students

Researchers who make tutoring systems would like to know which sequences of educational content lead to the most effective learning by their students. The majority of data collected in many ITS systems consist of answers to a group of questions of a given skill often presented in a random sequence. Following work that identifies which items produce the most learning we propose a Bayesian method using similar permutation analysis techniques to determine if item learning is context sensitive and if so which orderings of questions produce the most learning. We confine our analysis to random sequences with three questions. The method identifies question ordering rules such as, question A should go before B, which are statistically reliably beneficial to learning. Real tutor data from five random sequence problem sets were analyzed. Statistically reliable orderings of questions were found in two of the five real data problem sets. A simulation consisting of 140 experiments was run to validate the method's accuracy and test its reliability. The method succeeded in finding 43% of the underlying item order effects with a 6% false positive rate using a p value threshold of less than = 0.05. Using this method, ITS researchers can gain valuable knowledge about their problem sets and feasibly let the ITS automatically identify item order effects and optimize student learning by restricting assigned sequences to those prescribed as most beneficial to learning. (Contains 4 figures, 5 tables, and 3 footnotes.) [Additional funding for this paper was provided by the U.S. Department of Education's Graduate Assistance in Areas of National Need (GAANN). For the complete proceedings, "Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009)," see ED539041.]
Predicting Students Drop Out: A Case Study
Author: Dekker, Gerben W.
Date: 2009-07-00
Type: Reports - Research
Subjects: Information Retrieval,Engineering Education,College Freshmen,Case Studies,Data Collection,Prediction,Predictor Variables,Classification,Foreign Countries,Dropout Rate,At Risk Students,Matrices,Secondary Education,Technical Education

The monitoring and support of university freshmen is considered very important at many educational institutions. In this paper we describe the results of the educational data mining case study aimed at predicting the Electrical Engineering (EE) students drop out after the first semester of their studies or even before they enter the study program as well as identifying success-factors specific to the EE program. Our experimental results show that rather simple and intuitive classifiers (decision trees) give a useful result with accuracies between 75 and 80%. Besides, we demonstrate the usefulness of cost-sensitive learning and thorough analysis of misclassifications, and show a few ways of further prediction improvement without having to collect additional data about the students. Attributes in the pre-university dataset are appended. (Contains 4 tables and 1 footnote.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Detecting and Understanding the Impact of Cognitive and Interpersonal Conflict in Computer Supported Collaborative Learning Environments
Author: Prata, David Nadler
Date: 2009-07-00
Type: Reports - Descriptive
Subjects: Data Analysis,Computer Uses in Education,Cooperative Learning,Speech Acts,Interpersonal Relationship,Conflict,Piagetian Theory,Automation,Grade 6,Elementary School Students,Correlation,Achievement Gains,Student Behavior

This paper presents a model which can automatically detect a variety of student speech acts as students collaborate within a computer supported collaborative learning environment. In addition, an analysis is presented which gives substantial insight as to how students' learning is associated with students' speech acts, knowledge that will significantly influence how this model is utilized by running learning software. Within Piagetian theory, the cognitive conflict of ideas between students is seen as beneficial for learning. Which sorts of interpersonal behaviors lead to most effective learning, however, is open to debate, with some researchers arguing that cooperation is most effective and others arguing that interpersonal conflict is a natural part of collaborative learning. We find that, in fact, interpersonal conflict is associated with positive learning, a finding that must be taken into account, in designing interventions that rely upon detectors of students' speech acts in CSCL environments. (Contains 1 figure and 3 tables.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Using Learning Decomposition and Bootstrapping with Randomization to Compare the Impact of Different Educational Interventions on Learning
Author: Feng, Mingyu
Date: 2009-07-00
Type: Reports - Research
Subjects: Data Analysis,Intelligent Tutoring Systems,Sampling,Statistical Inference,Intervention,Instructional Effectiveness,Learning,Teaching Methods,Hypothesis Testing,Comparative Analysis,Regression (Statistics),Scaffolding (Teaching Technique),Feedback (Response),Grade 8,Middle School Students,Pretests Posttests

A basic question of instructional interventions is how effective it is in promoting student learning. This paper presents a study to determine the relative efficacy of different instructional strategies by applying an educational data mining technique, learning decomposition. We use logistic regression to determine how much learning is caused by different methods of teaching the same skill, relative to each other. We compare our results with a previous study, which used classical analysis techniques and reported no main effect. Our results show that there is a marginal difference, suggesting giving students scaffolding questions is less effective at promoting student learning than providing them delayed feedback. Our study utilizes learning decomposition, an easier and quicker approach of evaluating the quality of ITS interventions than experimental studies. We also demonstrate the usage of computer-intensive approach, bootstrapping, for hypothesis testing in educational data mining area. (Contains 4 tables and 1 footnote.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Automatic Concept Relationships Discovery for an Adaptive E-Course
Author: Simko, Marian
Date: 2009-07-00
Type: Reports - Descriptive
Subjects: Foreign Countries,Data Analysis,Individualized Instruction,Electronic Learning,Online Courses,Automation,Intellectual Disciplines,Computer Science Education,College Instruction,Resource Units,Material Development

To make learning process more effective, the educational systems deliver content adapted to specific user needs. Adequate personalization requires the domain of learning to be described explicitly in a particular detail, involving relationships between knowledge elements referred to as concepts. Manual creation of necessary annotations is in the case of larger courses a demanding task. In this paper we tackle a concept relationship discovery problem that is a step in adaptive e-course authoring process. We propose a method of automatic concept relationship discovery for an adaptive e-course. We present two approaches based on domain model graph analysis. We evaluate our method in the domain of programming. (Contains 2 figures and 1 table.) [Funding was provided by the Cultural and Educational Grant Agency of the Slovak Republic. For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Improving Student Question Classification
Author: Heiner, Cecily
Date: 2009-07-00
Type: Reports - Evaluative
Subjects: Classification,Questioning Techniques,Introductory Courses,Computer Science Education,Accuracy,Automation,College Students,Novices,Programming,Comparative Analysis,Natural Language Processing

Students in introductory programming classes often articulate their questions and information needs incompletely. Consequently, the automatic classification of student questions to provide automated tutorial responses is a challenging problem. This paper analyzes 411 questions from an introductory Java programming course by reducing the natural language of the questions to a vector space, and then utilizing cosine similarity to identify similar previous questions. We report classification accuracies between 23% and 55%, obtaining substantial improvements by exploiting domain knowledge (compiler error messages) and educational context (assignment name). Our mean reciprocal rank scores are comparable to and arguably better than most scores reported in a major information retrieval competition, even though our dataset consists of questions asked by students that are difficult to classify. Our results are especially timely and relevant for online courses where students are completing the same set of assignments asynchronously and access to staff is limited. (Contains 1 figure and 2 tables.) [For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]
Does Self-Discipline Impact Students' Knowledge and Learning?
Author: Gong, Yue
Date: 2009-07-00
Type: Reports - Research
Subjects: Data Analysis,Self Control,Knowledge Level,Learning,Intelligent Tutoring Systems,Grade 8,Students,Questionnaires,Deception,Inferences,Models,Prior Learning

In this study, we are interested to see the impact of self-discipline on students' knowledge and learning. Self-discipline can influence both learning rate as well as knowledge accumulation over time. We used a Knowledge Tracing (KT) model to make inferences about students' knowledge and learning. Based on a widely used questionnaire, we measured students' level of self-discipline. When we analyzed the relation of students' self-discipline with their knowledge attributes, we found that high self-discipline students had significantly higher initial knowledge, but there is no consistent relationship of learning while using the tutor. Moreover, higher self-discipline students seemed more careful with respect to making careless mistakes. (Contains 5 figures and 5 tables.) [Additional funding was provided by the Fulbright Program. For the complete proceedings, &quot;Proceedings of the International Conference on Educational Data Mining (EDM) (2nd, Cordoba, Spain, July 1-3, 2009),&quot; see ED539041.]