ethanporcaro/eric-fulltext-meta huggingface dataset:
&quot;What Do I Do with the Data Now?&quot;: Analyzing Assessment Information for Accountability and Improvement
Author: Pieper, Suzanne L.
Date: 2008-00-00
Type: Journal Articles
Subjects: College Outcomes Assessment,Accountability,Educational Research,Student Educational Objectives,Academic Achievement,Student Participation,Educational Indicators,Competence,Expectation,Student Improvement,Educational Strategies,Educational Benefits,General Education,Case Studies

Most colleges and universities have implemented an assessment program of some kind in an effort to respond to calls for accountability from stakeholders as well as to continuously improve student learning on their campuses. While institutions administer assessment instruments to students and receive reports, many campuses do not reap the maximum benefits from their assessment efforts. Oftentimes, this is because the data have not been analyzed in a way that answers questions that are important to the institution or other stakeholders. This paper describes four useful analytical strategies that focus on the following key educational research questions: (a) Differences: Do students learn or develop more if they participate in a course or program compared to other students who did not participate?; (b) Relationships: What is the relationship between student assessment outcomes and relevant program indicators (e.g., course grades, peer ratings)?; (c) Change: Do students change over time?; and (d) Competency: Do students meet our expectations? Each of these strategies is described, followed by a discussion of the advantages and disadvantages of each method. These strategies can be effectively adapted to the needs of most institutions. Examples from the general education assessment program at James Madison University are provided.
Exploring the Success of Transfer Programs for Community College Students
Author: Blaylock, Reginald S.
Date: 2011-00-00
Type: Journal Articles
Subjects: Community Colleges,Two Year College Students,College Transfer Students,Transfer Programs,Transitional Programs,Summer Programs,Outcome Measures,Program Effectiveness,Student Evaluation

The state's educational systems must collaborate together to enable transfer students to gain the necessary skills that support degree completion strategies. Given the current economic state, an investment in California community college transfer students in order to provide the best possible university transition would seem wise and fiscally responsible. This outcomes-based assessment evaluation focused on assessing outcomes of a new transitional summer program, called Transfer Bridge; designed to aid underrepresented community college students transfer to a public comprehensive regional university. The results showed that successful transfer programs must be customized for the transferring students they are intended to serve in order to effectively address their transition needs to the university.
Reflective (Ac) Counting: Institutional Research, Evaluation, &amp; Assessment in a Time of Cholera
Author: Heinecke, Walter F.
Date: 2011-00-00
Type: Journal Articles
Subjects: College Outcomes Assessment,Retrenchment,Accountability,Educational Quality,Efficiency,Excellence in Education,Educational Policy,Politics of Education,Institutional Research

Institutional researchers (IR) and assessment professionals in higher education are living in interesting and challenging times, one might say in times of crisis. As federal and state coffers have shrunk in a time of rising costs of higher education, university budgets are tightening while at the same time calls for accountability are increasing (U.S. Department of Education, 2006). The discourse of higher education quality has changed to focus on &quot;return on investment&quot; as the criteria for college and university success. As a result, institutional researchers are caught in a conflicting press for efficiency, accountability and excellence. The research on the role of IR professionals reflects more emphasis on &quot;accountability and performance&quot; and &quot;outcomes assessment&quot; among other job tasks (Volkwein, 2008, p. 9). This article attempts to offer perspective related to the &quot;issues intelligence&quot; as well as the &quot;contextual intelligence&quot; that make up the &quot;organizational intelligence&quot; of IR (Volkwein, 2008, p. 5; see also Terenzini, 1999).
An Integrated Model of Influence: Use of Assessment Data in Higher Education
Author: Jonson, Jessica L.
Date: 2014-00-00
Type: Journal Articles
Subjects: Higher Education,Evaluation Utilization,Evidence,Academic Achievement,Content Analysis,Definitions,Use Studies,Models,Influences,Educational Theories,Research Reports,Qualitative Research,Meta Analysis

A fundamental goal of student learning outcomes assessment in higher education is to use student learning evidence in decision making to improve educational programs. Such use of assessment findings, however, is atypical. This article argues that a narrow conception of use contributes to this conclusion and an accurate appraisal of the contribution of assessment requires a reconceptualization of the aims of assessment and a more inclusive model of possible uses of assessment evidence. To evaluate the heuristic value of a more inclusive model of influence adapted from the field of evaluation, a content analysis was undertaken of program assessment reports at a research university. Results indicate that existing definitions of use suffer from construct underrepresentation; assessment evidence may be more influential than realized, particularly with regard to enhancing understanding of how student learning occurs; and the more inclusive model has potential utility for faculty, administrators, and accrediting bodies.
Data Needed for Improving Productivity Measurement in Higher Education
Author: Massy, William F.
Date: 2012-00-00
Type: Journal Articles
Subjects: Productivity,Higher Education,Educational Principles,Barriers,Educational Practices,Data,Research Needs,Models,Measurement Techniques,Change Strategies,Program Improvement,Input Output Analysis

William Massy is an emeritus professor and former Vice President for Business and Finance at Stanford University, Teresa Sullivan is President of the University of Virginia, and Christopher Mackie is a Study Director with the National Academies' Committee on National Statistics. This article summarizes the authors' recent National Research Council report, &quot;Improving Measurement of Productivity in Higher Education,&quot; which reviews the principles and pitfalls of measuring university productivity and proposes a practical method for doing so at the sector and institutional segment levels. The summary emphasizes the method's data requirements and describes needed changes in IPEDS and other databases.
Technology for Mining the Big Data of MOOCs
Author: O'Reilly, Una-May
Date: 2014-00-00
Type: Journal Articles
Subjects: Data,Technology Uses in Education,Online Courses,Open Education,Standard Setting,Data Analysis,Data Processing,Open Source Technology,Visualization,Information Management,Computer System Design,Technological Advancement,Program Descriptions,Performance Technology,Higher Education

Because MOOCs bring big data to the forefront, they confront learning science with technology challenges. We describe an agenda for developing technology that enables MOOC analytics. Such an agenda needs to efficiently address the detailed, low level, high volume nature of MOOC data. It also needs to help exploit the data's capacity to reveal, in detail, how students behave and how learning takes place. We chart an agenda that starts with data standardization. It identifies crowd sourcing as a means to speed up data analysis of forum data or predictive analytics of student behavior. It also points to open source platforms that allow software to be shared and visualization analytics to be discussed.
A Model for Outcomes Assessment of Undergraduate Science Knowledge and Inquiry Processes
Author: Puncochar, Judith
Date: 2013-00-00
Type: Journal Articles
Subjects: Undergraduate Students,Models,Science Achievement,Inquiry,General Education,Outcome Measures,Scoring Rubrics,Sampling,Test Construction,Psychometrics,Likert Scales,Rating Scales,Student Evaluation,Evaluation Research,Educational Practices

The goals of a Liberal Studies education are designed to prepare citizens to live responsible, productive, and creative lives in a changing world. Ideally, a liberal education fosters well-grounded intellectuals with dispositions toward learning and an acceptance of responsibility regarding their ideas and actions. To measure the efficacy of a Liberal Studies education, a Midwestern regional university developed a systematic, rubric-guided assessment based on nationally recognized science principles and inquiry processes to evaluate student work in undergraduate science laboratory courses relative to a liberal education. The rubric presented a direct measure of student understandings of science inquiry processes. The assessment procedure used stratified random sampling at confidence levels of 95% to select student work, maintained anonymity of students and faculty, addressed concerns of university faculty, and completed a continuous improvement feedback loop by informing faculty of assessment results to assess and refine science-inquiry processes of course content. The procedure resulted in an assessment system for benchmarking science inquiry processes evident in student work and offered insights into the effect of undergraduate science laboratory courses on student knowledge and understanding of science inquiry. The current assessment was without additional burdening of faculty or supplementary testing of students.
Examining College Students' Gains in General Education
Author: Pastor, Dena A.
Date: 2007-00-00
Type: Journal Articles
Subjects: College Students,General Education,Pretests Posttests,United States History,History Instruction,Politics,Hierarchical Linear Modeling,Scores,Required Courses,Statistical Analysis,Multiple Choice Tests,Effect Size,Models

Do students change as a result of completing their general education requirement? This question was examined by using a pretest/posttest design with five different cohorts of students required to complete a general education program in American history and politics. Differences among various groups in Cohen's d (the standardized difference between pretest and posttest means) were examined using hierarchical linear modeling. Results indicated that differences could be explained by how the requirement was fulfilled; negligible gains were found for students using advanced placement or transfer credit (0.04-0.18), whereas moderate/large gains were found for students who had completed the university course(s) (0.42-0.90). The gain found for students yet to fulfill the requirement (0.28) was explained by the large presence in that group of students currently enrolled in the course. Different definitions of d used with the pretest/posttest design are described and implications of the results for assessment are discussed.
Assessment at North Carolina State University: Adapting to Change in the Workplace
Author: Bresciani, Marilee J.
Date: 2009-00-00
Type: Journal Articles
Subjects: State Universities,College Faculty,Tenure,Resistance to Change,Educational Assessment,Vocational Adjustment,Theories,Undergraduate Students,Student Evaluation

Effectively introducing change in job responsibilities, particularly when dealing with tenured faculty, can be challenging. More often, additions or changes to work tasks, such as integrating assessment procedures into existing work tasks, requires employees to apply new and/or more complex knowledge, skill, and ability. When compared to organizations practicing contemporary-type work methods, institutions practicing traditional-type work methods, such as those common to traditional university settings, can find adaptation to change particularly onerous. For example, tenured faculty may perceive introductions of new concepts or new terminology as substantive changes in their practice, even though the change is an introduction of new labels to their current practice or a systematization of a former practice. Consequently, the integration of new assessment procedures, as in this instance, can have a significant impact on faculty when learning to accommodate that change. Therefore, understanding why long-tenured employees may be particularly resistant to change in the workplace is important when adding assessment procedures to existing work responsibilities. To better understand faculty resistance to change and to help facilitate the change process, one can apply the integration of work adaptation theory. This paper reviews concepts included in the theory of work adaptation, with a focus on work adaptation theory developed by Petrini and Hultman. Petrini and Hultman cite six common beliefs that lie at the root of employee resistance to change and provide strategies for addressing such resistance. The six common beliefs include the following: (a) One's needs are currently met by the traditional methods already in place, (b) The change will make it more difficult to meet one's needs, (c) The risks involved outweigh the possible benefits, (d) There is no basis for the change--it's just another plan to get more work out of us with fewer resources, (e) The organization is mishandling the change, and (f ) The change will fail and go away. This paper addresses issues related to employee resistance when incorporating undergraduate assessment into the culture of a Research Extensive institution. Discussed are experiences in confronting Petrini and Hultman's six beliefs when working with tenured employees as well as the application of strategies they suggest when addressing employee resistance to change. Furthermore, the six beliefs and strategies are applied as a means to clarify key findings with regard to the institution's successful implementation of changes designed to improve student learning.
The Future of Data-Enriched Assessment
Author: Thille, Candace
Date: 2014-00-00
Type: Journal Articles
Subjects: Data,Online Courses,Electronic Learning,Case Studies,Educational Environment,Intelligent Tutoring Systems,Open Education,Interaction,Evaluation,Testing,Data Collection,Information Utilization,Inferences,Higher Education,Educational Practices,Regression (Statistics),Surveys

The article addresses the question of how the assessment process with large-scale data derived from online learning environments will be different from the assessment process without it. Following an explanation of big data and how it is different from previously available learner data, we describe three notable features that characterize assessment with big data and provide three case studies that exemplify the potential of these features. The three case studies are set in different kinds of online learning environments: an online environment with interactive exercises and intelligent tutoring, an online programming practice environment, and a massive open online course (MOOC). Every interaction in online environments can be recorded and, thereby, offer an unprecedented amount of data about the processes of learning. We argue that big data enriches the assessment process by enabling the continuous diagnosis of learners' knowledge and related states, and by promoting learning through targeted feedback.
An Ethically Ambitious Higher Education Data Science
Author: Stevens, Mitchell L.
Date: 2014-00-00
Type: Journal Articles
Subjects: Information Management,Information Science,Ethics,Higher Education,Educational Assessment,College Students,Data

The new data sciences of education bring substantial legal, political, and ethical questions about the management of information about learners. This piece provides a synoptic view of recent scholarly discussion in this domain and calls for a proactive approach to the ethics of learning research.
Insight and Action Analytics: Three Case Studies to Consider
Author: Milliron, Mark David
Date: 2014-00-00
Type: Journal Articles
Subjects: Case Studies,Communities of Practice,Data Analysis,Higher Education,Predictive Measurement,Academic Achievement,Graduation

Civitas Learning was conceived as a community of practice, bringing together forward-thinking leaders from diverse higher education institutions to leverage insight and action analytics in their ongoing efforts to help students learn well and finish strong. We define insight and action analytics as drawing, federating, and analyzing data from different sources (e.g., ERP, LMS, CRM) at a given institution to produce deep predictive flow models of student progression and completion coupled with applications (apps) that take these data and bring them to advisors, students, faculty, and administrators in highly consumable/ useable ways. Through three case studies, this article provides a closer look at this iterative work unfolding in diverse institutions, addressing diverse student success challenges, and achieving significant positive results on student progression. The article underscores a key finding: there is not a one-size-fits-all predictive model for higher education institutions. We conclude with a discussion of key findings from these cases and observations to inform future related work.
Student Employee Development in Student Affairs
Author: Athas, Christina
Date: 2013-00-00
Type: Journal Articles
Subjects: Student Personnel Workers,Student Personnel Services,Staff Development,Online Surveys,Student Employment,Part Time Employment,Program Length,Status,Group Dynamics,Cultural Awareness,Predictor Variables,Student Development,Transfer of Training,Career Readiness,Skill Development,Minimum Competencies,College Students,Student Attitudes,Regression (Statistics),Interpersonal Competence,Wellness,Self Efficacy

Employment within student affairs divisions offers environments in which students can apply the knowledge they have gained, as well as acquire new competencies, helping them to build solid foundations for their futures. Researchers used an online survey to assess the outcomes associated with part-time student employment within the student affairs division at a large Midwest university. Results show duration of employment, rank, sense of community, civic engagement, and cultural awareness to be strong predictors of student development in preparation for their futures.
ACES: The Development of a Reliable and Valid Instrument to Assess Faculty Support of Diversity Goals in the United States
Author: Ng, Jennifer
Date: 2013-00-00
Type: Journal Articles
Subjects: College Faculty,Diversity (Institutional),Teacher Attitudes,Attitude Measures,Test Construction,Test Reliability,Test Validity,Research Universities,Factor Analysis

Diversity is an increasingly important value for institutions of higher education. Yet, few measures exist to assess whether college and university faculty share in this objective and how their beliefs relate to specific aspects of their work. In this study, we gathered data from a sample of faculty at one American research university to develop a valid and reliable instrument useful for exploring how commitments to diversity are reflected in teaching, research and service. The resulting instrument, ACES, assesses four factors: (a) Attitude towards diversity, (b) Career activities and professional norms, (c) Environment conducive to diversity, and (d) Social interactions with diverse groups. Evidence for the validity and reliability of the scores produced by ACES is presented. How this psychometrically-sound instrument might benefit higher education research and practice in the assessment of diversity related goals is also considered.
Identifying What Student Affairs Professionals Value: A Mixed Methods Analysis of Professional Competencies Listed in Job Descriptions
Author: Hoffman, John L.
Date: 2012-00-00
Type: Journal Articles
Subjects: Student Personnel Workers,Occupational Information,Competence,Mixed Methods Research,College Administration,Administrators,Work Experience,Educational Experience

This mixed method study explored the professional competencies that administrators expect from entry-, mid-, and senior-level professionals as reflected in 1,759 job openings posted in 2008. Knowledge, skill, and dispositional competencies were identified during the qualitative phase of the study. Statistical analysis of the prevalence of competencies revealed significant differences between major functional areas and requirements for educational and work experience. Implications for institutional leaders, graduate faculty, and professional development planning as well as for mixed methods research are discussed.
Fair and Equitable Measurement of Student Learning in MOOCS: An Introduction to Item Response Theory, Scale Linking, and Score Equating
Author: Meyer, J. Patrick
Date: 2013-00-00
Type: Journal Articles
Subjects: Online Courses,College Students,Student Evaluation,Learning,Item Response Theory,Equated Scores,Prevention,Cheating,Scores

Massive open online courses (MOOCs) are playing an increasingly important role in higher education around the world, but despite their popularity, the measurement of student learning in these courses is hampered by cheating and other problems that lead to unfair evaluation of student learning. In this paper, we describe a framework for maintaining test security and preventing one form of cheating in online assessments. We also introduce readers to item response theory, scale linking, and score equating to demonstrate the way these methods can produce fair and equitable test scores. Patrick Meyer is an Assistant Professor in the Curry School of Education at the University of Virginia. He is the inventor of jMetrik, an open source psychometric software program. Shi Zhu is a doctoral student in the Research, Statistics, and Evaluation program in the Curry School of Education. He holds a Ph.D. in History from Nanjing University in China.
Combining Learning and Assessment to Improve Science Education
Author: Linn, Marcia C.
Date: 2011-00-00
Type: Journal Articles
Subjects: Science Education,Lecture Method,Student Evaluation,Concept Mapping,High Stakes Tests,Learning Activities,Standardized Tests,Lifelong Learning,Visualization,Essays,Online Courses,Teaching Methods,Evaluation Methods,Scientific Concepts,Educational Technology

High-stakes tests take time away from valuable learning activities, narrow the focus of instruction, and imply that science involves memorizing details rather than understanding the natural world. Current tests lead precollege instructors to postpone science inquiry activities until after the last standardized test is completed--often during the last week of school. Students spend countless hours practicing and taking multiple-choice tests that have little educational value. Learning tests that help students understand science and measure progress at the same time are needed. This article discusses the following topics regarding science education: (1) Learning Test Goals; (2) Teaching and Assessing Lifelong Learning; (3) Visualizations and Assessment; (4) Teaching and Assessing with Concept Maps; (5) Essay Questions, Learning, and Assessment; and (6) Improving Assessment in Lecture Classes. Valuable classroom time can be reclaimed by using online learning environments that incorporate learning tests to measure lifelong learning skills. Think about what would happen if scientists spent time memorizing new facts rather than investigating compelling problems. A focus on lifelong learning to the classroom needs to be restored to retain a competitive advantage in science.