DOI: 10.7280/d1nh6n
title: How syllabi relate to outcomes in higher education: An evaluation of syllabi learner-centeredness and grade inequities in STEM

description: Fostering equity in undergraduate science, technology, engineering, and
 mathematics (STEM) programs can be accomplished by incorporating
 learner-centered pedagogies, resulting in the closing of opportunity gaps
 (defined in this research as the difference in grades earned by
 minoritized and non-minoritized students). We assessed STEM courses that
 exhibit small and large opportunity gaps at a minority-serving,
 research-intensive university, and evaluated the degree to which their
 syllabi are learner-centered, according to a previously validated rubric.
 We specifically chose syllabi as they are often the first interaction a
 student has with a course and can serve to establish expectations for
 course policies and practices. We found that STEM courses with more
 learner-centered syllabi had smaller opportunity gaps. The syllabus rubric
 factor that most correlated with smaller opportunity gaps was Power and
 Control, which reflects the Student's Role, Outside Resources, and
 Syllabus Focus. This work highlights the importance of course syllabi as a
 tool for instructors to create more inclusive classroom environments, and
 the need for instructors to be supported by their institutions to create
 learner-centered courses.
description: This dataset is composed of rubric scores for 50 course syllabi
 of STEM classes in a research-intensive university with a large population
 of minoritized students as well as some institutional data (here defined
 as African-American, Latinx, Pacific Islander, and American Indian). We
 wanted to examine the relationship between racial grade gaps (here labeled
 as opportunity gaps) and the degree of learner-centeredness of the syllabi
 since course syllabi are good representations of classroom pedagogy
 according to the previous literature. The 50 syllabi were evaluated with a
 previously validated and peer-reviewed rubric designed by Cullen and
 Harris in 2009 and published in <em>Assessment and Evaluation in
 Higher Education</em> journal. The rubric measures the degree of
 learner-centeredness of syllabi. It has 13 items categorized under 3
 factors plus the number of pages of the syllabi. We have modified the
 rubric to be on a 5-point scale (0-4). Zero represents the lowest degree
 of learner-centeredness and 4 shows the highest. The dataset includes the
 scores for the 13 rubric items, 3 factors, and the length of the syllabi
 (number of pages). Syllabus factors are calculated as the average of the
 rubric items that belong to a certain factor. The 50 syllabi were divided
 into 2 groups: Syllabi of courses with small opportunity grade gaps and
 those with large gaps (0 or 1 under the 1st column labeled small
 opportunity gaps). The gaps are calculated as the differences between
 course grades received by minoritized students vs non-minoritized ones on
 a 4.0 scale. All variable names in the dataset are clear and easy to
 understand. The dataset also includes institutional data such as
 information regarding instructor type, the number of minoritized students
 (PEERs), and non-minoritized students (non-PEERs) in a class.
 <strong>Methods</strong> Course
 syllabi were requested from the instructors by the study team. Instructors
 were contacted via email (as well as two follow-up emails for
 non-respondents). All instructors who responded were provided with the
 study information sheet, gave electronic consent for participation, and
 sent their course syllabi to a member of the research team. The syllabi
 were de-identified before analysis by eliminating any identifying names
 (of instructors, teaching assistants, and readers), emails, and course
 numbers to decrease bias during the coding process. We used a modified
 version of Cullen and Harris’s (2009) rubric to code the collected
 syllabi. There are 13 items included in the rubric: (1) Accessibility of
 Teacher, (2) Learning Rationale, (3) Collaboration, (4) Teacher’s Role,
 (5) Student’s Role, (6) Outside Resources, (7) Syllabus Tone, (8) Syllabus
 Focus, (9) Grades, (10) Feedback Mechanism, (11) Evaluation, (12) Learning
 Outcomes, and (13) Revision/Redoing. The 13 items are categorized under
 three factors: (1) Community, (2) Power and Control, and (3)
 Evaluation/Assessment. <strong>Data Processing
 and Analysis</strong> Syllabi were separated into
 cases and controls based on the size of the opportunity gap between
 minoritized and non-minoritized students for the respective
 course-instructor pairing.  The difference in the grade points (on a 4.0
 scale) received by minoritized and non-minoritized students is denoted as
 an opportunity gap. Syllabi in the large opportunity gap group (25th
 percentile) were designated as cases and syllabi in the small opportunity
 gap group (75th percentile) were identified as the controls. We used a
 logistic regression model using the stats and MASS package in R and
 performed the best subsets logistic regression using the bestglm package
 in R. We used a previously validated rubric to measure the degree of
 learner-centeredness of the syllabi. Two members of our group coded the
 syllabi after a 3-hour training session in which they reached 100%
 consensus. <br>The syllabi used for training were not used for study
 data. They scored the syllabi in sets of 10 and calculated the interrater
 reliability. There were calibration and consensus sessions after scoring a
 set of 10 syllabi. In only 3 cases where there was disagreement between
 the two main raters, a third rater also scored the syllabi. The
 Cohen's weighted kappa was calculated as a measure of the degree of
 interrater-reliability, provided in the preprint. The syllabi were
 deidentified before coding for fair scoring. The correlation matrix for
 the 13 syllabus items and also the 3 syllabus factors are provided in the
 preprint. Cronbach's alpha for the 13 rubric items and also for the 3
 rubric factors was calculated and is provided in the preprint.
description: # How syllabi relate to outcomes in higher education: An evaluation of
 syllabi learner-centeredness and grade inequities in STEM
 [https://doi.org/10.7280/D1NH6N](https://doi.org/10.7280/D1NH6N) ##
 DATA-SPECIFIC INFORMATION ###
 Eslami\_2022\_How\_Syllabi\_Relate\_to\_Outcomes\_in\_Higher\_Education.csv Number of variables: 26 Number of cases/rows: 50 Variable names with descriptions and/or their values in parenthesis: small_opportunity_gap; delta_GP (average grade point difference between minoritized and non-minoritized students in a STEM course); additional_item_Length_of_Syllabus (number of pages for each syllabus); For the definition of rubric items, refer to the rubric designed by Cullen & Harris in 2009 published in Assessment and Evaluation in Higher Education journal; Rubric items are scored on a 5-point scale (0, 1, 2, 3, and 4): rubric_item_Accessibility_of_Teacher; rubric_item_Learning_Rationale; rubric_item_Collaboration; rubric_item_Teachers_Role; rubric_item_Students_Role; rubric_item_Outside_Resources; rubric_item_Syllabus_Tone; rubric_item_Syllabus_Focus; rubric_item_Grades; rubric_item_Feedback_Mechanisms; rubric_item_Evaluation; rubric_item_Learning_Outcomes; rubric_item_Revision The 3 rubric factors are calculated as the average of the rubric items categorized under each factor in the rubric: rubric_factor_Community; rubric_factor_Power_and_Control; rubric_factor_Evaluation_and_Assessment (Community is the average of rubric items 1-3, Power and Control is the average of rubric items 4-8, and Evaluation/Assessment is the average of rubric items 9-13) rubric_score_total; instructor_type (research faculty, teaching faculty, and lecturers); discipline (type of STEM discipline); average_PEER_representation (PEER= minoritized students); average_number_PEER; average_number_non_PEER (non-PEER= non-minoritized students); average_number_students PEER = minoritized students including Black, Latinx, Pacific Islander, and American Indian students; non-PEER = non-minoritized students including White and Asian