Creating Equitable Grading Practices with Rubrics: A Teaching Assistant Training Activity

Manually grading coding assignments in large computer science (CS) classes is a challenging logistical task. The evaluation of code correctness is subjective, leading to grading bias and inconsistencies. This problem is exacerbated when multiple teaching assistants (TAs) grade different submissions of the same problem. One potential solution to these challenges is the use of rubrics, which offer a structured approach to grading by adopting specific criteria for the quality of student work [1, 2]. They can help mitigate instructors' implicit biases and minimize grading variance even when multiple raters are involved [2, 3]. Moreover, rubrics can motivate students to improve their work [4], enhance their learning in CS courses [4], and provide an objective evaluation of multimedia projects [5]. However, validity, reliability, and fairness issues still apply to their use [6]. To account for the subjective nature of coding, assigning partial points based on conceptual understanding rather than just the final output promotes equitable grading since the final output may not always reflect the student's grasp of the topics [6]. Therefore, rubrics must be designed to be simple, specific, and easy to understand for instructors, graders, and students. In this study, we conducted a hands-on TA training workshop to explore the effectiveness of rubrics in grading coding assignments in a core undergraduate CS course. The workshop provided participants with rubrics and rater training to assess the impact of rubrics on grading bias and consistency. Additionally, the workshop introduced the use of partial points based on students' conceptual understanding, aiming to promote fair and objective grading while also addressing the limitations of this approach. The workshop was conducted for distinct groups of TAs, totaling 50 participants. Firstly, participants were given two sample incorrect solutions to a problem and asked to grade them at their own discretion. In the second round, they re-graded the solutions using a rubric with assigned points to different solution components. The results of the two grading rounds were reported anonymously, and a real-time graph displayed the grade distribution. Statistical analysis revealed a significant reduction in grade variance for both student groups, from 5.55 to 0.66 and from 4.28 to 1.11, respectively, on a scale from 1 to 10. This quantitative and visual analysis helped TAs understand the importance of rubrics and enabled them to discuss biases and best practices for developing and deploying rubrics. TAs' informal feedback indicated the workshop's effectiveness in demonstrating the relevance of creating good rubrics. The workshop design expands on previous workshops for other disciplines by showcasing the unique aspects of grading in computer science, such as the different approaches to solving coding problems and the introduction of visualizations to understand grading variances. This workshop showcases the application of rubrics and quantitative data visual analytics to promote equitable grading practices and underscore the significance of ethical issues in CS education training. Inspired by rubric-utilizing workshops in math, chemistry, and biology, we have innovated by accommodating the varied problem-solving paths inherent in coding, thus emphasizing conceptual over procedural grading. Unique to our approach is the integration of a data visualization component to help understand grading variance. This workshop's design can be extended to various pedagogical topics, allowing real-time analysis of small-scale experiments, thus broadening its applicability and shaping future teaching strategies.