Domain: Educational Services
Question:
The university is implementing a new policy requiring rubrics for all major graded assignments including final exams, capstone projects, research papers, and presentations worth 20% or more of the course grade. The Center for Excellence in Teaching prepared this comprehensive guide to support faculty during this transition. As Associate Provost for Academic Affairs, you have been asked by the Provost to review the document before it is distributed university-wide to ensure it will effectively support diverse faculty across all disciplines in successfully adopting this new assessment requirement. Please answer the following questions to improve the document's effectiveness and ensure successful implementation.                

The document provides analysis of creative or open ended assignments and acknowledges that analytic rubrics "can limit student creativity". Does the document offer sufficient guidance for addressing this concern in sections like "Why Use a Rubric" or "Research"? Many faculty across the humanities and liberal arts are concerned that detailed rubrics will hinder creative thinking and expression in the demonstration of competency in the classroom. How should the document be refined to help faculty design flexible rubrics that maintain clear objectives for students? What specific strategies or examples would address this pedagogical tension for assignments that encourage intellectual risk-taking in the classroom?

Also: How does the document address converting categorical values to numerical grades? Additionally, how does the document guide teachers in handling "edge" cases where students are directly in the middle of two categories in a particular rubric? The document provides numerous examples of rubrics where categories are labeled "exceeds expectations" or "meets expectations". Does the document sufficiently address how these categories, on average should convert to the university grading scale? Furthermore, if a student's submission falls directly in between these category with respect to other submissions, does the document offer sufficient guidance in terms of how to consistently handle these cases and establish fair performance thresholds?

Lastly: How should the document be revised to better integrate research findings? There are three citations on page 30 that are not discussed throughout that focus on academic performance, building rubrics, and assessment. These sources would be very helpful to discuss and cite throughout the document. What claims in the document are made without support or evidence? For instance, the authors claim that rubrics "can help reduce unconscious bias" but provide no citation. Is the research section at the end of the document sufficient in your opinion? What specific information from the existing citations (e.g. Panadero, E., Jonsson, A., Pinedo, L., & Fernández-Castilla, B. (2023). Effects of Rubrics on Academic Performance, Self-Regulated Learning, and self-Efficacy: A Meta-analytic Review. Educational Psychology Review, 35(4), 113-. USC Libraries link.) or additional research should be included to help faculty use the document's existing sections (e.g. why use a rubric, types of rubrics, examples of rubrics) more effectively? Is there information from outside the university that the authors could include to make the document more comprehensive?

Expected points in a correct answer:
Identifies both strengths and weaknesses of the existing document, noting what it currently covers (e.g., rubric types, tradeoffs) and what practical guidance is missing
Evaluates the document's existing guidance on rubric types and their tradeoffs, noting what it does well and where it falls short in helping faculty select appropriate rubrics
Discusses the tradeoffs between rubric structure and flexibility for creative assignments, noting that analytic rubrics can feel restrictive while single-point or holistic rubrics allow more openness
Evaluates how the document guides faculty in handling edge cases where a student's work falls directly between two rubric performance categories
Discusses how to integrate research findings already found in document by discussing key takeaways from those sources
Addresses the creativity concern with grading rubrics by stating that analytic rubrics can feel prescriptive while single point rubrics can be a less restrictive alternative
Recommends including a decision matrix or similar guide that helps faculty choose between analytic, single-point, and holistic rubric types
Recommends specific document revisions such as pairing each rubric type with 'when to use' guidance, step-by-step build instructions, or worked examples to help faculty design flexible rubrics
Identifies at least one specific strategy that addresses the tension between rubric structure and intellectual risk-taking in assignments that are open-ended or creative in nature by suggesting incorporating approaches like rubric criteria for creativity and risk taking explicitly
Provides at least one concrete example, such as a sample criterion or revised rubric language, that could be added to the document to help manage creative assignments
Evaluates how the document currently handles the conversion of categorical rubric labels to numerical grades by pointing to where this guidance exists or where it falls short
States that the document does not show how labels like 'exceeds expectations' or 'meets expectations' convert to letter grades, and recommends adding a mapping example
Recommends specific mechanisms for handling edge cases, such as calibration steps for graders or written threshold rules for borderline work
Clearly defines an "edge case" as when a student's work has fallen directly between two performance categories making the assignment of one overall grade difficult
Takes a clear stance on whether the research section at the end of the document is sufficient for its intended faculty audience, explaining what it does and does not accomplish
Recommends how the citations at the end of the document should be integrated into the existing document by providing clear steps for placing citations into existing sections
Identifies at least one specific claim in the document that lacks citation, indicating what type of evidence would be needed to substantiate it
Recommends at least one type of external source or outside information that could be added to make the document more comprehensive for faculty across disciplines
States that the document explains rubric types and their tradeoffs adequately but lacks practical guidance such as a decision framework to help faculty choose the right type for their specific assignment or discipline
States that teachers want clear expectations
Advises that students want to think creatively and take risks
Identifies that the document lacks sufficient practical guidance for choosing and shaping rubrics that both protect creativity and maintain clear expectations
States that the document partially addresses limits to student creativity
Notes that the document warns analytic rubrics can be prescriptive and may limit creativity, and suggests discussing criteria with students
Recommends organizing the document so each rubric type is paired with purpose, build instructions, and examples to improve usability
Recommends showing annotated samples of high level creative work
Recommends showing annotated samples of weak work with information on how rubric criteria were not met
Recommends self-review steps where students apply the rubric to sample work or early drafts
Advises the importance of using a norming approach so interpretations stay consistent
Recommends adding a short mapping example showing how rubric categories convert to point values and then to letter grades
Recommends adding a clear rule or process for consistently placing 'in-between' work within a point range, such as criteria-based threshold rules
Notes that the document explains point ranges can provide flexibility when scoring but does not show how this translates into actual grades
Recommends including a brief calibration step for borderline cases
Recommends including written thresholds that explain when to bump or stay within particular rubric categories
Recommends summarizing key research findings and best practices in accessible language within the body of the document
Advises use of rubrics early in the semester
Advises against vague or ambiguous criteria due to scoring difficulty
Recommends concrete verbs in rubric language
Recommends splitting big ideas into separate rows in rubric design
Recommends testing the draft on a sample of student work
Recommends avoiding misalignment by only grading for what is taught in the classroom
Recommends including information on how to include rubrics in the university's learning management software
Recommends adding a specific example showing how a student scoring between two rubric levels should be placed within a point range (e.g., "if a student satisfies all required criteria of the lower level and at least half of the next level's criteria, place the score in the upper half of the lower level's range")
Recommends that the document include guidance on explaining rubric-to-grade conversion in the syllabus so students understand how rubric scores become grades
Recommends an in-class review of the rubric to help students understand expectations and use the rubric as a guide while they work
Recommends adding parenthetical citations or footnotes after specific claims in the "Why Use a Rubric" section (e.g., claims about saving time, improving consistency, clarifying expectations)
Notes that rubrics improve consistency and fairness when criteria are clear and performance levels are distinct
Recommends that the document connect each research-based best practice with a one-sentence actionable step for faculty
Recommends recording final calibration decisions so graders can reference them during grading