Imagine student A submits an upper-level writing assignment. One Graduate Student Instructor (GSI) gives the writing an A, while another GSI assigns it a C+. How could this happen? Inconsistencies like this can happen when the GSIs have different ideas and assumptions about the grading criteria. This is why it is important for instructors to work with all GSIs across a course to standardize grading practices and ensure consistency and fairness in evaluating student work. The best way to accomplish this is with a grade-norming session.

Norming is a process where lead instructors and GSIs come together to standardize grading practices. During a norming session, instructors work with GSIs to discuss and align their understanding of grading criteria and rubrics by grading sample assignments and comparing their evaluations. The objective is to evaluate how well the GSIs’ assessments align with the grading criteria established by the instructor. The most meaningful discussion occurs when GSIs explain their grading decisions, especially those that significantly deviate from the norm. This dialogue enables GSIs to explore their interpretations of the grading criteria and helps them better align their understanding with the instructor's intended standards. This collaborative approach helps reduce subjective biases and ensure that all students are assessed based on the same standards to promote equity.

How To Run a Grade-Norming Session

Preparing for a norming session 

Select an assignment:

• Choose an assignment that is the same or similar to the one being assessed. It should be familiar to all GSIs or one they can quickly understand.
• Make copies of the detailed assignment sheet for all instructors involved.

Create or review a grading rubric:

• Provide each GSI with a grading rubric or guide for the assessment.

Obtain sample student work:

• Gather at least two completed assignments, done by actual students.
• Remove all identifying information about students, instructors, and courses.
• Make enough copies of these samples for all participants to grade.

During the norming session

Introduce the assignment and rubric:

• Spend a few minutes ensuring all GSIs understand the assignment and rubric or grading guide. GSIs must review the entire rubric, especially if they did not help create it. Any concerns or questions about definitions, performance indicators, or descriptors should be discussed and clarified to ensure a shared understanding.
• Clarify any questions that may arise, as these could also be questions students might have.

Grade and comment on samples:

• Allow GSIs to review/read the samples and grade and comment on the samples independently. They should not be communicating with each other at this time or know how other GSIs are scoring the work.
•  It might be helpful to work with one assignment at a time so GSIs can apply new insights to subsequent samples.

Survey grades:

• Collect and review the grades assigned by GSIs to get an overall impression of the grading consistency.
• Give GSIs the rubrics or guides they scored on. Review how each GSI scored by going around the table and hearing the reasoning behind how they scored each criteria.
• Highlight similarities and differences before initiating discussions. The goal is to help GSIs align with the lead instructors expectations.

Discuss and align grading standards:

• Facilitate a discussion to achieve consensus on the correlation between the rubric criteria and student performance.
• Identify if discrepancies are due to misunderstanding of the rubric or if the rubric itself needs adjustment.
• Discuss possible ambiguities in the assignment description or rubric and address any subjective biases.

Norming across sections in larger courses:

• Ensure all graders apply the same standards to maintain fairness across different sections.
• Use norming sessions to practice and possibly revise the rubric based on real-life scenarios.

Evaluate and tweak rubrics or guides:

• Discuss the practical application of the rubric, and adjust criteria if necessary.
• Consider the varying emphasis different sections might have on certain elements of the assignment.

Achieve consensus on criteria application:

• Ensure all GSIs understand and agree on how to apply the rubric consistently.
• Foster confidence in how well the rubric reflects the assignment goals and promotes fairness.

Grade-norming sessions help standardize grading practices by bringing GSIs together to align their understanding of rubrics and grading criteria through discussion and the evaluation of sample assignments. These sessions facilitate dialogue on grading decisions, reduce subjective biases, and ensure fairness and consistency in student assessment. Regular norming sessions throughout the semester help maintain these standards across all course sections.

References:

2020. Washington State University. Quick guide prepared by the WSU Office of Assessment for Curricular Effectiveness. https://ace.wsu.edu/documents/2015/03/rubrics-norming.pdf/

Schoepp, K., Danaher, M., & Kranov, A. A. (2018). An Effective Rubric Norming Process. Practical Assessment, Research & Evaluation, 23(11), 1–12.