The university regulations concerning moderation (MARP GR 2.4) now include marker-team calibration in advance of marking and moderation as an option.
This page contains guidance on what constitutes calibration and examples of practice to guide its integration with other marking and moderation tools.
Calibration does not replace post-marking moderation, but may be used in conjunction with it. Where departments deploy calibration methods these should form part of the sample / workload for moderation (i.e., not in addition to).
MARP has been updated (GR 2.4.2(c)) to clarify that calibrated marked papers be included in the sample presented. Where sampling is the moderation method and calibration has taken place, the sample does not need to apply the latter square root rule per marker. However the sample should still involve work from each separate marker.
In addition the square root rule functions with calibration such that this applies to the sum total of scripts, rather than number of markers involved.
The definition of moderation in MARP will be widened in AY 24/25 to include feedback as part of the moderation process — that is, markers will ‘moderate’ each other’s feedback. Once updated in MARP, feedback should be included as one of the documents in the pack for moderation.
What is calibration?
Calibration approaches aim to harness marker-team peer discussion in order to reach a common understanding of standards and thresholds. It is a process of calibration conducted internally within an institutionally defined marking team, and is distinctive from moderation practices that typically take place at the end of the marking process. Standardisation practices are based on the whole team marking a sample of live or historic assessment artefacts in a semi structured environment, prior to the process of marking and moderating the full cohort.
Research on calibration practices offers evidence of decreased variability, improved marker team consensus and indicates improved marker confidence in making reliable judgements about standards (O’Connell et al, 2016).
A small scale investigatory project identified similar benefits across a range of Lancaster and institutional partner contexts, resulting in the following outputs to support colleagues at Lancaster University and its partner institutions to advocate for adopting the practice of calibration amongst marker teams.
- A discussion guide for marking teams
- Case studies of practice (accessible only by colleagues at Lancaster University and its partner institutions)
- A detailed presentation of analysis and discussion of project findings made at the LU Annual Education Conference on 5th July 2023.
__
Please select and click on the icon link below for further information: