How to write a rubric that gives consistent marks
A rubric gives consistent AI marking when each criterion covers one skill and each level describes something a marker can point to on the page, such as "supports each claim with a specific detail from the text". Replace words like "good" and "adequate", weight criteria by what matters, and test the rubric on borderline papers before using it on a class. evaleasy accepts a rubric typed in plain words or uploaded as a file.
Updated
Step by step
- List what the task assesses. Write down the separate skills you are looking for, and nothing you won't be marking.
- Write the top and bottom levels first. The extremes are the easiest to pin down.
- Fill in the middle levels. Make each differ from its neighbour by something you can name.
- Set the weights. Share out the marks by how much each skill matters for this task.
- Try it on borderline papers. Rewrite any wording that lets two markers disagree.
What makes a rubric work for AI marking?
A rubric works for AI marking or grading when two careful readers would choose the same level for the same paper without talking to each other. A new colleague can ask what you meant by "good" last year. An AI marker can't, so everything it needs has to be on the page. If a descriptor needs a conversation to apply, rewrite it.
How do you write level descriptors a marker can check?
Describe what can be seen in the work, using things a marker can find, count or quote.
| Weak descriptor | Strong descriptor |
|---|---|
| Good use of evidence | Supports each main claim with a quotation or a specific detail from the text |
| Shows some understanding of photosynthesis | Names light, water and carbon dioxide as inputs, but doesn't explain that glucose is made |
| Well organised | Opens by stating the argument, keeps to one main idea per paragraph, and ends with a conclusion that follows from them |
| Few language errors | Spelling and grammar errors never make a sentence hard to understand |
| Clear working | Shows each step, so the final answer can be traced from the first line |
| Adequate analysis | Explains why the evidence supports the claim, rather than only restating it |
Which words should you avoid in a rubric?
Avoid words that rely on the marker's own standard: good, adequate, appropriate, some, clear, effective, sufficient and various. Each one needs a definition, and once you have written the definition you rarely need the word. At lower levels, say what is missing instead of how much is lacking.
How should the levels differ from each other?
Each level should differ from the next by something you can name, so a marker can say why a paper isn't one level higher. Here is one criterion worth 8 marks.
| Marks | Use of evidence |
|---|---|
| 8 | Every main claim has a quotation or specific detail, and the student explains how it supports the claim |
| 6 | Every main claim has specific support, but the link to the claim is left for the reader to work out |
| 4 | Some main claims have specific support, and others rest on general statements |
| 2 | Refers to the text only in general terms, with no quotation or specific detail |
| 0 | No reference to the text |
How many levels and criteria should a rubric have?
Three to five levels and three to six criteria suit most assignments, though that is a starting point and not a rule. More levels only help if readers can really tell them apart. Be careful with counts: "three quotations" is easy to check but rewards padding, so say what the quotations have to do.
Why should each criterion cover only one thing?
Each criterion should measure one skill, because a joint criterion has no right answer when a student is strong on one part and weak on the other. "Argument and spelling" has no level for a fine argument with poor spelling, so split it in two. Avoid counting one fault twice as well: if "Evidence" covers missing quotations, don't take marks for them under "Argument".
How should you weight rubric criteria?
Weight each criterion by how much it matters for this task, then check the result on a paper you already have a view about. This made-up essay is polished but says little: it earns all the marks for organisation and language, and a quarter of the marks for argument and evidence.
| Criterion | Equal weights | Weighted by importance |
|---|---|---|
| Argument | 1.25 of 5 | 2 of 8 |
| Evidence | 1.25 of 5 | 1.5 of 6 |
| Organisation | 5 of 5 | 4 of 4 |
| Language | 5 of 5 | 2 of 2 |
| Total | 12.5 of 20 | 9.5 of 20 |
What else should a rubric tell the marker?
A rubric should also give the total marks, the task, and what to do in cases it doesn't cover.
- The total: an unnumbered essay with no marks stated is marked out of 10 by default in evaleasy, so write "Question 1 is out of 20"
- Which question it applies to, on a paper that also has objective questions
- What to ignore, such as handwriting, length, or spelling in a science answer
- What to do with off-topic work, or work answering a different question
What happens when work sits between two levels?
Work between two levels needs a rule, or markers will split it differently. In evaleasy the strictness setting gives the AI model one to follow: Lenient says take the higher level, Strict the lower, and Standard the better-supported level, or the lower one if neither is better supported.
How do you test a rubric on borderline work?
Test a rubric on borderline work, because clear-cut papers get the same level under almost any wording. Choose a handful of papers you found hard to place, then:
- Have two markers, or a teacher and a tool, mark them separately using only the rubric
- Compare the levels criterion by criterion, ignoring the totals
- For each disagreement, find the exact word that allowed two readings
- Rewrite that word as something visible, then try again on different papers
- Keep the papers you settled on as examples of each level
How does evaleasy use a rubric?
evaleasy applies a rubric in one of two ways, depending on how you give it. Typed or uploaded in plain words, it goes to the marker as your rules: each written answer gets a mark from the allowed values in 2 or 3 independent passes, and the median counts.
Through the API you can send a structured rubric, where each criterion has fixed levels with fixed points. The model can only pick a level, code adds up the points, and with flags on, a paper is flagged if the passes disagree noticeably.
Common questions
- Do I need a rubric if I already have an answer key?
- Use a key where answers are right or wrong, and a rubric where quality is a matter of degree. A mixed paper can use both in one document, with the rubric tied to the essay's question number.
- Can I reuse my exam board's or department's rubric?
- Yes, as a starting point. Those rubrics are often written for markers who trained together on sample scripts, so their wording can lean on shared understanding. Add details specific to this task, such as what counts as evidence.
- Should a rubric include sample answers?
- Short examples help at the levels people tend to confuse, because a quoted sentence shows the difference better than another adjective. Label them as illustrations, so a marker doesn't simply hunt for those exact words.
- Does a good rubric guarantee reliable AI marking?
- No. Clear wording removes one source of inconsistency, but it can't show how a tool handles your students' work. Compare its marks with a teacher's on papers already marked, criterion by criterion, before relying on it.
Your next stack of papers doesn't have to be your problem.
Signing up is free. Marking uses credits, at 1 credit per page.