Language fairness
Is AI grading biased against non-native English writers?
Recent studies say yes—often in ways that are hard to see. Most AI marking tools reward how something is written, so a strong argument in plain English can earn a weaker mark.
What the research shows
Scoring penalties linked to learner writing. Park et al. (2026) examined how a large language model scored the PERSUADE 2.0 essay corpus. They found an implicit pathway in which text features associated with English-learner writing led to lower scores, with a 4:1 penalty-to-benefit structure.
Differences by first language. In a preprint testing a fine-tuned open model on 12,100 TOEFL essays, Gayed (2026) reported that, within every proficiency band, essays from European-language backgrounds scored consistently higher than essays from East-Asian-language backgrounds.
Models can infer background from writing. Yang et al. (2025) studied whether GPT-4o can infer demographic attributes such as first-language background from student essays, and how that relates to bias in its scores.
Detectors show the same pattern: Liang et al. (2023) found 61.22% of non-native English essays wrongly flagged as AI-written.
Why it happens
Fluency is a signal, not a warrant. Systems that learn from polished text tend to treat polish as quality, so plain or second-language phrasing gets read as weaker thinking—even when the argument is sound.
How universities can reduce it
- Score reasoning and expression separately, so language accuracy cannot drag down a strong argument.
- Tie each mark to specific passages, so a reviewer can see what was actually rewarded.
- Compare results by language background every marking cycle, against expert human markers.
- Escalate borderline cases to a person rather than trusting a single score.
CERENO Eval is designed around these steps: its reading layer (Cerebrum) collects evidence for each criterion, its reasoning layer (Nous) tests whether that evidence supports the criterion, and expression is scored on its own line. Language and cohort bias reports are part of every pilot.
Sources
- Park et al. (AIED 2026) — Implicit and explicit bias toward English language learners in LLM-based essay scoring
- Gayed (2026, preprint) — First-language bias in LLM-based automated essay scoring on TOEFL essays
- Yang et al. (2025, preprint) — Does the prompt-based LLM recognise students’ demographics and introduce bias in essay scoring?
- Liang et al. (2023) — GPT detectors are biased against non-native English writers