Can AI Decisions Be Unfair?
Can a computer be unfair? Many people think not: after all, an algorithm has no emotions or personal preferences. Yet AI can produce outcomes that systematically disadvantage groups of pupils. That is precisely why bias in AI is not a side issue but a fundamental problem for education: it undermines the core promise of equal opportunities for every learner.
TL;DR Summary
AI may seem neutral but can unintentionally discriminate due to skewed data or assumptions. In the classroom, this leads to unfair exam questions, biased feedback, or unequal study advice. Teachers must remain critical, and school leadership should demand transparency.
What Is Bias in AI?
Bias means systematic distortion: a pattern that treats certain groups differently. This is not always bad – an AI tool might, for example, deliberately provide extra support for pupils who are still learning Dutch as an additional language.
The problem arises when bias is unintended, such as when caused by:
- Skewed datasets – historical inequalities are simply repeated.
- Hidden assumptions – design choices (e.g. what counts as “correct Dutch”) unintentionally disadvantage some groups.
- Lack of context – AI cannot take account of individual circumstances such as dyslexia or multilingualism.
The result: decisions that appear objective but in fact are unfair.
Examples from the Classroom
- Culturally skewed test questions: An AI generates tasks with references to typically Dutch holidays or contexts. Pupils unfamiliar with them are unfairly disadvantaged, even if their subject knowledge is sound.
- Distorted language feedback: AI writing tools flag code-switching or dialect use as incorrect. Pupils with a multilingual background may therefore receive lower scores, even though their language development is actually richer.
- Study advice that repeats inequality: When AI predicts the school type or course most suitable for a pupil, it can reproduce historical patterns of inequality.
What Does Fairness in AI Mean?
Fairness in AI has several dimensions. One important principle is procedural fairness: decisions must not only be fair, they must also appear fair. In education this means that pupils, parents, and teachers can understand how a grade or piece of advice has been produced. Transparency is therefore essential to trust.
What Does This Mean for You as a Professional?
For Teachers
- Treat AI output as a tool, never as the final word.
- Vary prompts and check whether results are fair for all pupils.
- Spot patterns: if certain groups consistently score worse, investigate why.
For School Leadership
- Ask suppliers explicitly how their tool has been tested for bias.
- Pilot AI applications in diverse classes before adopting them school-wide.
- Set clear boundaries: never let AI be the sole basis for high-stakes decisions such as progression or study advice.
Consequences of Unfair AI
Unfair AI decisions affect more than individual pupils. They can also:
- Reinforce stereotypes – pupils may internalise the idea that their background limits their potential.
- Undermine trust – parents and pupils lose faith in both the school and the technology.
- Threaten equal opportunities – the central mission of education is compromised.
International frameworks, such as the UNESCO AI guidelines and the EU AI Act, therefore stress fairness as a core principle: AI should not entrench inequality but actively contribute to inclusion.
Fairness as a Foundation
Education is about equal opportunities. Unfair AI outcomes undermine this directly. Fairness is not a luxury or an afterthought – it is a foundation. Only when schools approach AI critically and responsibly can the technology genuinely support equal opportunities.
Read the Full Article Series
Want to explore the bigger story behind AI fairness in education? Dive into our 3-part series:
- Can AI decisions be unfair? (this article) - The basis: why bias in AI is a structural problem.
- The struggle for AI fairness in schools - How fairness becomes a policy choice.
- Bias in AI and equal opportunities for pupils - How AI impacts real classrooms.