Full Citation
Jerrim, J. (2018). PISA 2015: how big is the ‘mode effect’ and what has been done about it? Oxford Review of Education, 44(4), 427-448.
Publication Type
Analysis of international assessment data
What They Studied
PISA (Programme for International Student Assessment) switched from paper-based to computer-based testing in 2015. This study examined whether the testing medium affected student performance.
Key Findings
The testing medium had a substantial impact on scores:
“The paper-based group did 20 scaled score points better than the computer-based group – the equivalent of about 6 months of additional schooling.”
This is an enormous effect. Students taking the same test performed significantly worse when it was administered on computer rather than paper.
Implications:
- Reading comprehension on screen is genuinely more difficult
- Test scores may not accurately reflect student knowledge if medium isn’t controlled
- International comparisons became problematic after the medium switch
Why This Matters for Schools
If students perform worse on assessments simply because they’re on screens rather than paper, this has serious implications for:
Exams and testing:
- Schools increasingly use computer-based tests
- Children may be underperforming due to medium, not knowledge
- GCSEs and A-Levels moving to digital formats may disadvantage students
Learning assessment:
- If we can’t accurately assess what students know via screen-based tests, how can we use screen-based learning platforms effectively?
- The data from EdTech platforms may not reflect genuine learning
What Parents Should Know
If your child is underperforming on computer-based assessments, it may not reflect their actual knowledge or ability. The medium itself may be the problem.
For high-stakes exams: Some exam boards are moving toward computer-based testing. This research suggests students who’ve practiced primarily on paper may be disadvantaged.
Questions to ask:
- Are school assessments being done on paper or screen?
- Is my child getting practice with the actual medium they’ll use for exams?
- Can poor test performance be partially explained by the testing medium?
