What parents need to know
Students randomly assigned to take tests on paper scored 20 points higher than those on computer – equivalent to 6 months of additional schooling. When we test children on screens, we’re underestimating what they know by half a year.
Full Citation
Jerrim, J., Micklewright, J., Heine, J-H., Salzer, C., & McKeown, C. (2018). PISA 2015: How big is the ‘mode effect’ and what has been done about it? Oxford Review of Education, 44(4), 476-494. [Analysis discussed by Daisy Christodoulou in “Paper and On Screen Assessments”]
Publication Type
Academic research analyzing PISA data, discussed in educational commentary by assessment expert Daisy Christodoulou
What They Studied
Researchers analysed results from over 3,000 students in Germany, Ireland, and Sweden who participated in the 2015 PISA tests. Crucially, students were randomly assigned to take identical tests either on paper or on computer. This randomization allows researchers to isolate the effect of the testing medium itself, separate from other factors that might influence performance.
Key Findings
- Students randomized to the paper-based test group scored 20 scaled score points higher than the computer-based group
- This difference is “equivalent to about 6 months of additional schooling”
- The mode effect was consistent across all three countries studied
- Random assignment means this is genuine causation, not just correlation
- As Daisy Christodoulou explains: “Why might this effect happen? There is some evidence to suggest that we don’t think as deeply when reading and writing on screen”
- “When we read a text on screen, we scan and scroll far more than when we read it on paper”
- “When we hand-write notes, we abbreviate and summarise far more than when we type, and this helps us to remember more of our notes”
- When schools or exam boards move tests online, they are systematically underestimating student knowledge
- A 6-month equivalence is educationally huge – it could be the difference between passing and failing, or between grade levels
- For high-stakes testing, the medium effect raises serious fairness questions
- Students tested on screens may be disadvantaged compared to their actual knowledge and abilities
