What parents need to know
When PIRLS reading exam moved online in 2021, average scores dropped 27 points. Test designers acknowledged that “mode effects significantly contributed to the decline.” Moving tests online makes all children appear less capable than they are.
Full Citation
Horvath, J.C. (2025). The Digital Delusion: How Classroom Technology Harms Our Kids’ Learning. Analysis based on PIRLS transition data. Penguin Random House.
Publication Type
Research synthesis analysing PIRLS (Progress in International Reading Literacy Study) data across the transition to digital testing
What They Studied
Jared Cooney Horvath examined what happened when PIRLS, the only long-term international examination focused exclusively on reading skills in young children, made the shift from paper-based to computer-based testing in 2021. PIRLS assesses reading comprehension and literacy skills in fourth-grade students (typically age 9-10) across multiple countries. The timing coincided with COVID-19 disruptions, but test designers were able to identify the specific contribution of the mode effect.
Key Findings
- “PIRLS (the Progress in International Reading Literacy Study) is the only long-term international exam focused exclusively on reading skills in young children”
- In 2021, PIRLS made the shift from paper-based to computer-based testing
- “That year, average scores dropped by 27 points”
- “While COVID disruptions likely played a role, PIRLS designers acknowledged that the mode effects significantly contributed to the decline”
- The 27-point drop is even larger than the PISA mode effect, suggesting younger children may be particularly vulnerable
- Test designers themselves acknowledged the mode effect – a rare admission that the testing medium fundamentally changes what’s being measured
- “The mode effect is the tendency for students to perform worse on computerised tasks compared to pen and paper ones”
- For reading assessment specifically, the mode effect is particularly pronounced
- This has direct implications for how we interpret reading scores and literacy levels
- Children may appear to have lower reading skills than they actually possess when tested on screens
- The finding raises questions about the validity of digital reading assessments
Read the book
Horvath, J.C. (2025). The Digital Delusion. Penguin Random House.
