What parents need to know
When PISA moved from paper to computer testing in 2015, scores dropped 14 points across the board. Everyone did worse – even students who used technology heavily. The testing medium matters more than we thought.
Full Citation
Horvath, J.C. (2025). The Digital Delusion: How Classroom Technology Harms Our Kids’ Learning. Analysis based on PISA transition data. Penguin Random House.
Publication Type
Research synthesis analysing PISA (Programme for International Student Assessment) data across the transition from paper to computer testing
What They Studied
Jared Cooney Horvath analysed what happened when PISA, the world’s largest standardized educational assessment, transitioned from paper-based to computer-based testing in 2015. PISA tests hundreds of thousands of 15-year-old students across dozens of countries every three years in mathematics, reading, and science. The 2015 transition provided a natural experiment to examine how the testing medium affects performance at a massive scale.
Key Findings
- “PISA (the Program for International Student Assessment) is the largest standardised test in the world. Every three years, hundreds of thousands of 15 year old students across dozens of countries complete the exam, which assesses knowledge in maths, reading and science”
- In 2015, PISA transitioned from a paper-based to a computer-based test
- “The shift from paper to digital testing reduced scores by 14 points across the board. Everyone did worse – even the heavy tech users”
- The universal score decline suggests the mode effect isn’t about familiarity or comfort with technology
- Even students who used computers extensively still performed worse on computer-based tests
- The 14-point drop is substantial and consistent across countries with different levels of technology integration
- This finding contradicts assumptions that “digital native” students would perform better on computer tests
- The mode effect operates even for students who are highly comfortable with technology
- PISA test designers have struggled to account for and adjust for this mode effect
- The finding has profound implications for educational assessment and for understanding what standardized test scores actually measure
Read the book
Horvath, J.C. (2025). The Digital Delusion. Penguin Random House.
