Six hours. Not every week. Not every month. Every year. That’s about how long students in a big study led by Stanford spent on a mastery-based math platform on average, and it was enough to make a difference.
This is the kind of finding that sounds almost too small to be important. Still, when researchers looked at data from more than 200,000 students over three school years, they saw a clear, roughly linear pattern: students who spent more time on the platform did better on math tests given by outside sources. Scores tended to drop when there was less use. The authors of the study in the Proceedings of the National Academy of Sciences make it clear that they did not use a randomized control trial. But the size of the data and the fact that it was tracked over several years to account for possible confounding factors give it real weight.
Emma Brunskill, a computer science associate professor at Stanford who helped write the study, made it clear. Since the pandemic, students in fourth and eighth grades have done much worse on national math tests. That’s not just a problem for school. Their futures are limited in what they can do and what they can say in the future. Brunskill has spent years developing AI methods that estimate the results of interventions. She used these methods to look at one of the biggest problems in education: why does something that clearly works keep being ignored in classrooms?
The software is based on a method called “mastery learning,” which means that students don’t just follow along with the class as they learn. Instead, they move on to new material only after showing that they understand it. In theory, it makes sense. In real life, it makes things hard for teachers. Someone in the class might be working on third-grade material while someone else is working on second-grade. That’s not the same as giving everyone the same worksheet to manage, and that’s part of the reason why adoption has been so far behind the evidence.
There’s also a bigger problem with trust. Brunskill said that most people, including many teachers, aren’t naturally good at figuring out which learning technologies work. When intuitions about education meet data, they don’t always hold up. And when software is tested in controlled trials with supportive administrators and extra training for teachers, the results might not be the same in a regular classroom in a regular district with little help. It’s not as close as it seems between a good place for research and a lot of people using it in real life.
This study feels different because of its scope and the fact that it is open about its limits. They didn’t pick a group with a lot of usage just to show off. They looked at students who did anything from nothing to about thirty hours a year, which is a much more realistic sample size than what previous research had found. “The 5 Percent Problem” is the idea that most of the good results come from students who were already very motivated and likely to do well. This idea has been a shadow over ed-tech research for a long time. The five percent of users who spend twenty hours a year on a math platform are in the top twenty percent. That kid isn’t like most kids.
There was one thing in the study that even Brunskill thought was important. The difference between students who did well and those who didn’t got smaller when learning outcomes were measured by skills mastered instead of hours logged. Students who were already behind didn’t get further behind. That makes sense. It suggests that the technology might work better for more students if teachers use it to track progress through the curriculum instead of time.
There is still a long way to go before this evidence is really used in the classroom. But it’s impossible not to notice that the case for these tools keeps getting stronger, even though people are still not making the most of them. Somewhere along the line, that gap turns into its own issue, one that can’t be fixed by research alone.
