There is something slightly silly about how AI is used in higher education right now. Policies have been written, consultants have been hired, and all-staff meetings have been held. They have changed the academic integrity handbooks, put disclaimers in assignment briefs, and put in detection software that most students already know how to get around. A proper, structured, and open pilot is something that almost none of them have done, but it might be the one thing that tells them what works.
The Wonkhe and Kortext research team brought this up in their project called “Educating the AI Generation.” It’s worth thinking about for a moment because their point is not a radical one. It’s so sensible that it’s almost boring. If you aren’t sure if a tool will work in your school with your students and teachers, you might want to find out before committing to it. This is a general way that responsible groups should decide what to do. We need to remind higher education of this because AI has made things very confusing right now.
They make cases that are exact copies of what a well-designed pilot needs. There’s more to it than just turning something on for one department and asking people if they liked it. It means putting the technology in the context of a real strategic question: what are we really trying to achieve here? It also means being willing to admit when “this didn’t deliver.” It means getting real baseline data, treating skeptics as participants instead of problems, and planning the exercise so that the results can be used. People often skip that last part more than any other.
There is a pattern that shows up in most stories about how universities have adopted AI. An early adopter who is really excited about a platform promotes it. A small group of staff members who are interested try it out. Feedback that isn’t formal goes around. A decision is made about buying something. It’s too late to ask if the thing really changed how well students did; the contract has already been signed and the sunk cost logic has taken over. In that case, the pilot stops being an important part of the learning process and turns into a formality.
The interesting thing about the Wonkhe is that they are not naive about the limits of piloting either. As a side note, they do say that well-run pilots often benefit from the strategic focus and genuine enthusiasm of the people directly involved. This means that results often look better during the pilot than they do when something is rolled out to the whole institution.
That is a real problem that doesn’t get enough attention. A carefully managed experiment in a controlled setting is not the same as a system that works across dozens of departments, with different levels of staff support and students who weren’t in the test group. It doesn’t hurt the case for piloting to say that there is a limitation right away. If anything, it makes the case for doing it very carefully stronger.
There is also a human side to this that gets lost in the talk about policy. A separate study by Wonkhe found that 38% of UK undergraduates have turned in work that they couldn’t fully explain. The study also found that students are mostly making their own ethical frameworks for using AI without much help from their institutions. That’s not really a picture of chaos, but it does show people making important choices without any help. No amount of careful experimentation can answer the question of how AI changes learning as well as production. It won’t be solved by any policy document, no matter how detailed it is.
It’s possible that not wanting to pilot properly isn’t just because they’re lazy or forget to. To really experiment with institutions, you have to be ready to find out that something you put money into doesn’t work. That makes people uncomfortable, especially when senior leaders have pushed a tool or platform in public. At its core, a pilot study with open methods and clear results is a risk. It is much easier to use broadly and see the ambiguity in a good light.
In the end, that’s what makes the Wonkhe argument worth going over again. It’s not telling institutions to move more slowly. They are being told to pay attention to what they are learning. It shouldn’t be hard to make that case right now, when technology is changing quickly and students’ lives are at stake. It says more about higher education than it does about AI that it seems to still be the case.
