Piloting learning technologies: from dreams to reality
By Dr Steph Comley (University of Exeter) and Cat Bailey (Jisc)
Piloting. Proof of concept. “Let’s use this new tool.” The way learning technologies are introduced to colleges and universities varies. Sometimes pilots are undertaken, other times a learning tool is introduced and no one is quite sure why.
If a pilot is undertaken, often they’re perceived as symbolic and being driven by an agenda rather than a genuine evaluation. In today’s climate of stretched budgets, heavy workloads and sustainability goals, we need pilots that are focused, decisive and genuinely useful. We should be clear on why technologies are being introduced and have a transparent methodology to achieve this.
What is a pilot?
So what is a pilot and what is involved? What will take you from dreams to reality?
Piloting can involve:
- Scoping the need,
- Discovering what’s already happening in the sector
- Costing and writing a business case
- Liaising with legal and procurement
- Consideration of ethical, environmental and accessibility impact
- Collaborating with teams such as IT, leadership and library colleagues
- Consulting user voice
as well as actually running the pilot, and feeding back results at the conclusion.
Pilots should be real pilots and not soft rollouts. There must be confidence and support to say, ‘this tool isn’t fit for purpose’ if that’s what the evidence shows.
Building on ALTC23 and Anne-Marie Scott’s challenge
At ALTC 2023, Anne-Marie Scott challenged us to think critically about how we adopt technology and the processes behind it. We need to give real thought to why we are using and procuring new tools. Do we need them? What is the benefit? What is the cost from a monetary and environmental viewpoint?
This raises further questions:
- What does a real pilot look like?
- How do we avoid pilots becoming soft rollouts?
- What frameworks can help ensure pilots are robust, inclusive and purposeful?
Why frameworks matter
Positively, Digital Learning teams are being involved more in the decisions and planning around pilots and, in many cases, are leading on these. With this enhanced presence comes the need for a robust and joined up process to help successfully run the pilot and ultimately provide the tools that our learners and colleagues need.
We have both been involved in running pilots of new learning technology and feel that a framework to help guide us would have been really valuable as we began our journey. It would have helped with confidence in our decision making, as well as supporting us to articulate and manage expectations around the process and outcome.
Join the conversation
Join us (Steph Comley and Cat Bailey) at ALTC on the Friday to explore some piloting scenarios and share your own ideas on what makes a successful pilot. These ideas will all feed into a framework that can be shared across the sector and help us run successful pilots for learning technology that will support learners in their learning journey.
We hope to see you then!
ALT’s Annual Conference is one of the UK’s largest conferences for learning technology and digital education professionals. The conference provides a valuable and practical forum for practitioners, researchers, managers and policy-makers from education and industry to solve problems, explore, reflect, influence and learn.
ALTC25 will take place in Glasgow on 23 and 24 October 2025. Register closes 20 October 2025.