This presentation advocates the adoption of a realist approach (Pawson, 2013) to the evaluation of learning technology in Higher Education with a suggestion of how the approach has been tailored specifically for learning technology initiatives. The realist approach does not merely ask ‘whether’ a particular learning technology development worked but more broadly, build theories on ‘what works, for whom, in which contexts and why?’ Both realist evaluation and realist synthesis are forms of theory driven evaluation that is used to uncover the very nature of programmes as being complex interventions that are introduced into social and organisational systems that are in a state of continuous change. It can be used both formatively and for summative evaluations but crucially, because it is theory-driven, it can be used before an initiative has even been implemented, in order to evaluate the potential success of a planned programme of work.
A case study is presented demonstrating a realist evaluation that uses participatory theory generation and realist synthesis of existing primary research to answer the policy question, “Where should we invest in lecture capture technology, and if so, should it be mandated for all departments for all sessions?” A summary of the findings is presented along with a reflection on the practical use of the method for budding realist learning technology evaluators. Recommendations are also provided on how the technique and the findings have the potential to impact on participatory policy making within an institution.
Sector wide participation and consistency is the key to generating an evidence base for learning technology initiatives, building upon the experiences and knowledge of others. Seven principles are articulated to help guide a Community of Inquiry using realist evaluation and synthesis. It is hoped that a giant leap forward in evidence informed policy making can be made by; building the skills of expert edtech evaluators; using a common taxonomy of learning technology terms; contributing to a shared evidence base; and using common publication standards (Wong et al., 2013).
Pawson, R. (2013). The Science of Evaluation: A Realist Manifesto. London: Sage.
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES Publication standards: realist syntheses. BMC Medicine , 11 (21).