Authentic Assessment: An example

by Rachel Heyes, Learning Technologist, University of Manchester

All UK medical students take a comprehensive national examination in their final year of study: the Prescribing Safety Assessment (PSA). The PSA is a high stakes exam; a pass is required by the General Medical Council (GMC) for junior doctors to progress through their training programmes.

As a Learning Technologist, I am responsible for learning content on 1Med Learn, a bespoke VLE for the medical programme at the University of Manchester. I work with academics and clinicians to design and develop teaching and learning materials. Materials on 1Med Learn are regularly structured as case presentations, where a fictional patient presenting with a set of specific symptoms is presented to students who work through information and interactive learning activities to develop clinical reasoning skills in presenting differential diagnoses and determining treatment options.

Figure 1 – University of Manchester (2020) 1Med Learn

Currently, the prescribing and medicines safety team set formative assessment questions related to the case material for students to practice their prescribing skills on the PSA platform. It makes sense to have these activities linked to their 1Med Learn case materials to replicate clinical work and create authentic assessment. The extensive use of electronic prescribing systems (EPS) throughout primary and secondary care settings means that all undergraduate medical students need to learn with, and alongside, such technology rather than through the creation and review of handwritten practice prescriptions.

Figure 2 – British Pharmacological Society (2023) prescribingsafetyassessment.ac.uk

The feedback from medical students is that they do not like having to move from 1Med Learn to the PSA platform because it requires a separate login, and they need to use the information within the case materials on 1Med Learn to answer the questions. Kurt Wilson, Professor of Medical Education at University of Manchester (UoM), a member of the prescribing and medicines safety (PMS) team
approached me to develop an Electronic Prescription Service (EPS) tool within 1Med Learn, with the aim of making these assessments more authentic.

The initial stage of this project involved discussions between eLearning specialists, and prescribing specialist academic clinicians, to identify and understand their initial ideas and the requirements of the proposed 1Med Learn PSA tool. The concept was to replicate the format of questions used within the PSA, within 1Med Learn. Initial meetings and discussions were an important element of the project’s success. It was vital to ensure that all parties understand each other and formed a good working relationship. Following an initial meeting with the PMS team, we were able to create a few options using existing eLearning tools. These options were then presented in a follow up meeting. Being able to present some options and discuss the pros and cons of existing tools allowed us to further understand the needs of the proposed 1Med Learn PSA tool. It also allowed the academic clinicians to understand the parameters of different technologies, and, as a result, we were able to better articulate our thoughts as a group.

Follow up meetings dedicated to creative exploration and discussion were valuable and important in relation to developing, not just the project ideas or concepts, but also the group’s working relationships. Once a supportive working environment was established, creative thinking and problem solving skills were enhanced and the ideas generated become more fine tuned, sophisticated, and likely to succeed. Academics may benefit from learning from the experiences of eLearning staff who have often implemented many different projects, to varying degrees of success. Many eLearning professionals form close working relationships with specific academics, or clinicians. This allows them to continue collaborating to further support teaching and learning activities, which enables both parties to achieve their goals and provide the best experience for students.

Once we had our concept for a 1Med Learn PSA tool, the next stage was to approach the British Pharmacological Society (BPS). The most exciting feature of our proposed tool was that it would utilise the BPA’s extensive database of medications. The database is invaluable to the students in making the assessment authentic as it allows them to select the medicines, doses, and routes in keeping with electronic prescribing used in clinical practice, whilst mirroring the platform used for the PSA. What benefited us in approaching and being the first university to work in partnership with the BPS was Dr Wilson’s existing relationship with the team, due to his work on the PSA Assessment Board. This meant when we were introduced to the BPS Team, they were already prepared to be familiar with the PMS team and were happy to collaborate with their eLearning colleagues. The BPS team developed the API for the tool, and with the help of an externally contracted developer, we were able to create the 1Med Learn PSA tool.

Figure 3 – Photo by Tony Schnagl from Pexels (2020). Edited by Rachel Heyes, University of Manchester (2022) 1Med Learn PSA Tool

There was, of course, a lengthy process of testing. I have been able to manage this process, and working with so many different people has been an invaluable experience. Using project management and communication skills I found that I was well placed as someone who understood the ‘language’ of both the academic clinicians and the developers/programmers, and so I was able to translate requirements between the two groups and lead the project to the point of implementation. In the next academic year, students on the medical programme will benefit from authentic assessment using the 1Med Learn PSA tool within their online cases, and we will review the impact of the tool alongside its continued development. Teamwork makes the dream work.

Rachel Heyes and Hiten Mitha will be presenting their work at #ALTC23. Watch their pre-recorded session ‘Authentic Prescribing Safety Assessment: Improving medical students’ prescribing skills through bespoke electronic prescribing systems’ anytime or at 15:30 on Tuesday 5 September.

References

British Pharmacological Society (BPS). (2023). Manchester improve medical students’ prescribing skills with our online platform. BPS Assessment. [online]. Available from: https://www.bpsassessment.com/manchester-improve-medical-students-prescribing-skills-with-our-online-platform [Accessed March 24, 2023].

Heyes, R. and Wilson, K. (2022). Improving medical students’ prescribing skills through bespoke electronic prescribing systems. FBMH News. [online]. Available from: https://blogs.bmh.manchester.ac.uk/blog/2022/12/01/improving-medical-students-prescribing-skills-through-bespoke-electronic-prescribing-systems [Accessed March 24, 2023].

Did you enjoy reading this? If so, consider becoming a Member of ALT. If your employer is an Organisational Member, membership is free! Find out more: https://www.alt.ac.uk/membership