Quality Assurance of MOOC-based curricula: a project report from Kiron

Summary

Kiron Open Higher Education (gGmbH) is a non-profit edtech organization that provides access to higher education (HE) for refugees through strategic partnerships with higher education institutions (HEIs) and online education providers. Kiron’s program consists of tailor-made curricula, a digital collaborative platform and support services for refugees to begin their studies online while they prepare to apply to a HEI. Our goal is to provide MOOC-based curricula that are recognized at partner institutions. A key requirement for partner institutions is quality assurance (QA), and so in this blog post, we report on the multi-layered QA approach we are developing at Kiron.

Kiron’s academic model

Kiron studies are suitable for learners who are either seeking asylum or do not yet have the required documents to apply to university in their host country. At Kiron we bundle massive open online courses (MOOCs) into curricula, and deliver them via the virtual Kiron Campus platform, adding live online tutorials. Once learners have met admission requirements, we assist them in applying to an HEI. If accepted, learners may apply for recognition of up to 60 ECTS of completed Kiron coursework, and continue studying offline towards an accredited Bachelor’s degree.  For more information about Kiron’s Model see the Kiron website and Suter & Rampelt 2017.

To enable recognition of credits, HEIs need proof of QA procedures and the assurance that internationally accepted quality criteria have been applied. At Kiron we believe that a strong network of institutions combining their quality processes can enable equivalence of non-formal curricula with traditional quality-assured programmes.

Layers of quality assurance

Curating MOOCs from different institutions and platforms, and relating them to modules, demands rigorous coordination. Kiron works as an intermediary between learners and partner institutions as well as the MOOC platforms, facilitating communication to enable the recognition of MOOC certificates when a learner transfers to an HEI. Our curricula therefore have to abide to high standards, secured by multi-layered QA (see the figure above).

MOOC-providing institutions

Kiron mainly uses MOOCs from established HEIs. To date, we have incorporated 233 MOOCs in our curricula. These MOOCs are provided by 52 different accredited HEIs, and four other institutions. The non-HEI institution used most in Kiron’s curricula is Saylor Academy, a nonprofit organisation that curates free and open online resources. Saylor is transparent about its QA approach, which involves review cycles and an external review of parts of its courses.

At Kiron we sought information on institutional support for online learning at the HEIs that provide our MOOCs, and found this information for over 91% of the institutions. Support reflects in three, often connected, strategies:

• Adaptation of existing QA strategies – for example, the University of British Columbia developed a QA framework based on an already-established instructional design model;
• Centralised support units to facilitate online learning – for example, RWTH Aachen University support their teaching staff with different service units such as “Media for Teaching”;
• MOOC Implementation as strategic development for the HEI – for example, the University of Texas at Austin established Project 2021 to develop excellence in learning and teaching.

In summary, the institutions involved in Kiron’s curricula provide a consistent level of QA through course design procedures based on recognised good practice, regular external review cycles, and institutional support structures.

MOOC platforms

While the general QA is secured by the MOOC-providing institutions, MOOC platforms have also developed standards for courses, and often offer learning analytics and other information to MOOC providers. EdX, for example, provides not only a MOOC development checklist (MDC) to ensure consistency on its platform, but also educational services such as “training, onboarding, high-level program management, learner technical support, course strategy” (edX website), and collects data for MOOC providers to analyze their learners’ learning behaviour and iteratively develop their courses.

Kiron

Kiron chooses MOOCs based on their learning outcomes, learner-centered learning approaches, and the sustainability of modules. We have also established innovative tools to provide transparent descriptions of MOOCs and to supports our partners in analyzing the courses. The Learning Outcome Comparison Matrices show how each MOOC contributes to Kiron’s module-level learning outcomes. Additionally, Kiron MOOklets (MOOC Booklets) provide detailed information on learner workload, course design, assessment and authentication, and key facts about the MOOC-providing institution and its accreditation status. The data collected in these tools allows HEIs to judge the quality of the courses, thereby addressing the recommendations in the EAR (European Recognition Manual for Higher Education Institutions, Nuffic, 2016: link to PDF), and helping HEIs to ascertain the quality dimensions of online courses.

The fit between Kiron’s module requirements and the quality of MOOCs implemented in its curricula is reviewed repeatedly. Feedback by partners is an important element in constantly developing the curricula. More information on Kiron’s QA procedures is available at Rampelt & Suter 2017 and via our Quality Handbook, which was developed through funding from the Bertelsmann Foundation and the German Federal Ministry of Education and Research.

Finally, recognition by Kiron’s partner institutions is also a step in verifying the different layers of QA (see figure below). In 2017, Kiron piloted its university transfer process. Three students received recognition by partner institutions while a fourth student was preliminarily enrolled in a Master’s programme and was given the missing Bachelor-level credits via recognition of Kiron modules. This shows that, under certain quality measures and with transparency, MOOC-based curricula can be comparable to offline curricula.

Conclusion

The combination of lack of transparency from MOOC providers and the difficulty in assessing learners make recognition of MOOC-based learning challenging. Through this multi-layered approach, Kiron aims to fill the gaps. However, this model does not imply a general recognizability of MOOCs by HEIs, since MOOCs at Kiron are carefully selected to meet quality criteria and are combined in modules to reflect specific learning outcomes.

Kiron still faces obstacles, especially in implementing assessment models that meet university quality criteria. While some MOOCs already include assessment with authentication and verification, this is not always the case. Assessment methods at Kiron have to address the target group and its limited capacities in terms of mobility and finances, as well as discipline cultures and demands of partner institutions. Through funding by the German Federal Ministry of Education and Research, Kiron is continuing to seek ways to address this issue, together with our partner universities, RWTH Aachen University and Lübeck University of Applied Science.

We welcome feedback and comments on this post.

Hannes Niedermeier is Quality Assurance Manager at Kiron Open Higher Education, and has worked in TeacherEducation at the University of Passau, Germany. Contact: hannes.niedermeier@kiron.ngo, or follow Hannes on Twitter.

If you enjoyed reading this article we invite you to join the Association for Learning Technology (ALT) as an individual member, and to encourage your own organisation to join ALT as an organisational or sponsoring member