This presentation explores potential parallels between predictive analytics in higher education settings and the use of diagnostic and screening tests in medicine. For example, in the medical domain where diagnostic tests are used to support clincial decision-making, poor understanding among doctors of the relationship between the result of the diagnostic test, its associated sensitivity and specificity and the likelihood of having a particular disease is well documented (See for example, Steurer et al., 2002). This example raises the question, are there situations in higher education settings where errors in the interpretation or application of predictive analytics may result in poor advice for students?
Recent predictive analytics studies are reviewed and the focus is empirical work where accuracy and error rates are reported. Working from this review, a conceptual framework for the application of predictive analytics in practice is developed.
Potential issues arising from the incorrect identification of students as ‘at risk’ will also be discussed. While the erroneous declaration of a positive test result to a patient is unlikely to induce the disease, the incorrect identification of a student as likely to fail a course may have profound negative consequences for the student.
Campbell, J., DeBlois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era.Educause Review, 42 (4), 40–57. Retrieved from http://net.educause.edu/ir/library/pdf/ERM0742.pdf
Arnold, K. E., & Pistilli, M. D. (2012). Course signals at Purdue: Using learning analyticsto increase student success. In S. B. Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings ofthe 2nd International Conference on Learning Analytics and Knowledge (pp. 267–270).New York: ACM. Retrieved
Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an early warning system foreducators: A proof of concept. Computers & Education, 54 (2), 588–599
Romero Zalďıvar, V. A., Pardo, A., Burgos, D., & Delgado Kloos, C. (2012). Monitoring studentprogress using virtual appliances: A case study. Computers & Education, 58 (4), 1058–1067
Steurer, J., Fischer, J. E., Bachmann, L. M., Koller, M., & ter Riet, G. (2002). Communicating accuracy of tests to general practitioners: a controlled study. BMJ, 324(7341), 824-826.