Harnessing the power of predictive analytics and the powerful consequences of doing it badly [842] Home : Annual Conference 2015

Harnessing the power of predictive analytics and the powerful consequences of doing it badly [842]

Public Group active 4 years, 3 months ago

Tue, Sep 8 2015, 2:00pm – 3:00pm
Authors: Jenny McDonald
Room: Room 2.220 (Second Floor)
Theme: Harnessing the power of the crowd – collaboration and connectivist learning
Type: Standard Presentation

Description

One application of learning analytics with particular appeal for tertiary institutions is the provision of ‘early-warning systems’ to identify students at risk of failing. This type of predictive analytics is believed to help improve progression and retention rates through the early identification of at-risk students and the subsequent provision of timely and appropriate support. Examples include the Purdue Signals project (Campbell, DeBlois and Oblinger, 2007; Arnold and Pistilli, 2012), an LMS mining project (Macfadyen and Dawson, 2010) and a project which used virtual appliances to monitor student progress (Romero-Zaldivar et al., 2012).

This presentation explores potential parallels between predictive analytics in higher education settings and the use of diagnostic and screening tests in medicine. For example, in the medical domain where diagnostic tests are used to support clincial decision-making, poor understanding among doctors of the relationship between the result of the diagnostic test, its associated sensitivity and specificity and the likelihood of having a particular disease is well documented (See for example, Steurer et al., 2002). This example raises the question, are there situations in higher education settings where errors in the interpretation or application of predictive analytics may result in poor advice for students?

Recent predictive analytics studies are reviewed and the focus is empirical work where accuracy and error rates are reported. Working from this review, a conceptual framework for the application of predictive analytics in practice is developed.

Potential issues arising from the incorrect identification of students as ‘at risk’ will also be discussed. While the erroneous declaration of a positive test result to a patient is unlikely to induce the disease, the incorrect identification of a student as likely to fail a course may have profound negative consequences for the student.

Campbell, J., DeBlois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era.Educause Review, 42 (4), 40–57. Retrieved from http://net.educause.edu/ir/library/pdf/ERM0742.pdf

Arnold, K. E., & Pistilli, M. D. (2012). Course signals at Purdue: Using learning analyticsto increase student success. In S. B. Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings ofthe 2nd International Conference on Learning Analytics and Knowledge (pp. 267–270).New York: ACM. Retrieved

Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an early warning system foreducators: A proof of concept. Computers & Education, 54 (2), 588–599

Romero Zalďıvar, V. A., Pardo, A., Burgos, D., & Delgado Kloos, C. (2012). Monitoring studentprogress using virtual appliances: A case study. Computers & Education, 58 (4), 1058–1067

Steurer, J., Fischer, J. E., Bachmann, L. M., Koller, M., & ter Riet, G. (2002). Communicating accuracy of tests to general practitioners: a controlled study. BMJ, 324(7341), 824-826.

Go Back

Supported By