Supporting Learners with Activity Data

Whenever your students do any activity online they leave traces of that activity in log files. These traces can be used to monitor their behaviour and support them to improve their learning processes.

The idea is not a new one. Barclaycard have been analysing transactions for early fraud identification for over 25 years. Tesco use loyalty cards to individualise their marketing based on your actual purchase history. Thus if you buy cat food then they might also be able to sell you other things for your cat. If you have never bought cat food then there is not much point in trying to sell you other things for a cat (although I suppose they might try selling you a cat!). The point is that through sophisticated analysis of the information they can gain insight into their customers’ behaviour. Obviously, Tesco does this in order to increase their profits. Can we do something similar in education, not to increase profits but to support our learners?

The answer is yes. It goes under a variety of names including academic analytics (in the US), activity data (JISC) and business intelligence (guess where). Not only is the answer yes, but some universities and colleges are already making effective use of activity data to support their students. One of the earliest to make use of activity data was Purdue University, which introduced a system called Traffic Lights which it uses to inform students of how their engagement with the course compares to their peers (Figure 1).  This system gives student a green / yellow / red light for each of their courses; yellow or red lights come with suggestions as to how they can improve their engagement (?).  One lecturer highlighted that “Before, no one would e-mail me and say, ‘I’m at 58 per cent and I want to get to 72 per cent, what do I need to do?’ But students who get a red light almost all contact me immediately to ask how to raise their grades.”.

The above example highlights one of the key ways in which activity data can be used to empower students.  By providing them with feedback on their studies, together with supporting information, it can help the students to take control of their study and perform better.  Of course, the information can also be used by lecturers and support services to target their support at those students most in need of it; which may not necessarily be those students who are asking for help.

Perhaps most telling in the effect that using activity data can have is provided by Iowa Community College Online Consortium (ICCOC), who wanted to raise the proportion of students completing their degrees.  In the US 30% of college students drop out in their first year, and 50% never graduate.  ICCOC wanted to raise their completion and success (C grade or higher) rates, and already had in place a manual system to identify students at risk.  However this system was slow and expensive and interventions were often too late to have much impact.  With the introduction of an automated system (based on Pearson LearningStudio’s Enterprise Reporting suite) they have raised the completion rates from 77% to 85% and success rates from 62% to 68%.

It is worth thinking about the types of data that might be used to identify sub-optimal behaviours.  One JISC funded project Student Tracking And Retention (Next Generation): (STAR-Trak) at Leeds Metropolitan University created a dashboard to support students drawing on data from a wide variety of sources:

• Student record system (student information, course affiliation, year, module registrations, grades)
• Library management system (resources used)
• Security gates (entry to library etc)
• Lecture attendance system
• VLE (log ins, posting, resources accessed etc)

Each of these data sources can then be given a weighting and a decay rate.  For instance, missing face-to-face sessions such as lectures and seminars has a very high weighting, but having missed a lecture a couple of months ago is not as important as missing one this week.  Further, missing two or more in a row is seen as very important.  Currently all these weightings have to be inputted manually for each course (though there are sets of defaults for different types of course). However, there is no reason that the methods such as neural nets shouldn’t be used to calculate appropriate patterns that match behaviour to outcomes and find the most useful data and appropriate weightings.

Given time the system might also be able to recommend effective interventions based upon particular patterns of behaviour.  Note that this does not remove the need for lecturers and support staff to monitor student progress; rather it provides both the student and the staff a tool that can support their work.  Indeed, by giving the student access to the data and any concerns that the system may note it can empower the student to take control of their own learning and, as the Traffic Light system has demonstrated, increase the likelihood that students will ask for help when they need it.

One of the great things about using activity data is that much of it is being collected anyway, or can be collected at very low cost.  Most of the activity data comes from log files associated with systems like those listed above.  Therefore it is important to ensure that the appropriate data is being logged and retained.  The Exposing VLE activity data (EVAD)project at Cambridge found that their VLE was generating millions of log records that were of no interest and turned off that feature.  On the other hand, a partner in the Library Impact Data Project (LIDP) had checked that the information they were interested in was being logged, but then found that it was being disposed of after three months, so that they could not get the several years’ worth of data that they needed.

If you are interested in exploiting your activity data then the following may be useful:

• Look at the activity data web site at http://www.activitydata.org/.
• Consider what problem you want to address.
• Decide what information would be useful.
• Check that the information that you need is being logged.
• Ensure that the use you are making of the data falls within the scope of the permission that you have sought from users (if not you will need to change the permissions).

All of these issues are discussed on the activity data web site http://www.activitydata.org/.

Tom Franklin
Tom Franklin Consulting Ltd

tom@franklin-consulting.co.uk