Applying Learning Analytics to Measure the Impact of Digital Practice Enhancements on Student Engagement and Digital Capabilities
Project Overview
This project measured the impact of digital practice enhancements on student engagement and digital capabilities across different demographic groupings, using learning analytics data and student feedback collected via questionnaires and focus groups. Twenty undergraduate programmes were chosen for this research project, from a spread of academic disciplines available at the University of Derby.
By deploying learning analytics to inform on patterns of student behaviour, the project found significant positive relationships between the use of the institutional VLE and student attainment and retention. Through the adoption of unsupervised machine learning, the project uncovered distinct differences in study behaviours and study patterns between different demographic groupings. For example, observed differences in the use of recorded lectures and electronic library resources amongst different demographic groupings.
Whilst all of the different student demographic groupings, including POLAR3, gender, age, BME status, demonstrated differences in their digital practice, it was the commuter student group that emerged as the most distinct subset; offering insights into patterns of behaviour that the sector usually associates with online learners (Jisc 2017). Throughout the course, the commuter students engaged with the VLE earlier, and demonstrating often shorter bursts of online activity than those students who live on-campus. Follow-up, focus groups revealed that these bursts of online activity were often taking place whilst commuting to University, with students making use of mobile devices and where possible, laptops.
Project Findings
The emergence of Commuter Students as a distinct tribe/grouping:
Across the twenty undergraduate programmes included in the project, ‘commuter students’ emerged as the demographic group with the greatest observed differences in study behaviours and patterns of digital practice (e.g. Commuter students, on average, use the VLE more heavily, earlier in the academic year than non-commuter students, but access it for shorter periods of time). Funding for this project has enabled the University to investigate patterns of behaviour amongst different demographic groupings and the intention is to further investigate the observed trends to inform and enhance our institutional approach to learning, teaching and student support.
Machine learning as a method of data exploration:
This project evidenced the value of ‘Machine learning’ as a tool to identify unanticipated patterns and behaviours that might otherwise be missed. For example, unanticipated relationships between students and their use of institutional digital resources, including the identification of new tribes/groupings of students. The use of machine learning techniques enabled the project team to investigate the relationship between different engagement measures and the likelihood of a student withdrawing. It also enabled the creation of a model to predict student ‘churn’ based on different engagement measures. Through the project, the University was able to explore this methodology and gain valuable knowledge and expertise in cutting-edge data gathering techniques, which can be adopted for future applications across the institution.
Reframing and enhancement of strategic approach to digital practice and digital capability development based on findings from this project:
The project has helped to reframe and raise the profile of our institutional approaches to digital practice and digital capability development. The research undertaken has informed our planned digital practice and digital capability curriculum developments, and given us a good understanding of students’ perspectives on digital practices. As such, the project has enabled us to be more effective in driving forward significant institutional initiatives, with greater buy-in from staff and students than would otherwise have been the case.
Lead institution & contact:
University of Derby Contacts:
Ruth Ayres: r.ayres@derby.ac.uk;
John Hill: j.hill3@derby.ac.uk;
Chris Bell: c.bell@derby.ac.uk
References:
Jisc (2017) Student digital experience tracker 2017: the voice of 22,000 UK learners Available here: http://repository.jisc.ac.uk/6662/1/Jiscdigitalstudenttracker2017.pdf
Further Resources: