Case Study

University of Edinburgh closes gap between national achievement and student satisfaction ratings

July, 2015

University of Edinburgh

University of Edinburgh

The University of Edinburgh is one of the UK’s top universities. But student satisfaction, as reflected in the annual National Student Survey, was not as high as the university would have liked . In response, the University embarked on an award-winning programme to improve students’ experience which included working with us to introducing a more reliable, streamlined and student inclusive digital assessment and feedback process. Find out how we helped to boost satisfaction scores while also improving engagement and attainment.

The challenge that University of Edinburgh faced
The University of Edinburgh wanted to introduce a slicker, more effective feedback mechanism, whilst also improving the reliability in some areas of assessment practice. These improvements would help student satisfaction rates to flourish.

How we helped University of Edinburgh overcome the challenge

To kick start the programme, our ‘compare’ feature, which utilises the Adaptive Comparative Judgement approach, was put to the test by a group of 72 School of Physics and Astronomy undergraduates to assess essay plans. Plans were uploaded to Digital Assess to enable each student to judge and leave feedback around their fellow students’ work.
Students were presented via a web-based interface with a number of anonymised pairs of work from the group. They picked the best of each pair, leaving constructive comments on each piece of work. When the judgements were completed, the multiple feedback – also anonymous – was shared with each student.
The process not only provided multi-perspective feedback, but also developed students’ evaluation skills, enhancing their understanding of what enhanced the quality of the work, which could then be fed back into their own work. Lecturers found that not only did the students engage with the process and comment on how useful it was, but there was a material positive effect on final attainment.

“Digital Assess gives you perspective on your work that you wouldn’t normally get. You get to see the ways that other students have tackled the same problems. The feedback from peers is richer, more personalised, contextualised and targeted than your tutor can give you, given the time constraints. It really helps you develop your work and general analytic skills, which will prove really useful after university in the workplace”.
Briana Pegado
Final Year Student, University of Edinburgh

The difference we made

The University of Edinburgh reached its goal of enhancing the student experience, by providing a richer means of collaboration and a deeper understanding of subject matter and assessment criteria.
The digital peer review and feedback mechanism proved very popular with tutors and students. For tutors, the system removed the need to review and feedback on each student’s plan, reducing their workload and freeing them up for other duties.
For students, multiple sets of feedback on their essay plan had a positive impact on the way that they tackled subsequent essays. Peer-to-peer assessment exposed students to what represented ‘good work’ and ‘bad work’ and gave them a better sense of the qualities that tutors were looking for.
The university has gone on to use our Adaptive Comparative Judgement approach in a number of other subject disciplines to support both formative peer-review and final summative assessment to great effect. In the case of summative assessments, final grades were really easy to calculate from a highly accurate rank order, created by our unique algorithm. The algorithm continuously responds to judgements, assigning each pair a value score and a place within the ranking.

“The University is exploring approaches to developing assessment literacy in our students. This will become a key component in their taking greater ownership of their own learning. We have found Adaptive Comparative Judgment to be highly promising and it has already enabled our students to summatively and anonymously peer-assess to a very high-degree of reliability ”
Ian Pirie
Assistant Principal of Learning and Development, University of Edinburgh

The Digital Assess legacy

Having seen the benefits that this technology brought to the classroom by improving students’ understanding of subject matter and boosting engagement and attainment, our ‘compare’ feature was rolled out more widely, including to the Edinburgh Award for study and employability skills.
In 2014, our pioneering work with the University of Edinburgh was recognised with an award for Best Use of Social and Collaborative Technologies for Learning from the eLearning Awards.