I managed to catch the afternoon of this really interesting day of webinars organised by Phil Anthony from the University of Kent. I found the whole afternoon really interesting, and if you are interested in watching any of the presentations, they’ve already been uploaded to the Digitally Enhanced Education Webinars YouTube channel – super efficient!
I’m not going to write about all of the talks, they are available to watch so you can pick the ones you are interested in and watch them. Most of them are 15-20 mins long so not a huge investment of time. I wanted to reflect on a few of the talks that resonated with me most.
The first was by Richard Ogundele (University of Kent) who’s talk was titled ‘Governing AI Agents in Education: Three Risk Patterns Pilots Keep Missing‘. This was a fabulous summary of the risks and governance required for using AI in Education. Richard broke it down really clearly and with helpful examples. Who’s accountable when the AI goes rogue? What happens if data leaks out where it wasn’t supposed to? How do we ensure that a human is always in control (and that AI doesn’t start to do more and more behind the scenes without anyone realising).
I think for me having spent years doing compliance documentation – this really does highlight why it’s important to do the documentation and review it regularly. Yes, it can feel like a chore but it forces you to reflect on things you’d accept without thought otherwise. When I worked at Edinburgh, I felt that completing a Data Protection Impact Assessment (DPIA) was a good chance to get your teeth into the inner workings of a tool and really think about how it worked behind the scenes and consider the risks to the institution. It helped me understand how the tools worked much more deeply and forced me to spend the time to ask the questions I needed answers to, working with the suppliers (who were normally very very forthcoming and helpful with the answers). It’s the same for AI – we need to understand how suppliers are implementing it and what the risks are. It’s easy to use AI and not consider what happens behind the scenes, it’s just so accessible now.
Richard ended his talk with a take-away ‘Innovation and governance are not opposing forces. They’re a sequence.’ This is so true.
Slightly later on , there were two talks which covered how the speakers were using AI to support assessment which were very thought provoking.
Dr Noorhan Abbas (University of Leeds) who’s talk was titled ‘From Feedback Delivery to Feedback Interpretation‘ showed a tool they’d developed at Leeds which allowed students to have a discussion (via AI) about the feedback given by staff. I thought this was a great idea – the marker marks and writes the feedback. If anything is unclear, the student can talk it through with an AI. It was a carefully designed and trained AI. Twenty minutes was not enough time to cover this really interesting tool, which I think has masses of potential.
Professor James Hutson (Lindenwood University) & Kyle Poyer (Integrevise) who talked about ‘What the Grade Misses: How AI Supported Oral Assessment Reveals Student Understanding‘. Again, another really interesting concept where they shifted trying to work out whether a submission was generated by AI to whether the student submitting it actually understood the work they were submitting. They did this by, at time of submission, using AI to complete an oral viva with the student based on the students submission. The AI would then flag to staff if there were any concerns. Again, a really interesting idea! There were a lot of questions from the audience about how to stop students using AI to cheat during the oral viva and most of the questions and discussion during the talk were about that. It’s absolutely true that technology is moving so quickly that it is getting easier to cheat in lots of very innovative way (I don’t even want to think about AI glasses and how they might be getting used) but this assumes that all students are wilfully cheating (most aren’t or if they are, they are doing it unwittingly – there’s a good summary in Miles, Campbell and Ruxton (2022)).
It also points to something I think we’re reluctant to confront: rather than adapt how we assess, we tend to add complexity to existing assessments in an effort to prevent cheating. We know an essay is an easy artefact for AI to produce, yet we keep setting essays and then try to retrofit assurances of academic conduct around the writing process, instead of rethinking the assessment itself (which, I’ll admit, is not easy…). But remember, cheating has always been possible for anyone who really wants to cheat…..
