By India Pemberton
AI detection software is almost as unreliable as AI – shocker.
It’s now every man, woman, and Large Language Model for themselves. You came up with that sentence yourself? AI got to it first, sorry!
Since generative AI software has become an extra appendage for some, AI usage detection software has assumed a critical role in universities’ marking process.
However, AI detection is working against Generative AI models of being undetectable. Meaning its sensitivity can be so high it sometimes flags just single words.
Many universities use Turnitin’s proprietary AI detection feature. The company claims a 98 per cent accuracy rate.
A Melbourne university student Jessica Mirabelli was found guilty under Turnitin’s unpredictable gavel.
Mirabelli’s assignment was a reflection on data from teaching pracs.
“I was talking about my personal experience, how is this getting flagged for AI?” she said.
She had to wait almost four months for the decision to be amended.
Jessica is studying a Bachelor of Education at Australian Catholic University’s Fitzroy campus.
She was on teaching placement when notified that a high percentage of her paper was generated by AI, three weeks after the paper was submitted in May.
“The unit was a semester long, if I failed that critical task, I would’ve failed the whole unit,” she said.
The university offered no explanation or indication of what triggered Turnitin’s result, and no information on how to appeal the result.
“I went through ask ACU and like nobody could help me,” Mirabelli said. “They said there’s a backlog, we can’t do anything about it, you just have to wait.”
Panicked, and knowing she was an honest student, Jessica had to take the initiative.
“I can’t handle this anymore; I need to know what’s going on. I need to know if I need to retake [this unit], I don’t want to finish my course late,” she said she told her tutor.
She compiled proof that she didn’t use AI and sent it to her tutor, only to be told her tutor was simply the middleman between Turnitin and the Academic Integrity team.
She received an email almost four months later in August from her lecturer informing her the investigation has now been resolved.
This meant Mirabelli received no grade for this unit on her semester one transcript, which was only updated transcript at the end of semester two.
Jessica said these “months and months of waiting” and being “in limbo” took a toll on her mental wellbeing.
“I was trying to teach and learn and in the back of my mind I’m repeating ‘I’m gonna have to redo this unit’.”
Like many others across the globe, the university has since stopped using Turnitin’s AI detection feature after just 18 months, due to too many false positives.
Featured Image: Generative AI can be a nightmare for students. Photo: Matheus Bertelli/Pexels/CC
