professors fret about AI-generated coursework

Thanks to the rapid development and evolution of artificial-intelligence (AI) chatbots, students can generate seemingly insightful writing with the click of a button. Although some academics blame these tools for the death of the college essay, a poll of Nature readers suggests that the resulting essays are still easy to flag, and it’s possible to amend existing policies and assignments to address their use.

Nature’s non-scientific, online questionnaire ran from 8 to 21 December and drew 293 self-selected responses, two-thirds of which came from North America and Europe. Through survey questions and open-ended answers, respondents described their first encounters with chatbot essays and what they are doing to address possible misconduct (see ‘Academics weigh in on AI’).

NatureTech hub

One of the most advanced and accessible chatbots, ChatGPT, was launched last November by OpenAI in San Francisco, California. ChatGPT can mimic natural conversations in response to prompts, including requests for essays or even queries about debugging computer code. The survey affirms that many professors are now considering how students might use — or misuse — AI to complete assignments. Only 20% of respondents have encountered this behaviour in their courses — or witnessed it at their universities — but roughly half expressed concern over the increasing proficiency of chatbots, and expect to come across AI-generated essays in the next year (see ‘Chatbot concerns’).

Royce Novak, a historian at the University of St. Thomas in St Paul, Minnesota, says that relying on chatbots to generate coursework “really only became viable for students this semester” and that he has since received a handful of suspected AI papers in his classes. Without a university-sanctioned method for flagging chatbot essays, however, it is unclear how he can address them with his institution’s ethics committee.

New concerns

Only 10% of respondents said that they or their universities have modified existing policies to address AI chatbots. This could be because many institutions already have sufficiently broad codes of conduct to cover chatbot misuse alongside other forms of misconduct.

Jon May, a professor of psychology at the University of Plymouth, UK, typically runs assignments through the plagiarism-detection program Turnitin before handing them over to student graders. In the past month or so, he has started slipping ChatGPT essays in to determine whether his existing methods can detect the fakes. At present, they’re pretty easy to pick out, May says, but warns that “we’re in an arms race with automation” as AI platforms get better at mimicking human speech. A number of automated tools for detecting chatbot-generated content have been developed, including the GPT-2 Output Detector. In one preprint posted last month¹, chatbot-written abstracts were able to fool both humans and software tools. An online plagiarism checker missed 100% of the generated abstracts, and the GPT-2 Output Detector and human readers missed about one-third.

May is considering adding oral components to his written assignments, and fully expects programs such as Turnitin to incorporate AI-specific plagiarism scans, something the company is working to do, according to a blogpost. Novak now assigns intermediate steps, such as outlines and drafts, that document the writing process.

These changes are meant to retain the critical-thinking skills that teachers are looking to teach their students. But not every instructor views chatbots negatively, and several respondents plan to integrate them into their teaching. “It’s like every other kind of tool; it’s how people choose to use them,” says Kateřina Bezányiová, a zoology master’s student at Charles University in Prague, who teaches undergraduate courses.

ChatGPT, Bezányiová notes, doesn’t just help students to shirk their homework. It can function as an assistant by helping students to generate ideas, overcome writer’s block, draft outlines or assess sentences for clarity — and using it in that way is no different from asking a friend, she adds. Indeed, at least four articles list ChatGPT as a co-author. Such applications, Bezányiová adds, might be especially useful for students for whom English is a second language; students still need to apply their own critical thinking to their writing, but the result is a more polished final product.

“Someone can have great English, but if you are not a native English speaker, there is this spark or style that you miss,” she says. “I think that’s where the chatbots can help, definitely, to make the papers shine.”