Tool claims to sniff out cheaters abusing ChatGPT AI for plagiarism

Cutting corners: People are reacting to ChatGPT’s potential for abuse, developing new tools to catch cheaters and plagiarized material. A student has developed a tool that can potentially (and quickly) discover when a text has been created by the AI rather than by a human writer.

ChatGPT can write code or school essays and any sort of content in response to a human (textual) prompt. OpenAI’s chatbot is also easy to abuse, creating new content to cheat during exams or filling entire websites with worthless – yet superficially convincing – textual garbage. As a response to this, 22-year-old computer science senior at Princeton University, Edward Tian has created a tool named GPTZero.

Tian spent his winter break developing a platform that could detect whether a human or the ChatGPT algorithm created a given essay. GPTZero, which is still in beta, uses two different indicators, “perplexity” and “burstiness,” to identify human-made or AI-based text excerpts. Tian says we “deserve to know” when a text has been created by a self-conscious brain or by a computer algorithm.

Perplexity measures the complexity of text, or rather how much GPTZero is “perplexed” by the text: a higher perplexity level suggests a high level of text complexity, meaning the excerpt was likely written by a human being. When perplexity is low, GPTZero has likely found text patterns it is more familiar with — meaning the text is more likely AI-generated.

here’s a quick demo with john mcphee’s “frame of reference” pic.twitter.com/WphxfxxFdr

— Edward Tian (@edward_the6) January 3, 2023

Burstiness, on the other hand, measures the variations of sentences. Humans tend to write with greater burstiness, using some longer or complex sentences alongside shorter ones. AI sentences tend to be more uniform, which is a red alert for a likely generated text. GPTZero isn’t perfect or foolproof, Tian admits, but the system can indeed provide a quick result to a plagiarism test.

The tool seems to be at least somewhat effective, and pretty interesting for many professionals working with text: after getting released online on January 2, GPTZero has attracted over 7 million visits and used by over 30,000 people. It was so popular that the service crashed, pushing the free platform currently hosting the code to support Tian with more server resources to better manage the increased internet traffic.

Tian explained on Twitter that he wanted to do something to curb growing AI-based plagiarism, bringing some light and transparency to the black box technology the AI is based on. “We really don’t know what’s going on inside” ChatGPT algorithms, Tian said, and GPTZero is design to fight against this approach.