Skip to content 
OpenAI Releases “Free” ChatGPT Detection Tool
OpenAI has released a free ChatGPT detection tool, although it’s not 100% reliable – here’s what you need to know…
After opening Pandora’s box and putting the fear of god into schools, colleges and anyone in the writing profession, OpenAI has now released a ChatGPT detection tool.
And this new tool, which runs alongside its existing products, including ChatGPT, is now available to use.
What does this mean? Is the jig up? Can publishers and writers rest easy, knowing that their content is being written by a human? Not quite. The new tool, while completely free, is not able to detect AI written content with 100% accuracy.
OpenAI’s ChatGPT Detection Tool – Classifier
The new tool, called Classifier, is now available to test out – you can get it here. OpenAI’s goal with Classifier is simple: make distinguishing between human and AI-generated content a process, available at the click of a button.
In the grand scheme of things, it is a nice step forwards for the company. The hype and hysteria around ChatGPT has been unlike anything we’ve seen since the days of COVID, so the fact that OpenAI is taking a pragmatic approach to its usage is fairly commendable.
And there’s now a bunch of third-party, feature-packed ChatGPT alternatives that run on the same tech as ChatGPT.
This is only first-generation software too, OpenAI’s work in this field will continue, so as ChatGPT gets smarter and more nuanced, so too will Classifier, effectively forming a kind of checks and balances system, with respect to AI content.
I mean, something had to give. ChatGPT may be costing OpenAI a reported $3 million a month to run, but the potential job market fallout from its wide adoption by companies pales in comparison.
How Accurate is Classifier?
How accurate is Classifier at detecting ChatGPT content? Here’s the data from the horse’s mouth:
In our evaluations on a ‘challenge set’ of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems.
OpenAI
On top of this, there are a bunch of limitations at play in this iteration of Classifier. First, the content needs to be quite long for it to get a proper feel of whether it was AI or human-created – upwards of 1000 words.
- READ MORE: Does Apple Have Its Own Version Of ChatGPT?
Second, it cannot distinguish between strict data sets, things like numbers and equation sets, as there is very little room for signs of either human or AI flair in these types of phrases and structures.
And thirdly, currently, Classifier will only work on text written in English. Other languages will almost certainly follow soon, but for now Classifier is limited to English.
How did OpenAI train Classifier?
Again, AI is a complex subject. I barely understand the processes involved in creating something as advanced as ChatGPT, let alone developing a tool for finding and understanding whether a piece of content was written by a human or an AI.
Fortunately, OpenAI – true to its name – has a pretty detailed overview of how it developed and trained Classifier. Here’s an overview from the company’s research notes:
Our classifier is a language model fine-tuned on a dataset of pairs of human-written text and AI-written text on the same topic. We collected this dataset from a variety of sources that we believe to be written by humans, such as the pretraining data and human demonstrations on prompts submitted to InstructGPT. We divided each text into a prompt and a response. On these prompts we generated responses from a variety of different language models trained by us and other organizations. For our web app, we adjust the confidence threshold to keep the false positive rate low; in other words, we only mark text as likely AI-written if the classifier is very confident.
OpenAI
Furthermore, OpenAI is now actively working with educators, both college and grade school level, to foster a better understanding of how ChatGPT can be used and abused by students.
I assume Classifier came about following the uproar from schools and colleges in the USA and elsewhere about the potential for cheating with the tool. I mean, if ChatGPT can write a college-grade essay, and the tool is free, only a fool would assume students wouldn’t be using it to tip the balance in their favor.
Keep Reading
Explore more →
