AI detective? New tool detects AI generated content with 99% accuracy

Researchers have found a way to detect artificial intelligence (AI)-generated theses and written drafts, as students and professionals from different fields are seeking help from the widely known OpenAI ChatGPT chatbot.

The ability of ChatGPT to generate text and conversations that resemble human speech recently caused quite a stir when it was made available to the general public.

However, a study that was released Wednesday in the journal Cell Reports Physical Science found that there are a few telltale signs that can help us tell AI chatbots apart from people.

These indicators were used by the team to create a system that, over 99% of the time, can identify academic science papers produced by AI.

According to Mashable India, writing with the aid of AI currently has some pretty obvious limitations, as it gathers text from various sources without performing any kind of accuracy check, and it is comparable to playing the game Two Truths and a Lie.

Despite the fact that there are many good AI text detectors online, none of them were created with academic writing in mind. For the same reason, the team set out to develop a powerful tool to close the gap.

The main emphasis of the research was on perspective articles, which offer a comprehensive summary of a specific study topic and are frequently written by scientists.

Therefore, the group selected 64 points of view and created 128 articles using ChatGPT that covered the same research topics in order to train the model. When they compared the writing, they noticed predictability, which is a hallmark of AI writing.

Despite ChatGPT’s close resemblance to human writing, humans can still be distinguished from AI because human writing has more intricate paragraph structures and a wider range of sentence lengths, words per sentence, and word counts.

Word choice and punctuation preferences also impact these patterns. Researchers used a 20-feature algorithm to successfully differentiate between human and AI-generated articles, with a 92% success rate.

As a result, the research team developed a better artificial intelligence text detector compared to a commercial one.

Furthermore, they plan to evaluate its applicability to larger datasets and scientific writing styles, focusing on robustness as AI chatbots grow more complex. The model distinguishes between AI and humans, but the research team pointed out that their model can be easily copied and created by others.