AI detectors are advertised as a fast, easy effective ways to uncover unwanted AI writing. It’s a whole new industry that has grown with the rise of AI writing tools. There’s just one problem: these tools have significant issues when it comes to accuracy and bias that raise serious questions about their usefulness.
This isn’t exactly news. In July 2023, Ars Technica revealed that AI detectors believed that the US Constitution was written by AI. While the Founding Fathers definitely could have beat this with some smart paraphrasing and carefully-placed typos, it’s clear that AI detection tools are perhaps not as reliable as they claim to be.
AI detectors may have a purpose, particularly in academia, but they can create more issues than they solve. For example, false positive results have the potential to ruin a student’s academic career. In professional settings, there is also a danger of misplaced accusations breaking trust in a professional relationship.
Despite these concerns, the AI detector industry is growing. These tools do have some value to offer to the people who use them, so long as they understand the limitations and use their own judgement for the final call.
The AI content detector market is growing fast. Valued at $25.13 billion in 2023, it is projected to hit $240.7 billion by 2030 – that’s a CAGR of 29.51%.
The leading players in this industry (among others) are Originality.ai, Copyleaks, ZeroGPT, Turniton. There are also less-known tools like Undetectable.ai which aggregate the results of multiple AI detectors for a more rounded result.
But – and this is a big but – research has shown that the overall accuracy of AI detection tools can be as low as 27.9%, with the best tools hitting a 50% success rate in identifying AI-generated text. Compare that with Originality.ai’s recent claims that it has an accuracy rate of up to 99% with a false positive rate of less than 2%.
That’s quite a large gap between expectation and reality!
Billions of dollars are being funneled into an industry that, most of the time, cannot be relied upon to give accurate results. As someone who has been around the digital marketing space for a couple of decades, this is a remarkable triumph of marketing hype!
We were all blown away by ChatGPT when it launched in late 2022. Up to this point, conversational chatbots were limited in their usefulness, with more experimental models like the Tay chatbot going rogue and causing international embarrassment for its developer, Microsoft.
ChatGPT changed all of this in an instant. It sounded natural, it seemed to know everything, and it could write. Many, like me, were amazed at how human it felt. It didn’t take long for the world’s students and entrepreneurs to wake up to the opportunities of AI writing.
Just as there was an explosion of AI-powered writing tools, a new breed of content detectors emerged alongside them. These tools promised to distinguish between machine-generated text and authentic human writing.
The target market is two distinct groups of people:
The second category of consumer is a great example of new-market disruption, or creating an entirely new target market as a by-product of their service. Whether the writer is covering up their tracks or simply trying to make sure their human writing doesn’t set off a false positive result, it’s not a service that would be needed if AI detectors didn’t exist!
Unfortunately, AI detection tools have capitalized on fear-based marketing tactics by targeting the insecurities of people around AI writing. But are their tools useful, and do they help people? For now, sadly, the answer is no. This mostly down to the limitations of their design.
AI content detectors work by analyzing the patterns and statistical properties of a piece of content to determine whether it was AI-generated or not. Their training datasets typically contain human and machine-generated text, helping them to distinguish between the two.
| Perplexity | Measures how “surprised” the detector is by the text. A low perplexity score can indicate AI writing. |
| Burstiness | Measures variation in sentence length and structure. AI writing tends to be more uniform in structure than human. |
| Stylometric features | Analyzes word choices, punctuation, and other linguistic markers commonly associated with AI writing |
None of these are foolproof, but put together, they can paint an overall picture that users typically see in a final score.
These algorithms are smart, but they are only as good as the training and datasets they have. This is a topic that AI detector companies shy away from, since it reveals a major flaw in their tools that impact the results significantly.
AI language models are developing at a furious pace, which means that AI detectors are constantly playing catch-up. AI detectors also need to account for human cunning. Someone who understands how AI detectors work and what they are looking for can use these same tools to fix their writing.
People have even developed language models like GPT-4chan (defunct) that are trained to evade AI detection. These language models are effective because they use datasets that AI detection tools are unlikely to include in their AI training.
A bigger issue is the lack of transparency around the training data and architecture that commercial AI content generators use. Without access to the inner workings of the models they are checking, AI detectors can only reverse-engineer patterns in an educated guessing game.
All AI detector tools celebrate their cutting-edge use of technology. More often than not, this celebration is a fiesta of dense jargon that flies over the head of the average person. The aim is to impress with supposed tech prowess, not to inform.
Bad writing isn’t the problem here. What snippets like this point towards is another issue in the AI detection industry: the lack of global standards and consistency. At the time of writing, none of these platforms have moved towards standardization, nor are they adopting existing standards like ISO:IEC 42001:2003.
There are no standardized benchmarks.
There are no test cases and performance metrics.
There is no way for people to compare the effectiveness of tools to gauge the reliability of AI detectors.
One piece of content passed through multiple AI detectors will yield different results, leading to situations where people argue about which content detector is correct! In the end, it’s hard to know whether to trust the results of any of these tools.
The most accurate AI detector today would come from the companies that create the AI writers. OpenAI could release an detection tool that would accurately detect the results of ChatGPT everytime, but it won’t – because the company fears that it could hurt their bottom line.
With no real industry regulation and plenty of motives to keep the status quo, AI detection companies are free to cater to a fear-based problem – one which they created! – yet they offers no real solution due to the limitations of their tech.
That’s a problem, because the fears that surround AI writing are damaging to people who don’t write with AI. Where there is fear, there is often indiscriminate finger-pointing, which means there are innocent victims.
The unreliability of these tools has proved to be very damaging in education, where AI detectors not only wrongly flagged students, but also discriminated against non-native English speakers. In August 2023, Vanderbilt University chose to disable Turnitin’s AI detection tool, citing serious concerns about its reliability.
Today, educators have resorted to other methods for screening AI that rely more on human cunning than AI analysis. These solutions target student laziness more than anything; a question on a college paper may hide an instruction for AI to write a specific phrase. The student who copy-pastes and prints will be caught red-handed with a quick ctrl+F.
It’s an interesting approach, not least because it reveals that no matter how smart AI might seem to be, the human capacity for creative solutions to problems still triumphs. As a race, we are capable of original thought, flashes of inspiration, creative solutions, and an obstinate will to fail our way to success.
A human who uses AI wisely can achieve a lot more than a human who uses AI as a replacement for their own mind. It doesn’t matter whether it is a student cheating their way through college or a professor blindly trusting the results of an AI detector.
Both suffer from the same logical fallacy: an idea that AI is somehow more intelligent than it actually is. LLMs like ChatGPT may seem smart, but as advanced predictive models, they are simply excellent and predicting the next word in a given sentence.
If anything, overreliance on AI can make humans lazier when it comes to decisionmaking. That’s a net negative, since advanced predictive models cannot innovate, only predict. A future without human innovation is a future with stalled progress.
The companies that provide AI detection tools are well aware of the failings of their product. Despite this, their marketing and sales content frequently points to their tools as reliable bellwethers of human deceit. But if academic institutions are rejecting AI detection tools, why do so many businesses use them?
I’ve noticed two main reasons cited by business owners and entrepreneurs for rejecting AI writing:
The first is more of a half-truth. Google does penalize AI spam, but the search engine giant is fine with high-quality writing no matter who (or rather, what) writes it.
The second is more understandable, particularly when freelance marketplaces such as Fiverr have AI content policies which effectively allow anyone to sell AI writing without announcing it. This is simply deceptive to the point of fraud.
When clients are paying $400 for a human-written article and get back something that looks like it took five minutes in an AI writing tool, that’s a problem, especially if it’s low quality enough that Google considers it spam.
It’s easy to understand why many clients are worried about AI content. These fears may drive them to use AI content detectors that are often inaccurate and unreliable. While false accusations in the world of freelance work might not be as damaging as false accusations in academia, they can destroy a business relationship.
The worst thing about these accusations is that they are hard to disprove, at least without the impractical, almost paranoid, step of filming yourself in the act of writing.
LLMs like ChatGPT and Claude are trained on human writing, which means that AI writes like humans. Of course, this also means that humans can and do write like AI! This has led to many professional writers having to contort their writing and actively make it worse, simply to avoid being detected as AI.
Others, like me, have chosen to practice client education. I offer my copywriting and content writing clients a choice of human writing, hybrid content writing, or a fully automated AI system and make sure that they understand exactly what is done by me and what is done by AI.
And, of course, I explain the problematic nature of AI detectors.
The best tool for detecting AI is not an AI detector: it’s human judgement. As a writer who does a lot of work in AI automation, I can spot (bad) AI writing at a thousand paces. Good AI writing, on the other hand, will be almost indistinguishable.
Using a very basic prompt, I asked ChatGPT 4o to write an article about AI detectors. When I ran it through undetectable.ai, it returned a 100% human result. Since Undetectable AI consolidates the results of several different AI detection tools, it effectively exposes that AI tools are unable to detect AI writing effectively – and even demonstrates that complicated prompts aren’t necessary to evade detection.
The real interest here, though, is Undetectable’s humanization feature. Like Quillbot, this tool will “paraphrase” AI writing so it appears more human. This is simply done by doing things like increasing perplexity and lowering burstiness. The results can be a mixed bag, with erratic grammar and questionable stylistic choices.
Humanization is best done by, well, humans – and if your goal is simply to avoid being detected as someone who as used AI writing, why bother when you have 100% human scores across the board?
AI-generated writing has its place in the world, just as AI detectors do. The real issue with both of these tools is how people use them. Those who use AI deceptively would likely practice deception with different tools were the available. It’s a human problem, not an AI one.
Should you use AI detectors? That depends on your needs. With this article, I’ve tried to give you all the information you need to make an informed decision. Rather than demonize AI writing or AI detectors, it’s better to accept that both tools have their limitations and that people can still find these tools useful.
Personally, I think they have some value. They can be helpful for a quick spot check, but they should not be your full and final analysis. Instead, read the content yourself and make your own judgment call.
But don’t ask the question “does this look like AI writing?”
Ask whether it is bad writing.
AI can write well, but it needs good prompting and some human optimization for best results. If the content reads well, answers the search intent (or question) your audience has and ultimately pushes them to take the action you want them to, then it is objectively good writing.
Remember, you’re running a business, not a literary salon. It’s also worth noting that most people can’t tell the difference between AI writing and human writing.
The vast majority of people who read your content are not going to be scrutinizing your piece for AI and a beautiful turn of phrase. Neither is Google. So long as you are aware that the writing you have purchased was created in part or full by AI, there isn’t really anything to get angry about.
If you have paid $400 for an AI-written article sold by a deceptive freelancer, that is a human issue – get a refund.
The people who are demonizing AI writing the most are AI detectors. From a marketing point of view, that’s perfectly understandable. But for the rest of us, it’s a sign to stop and think about why we don’t want AI writing in a particular situation.
Is AI writing really a problem for you, or have you fallen for fear-based marketing tactics?
AI writing has made it possible for any business to scale their content marketing efforts rapidly at a much lower cost. The only sticking point is the quality of the writing – not whether it was written by AI. More content means a larger digital footprint and more opportunities for your audience to find you, leading to more sales and growth.
AI writing is a powerful tool that you should be using as a part of your content marketing strategy, and now is the best time to start. Many businesses are still scared of AI writing due to fear-based marketing.
Now is the time to pull ahead of them.
At Slashrepeat, we create content creation automation engines that can research, write, post, and repurpose quality content for you, 24/7. We don’t focus on writing that “passes AI detectors”, because that’s a complete waste of time.
Instead, we focus on building a custom automation that fits your business needs like a glove. We integrate AI writing and research into our engines with prompts that retain your brand voice and ensure the content is written well. We write on conversion-focused articles. We create engaging content that deals in specific facts and entities, rather than fluffy, generic overviews.
We’re here to help you succeed with a powerful combination of AI, automation, and human smarts. Book a free consultation with us today to find out how we can help you to grow!
Book your consultation now. It’s free and friendly, with no pushy sales tactics. We can start your project as early as tomorrow!
