How AI Detection Actually Works, and Why It's Basically Guessing

I got asked recently how a computer can tell that a paragraph was written by ChatGPT instead of a person. The honest answer surprised the person asking. It doesn't "know" anything. It's making a statistical bet, and most people give that bet way more credit than it deserves.

Here's the mechanism. A language model writes by predicting the next word based on everything that came before it. That's it, that's the whole trick. Type "the cat sat on the" and the model has a very good idea what comes next, because it's seen millions of sentences that end the same way. It's not thinking. It's pattern matching at a scale no human brain manages.

The problem for the model is that this predictability leaves a fingerprint. People don't write the statistically safest sentence every time. We go off on tangents. We use a word that isn't quite the obvious choice because it feels right in the moment. We contradict ourselves halfway through a paragraph and don't bother cleaning it up. That noise is what detectors are actually hunting for, or rather the absence of it.

"Detectors aren't reading meaning. They're reading shape, and shape can be deeply misleading."

The Two Core Metrics: Perplexity and Burstiness

There are two main measurements doing the work here. The first is perplexity, which is a fancy way of asking how surprised a language model is by the words you chose. Predictable, safe word choices score low. A weirder, more personal choice of words scores higher. AI text tends to sit on the low end, because by design it's optimizing for the most likely next word, over and over, sentence after sentence.

The second is burstiness, and I think this one matters more than people realize. Real writing has rhythm the way real speech does. You'll write four words, then thirty. You'll ramble for a sentence and then land a short, blunt one right after. AI generated paragraphs tend to flatten that out. Every sentence lands in roughly the same length range, which sounds fine in theory but reads a little like a metronome once you notice it. I've caught myself doing this in early drafts before I go back and break things up on purpose.

Beyond those two measurements, a lot of modern detection tools are built on neural classifiers, models like RoBERTa that have been trained specifically to spot the difference between human and machine writing. These pick up on things that are harder to name than perplexity or burstiness, stylistic tics, structural habits, the kind of thing a good editor notices without being able to fully explain why.

Watermarking and Its Inherent Vulnerabilities

Then there's watermarking, which works differently from everything above. Instead of analyzing the finished text for patterns, the idea is to have the AI model quietly embed a signal while it's generating the text in the first place, something invisible to a reader but detectable if you have the right key. It's a neat idea on paper. It falls apart the second the text gets edited, paraphrased, or run through a model that never implemented watermarking to begin with, which is most of them.

None of this adds up to something reliable, and that's really the point worth taking away here. Every method above is a probability estimate. Not a fact. Not a fingerprint in the forensic sense. A guess, dressed up in enough math to look authoritative.

Which is exactly why a detector can flag someone who writes in a clean, formal style, or someone writing in their second language, or a technical writer whose vocabulary is naturally narrower than a novelist's. None of those people used AI. They just happen to write in a way that lands in similar statistical territory. The tools aren't reading meaning. They're reading shape, and shape can be misleading.