A detector produces a number, and a number looks like evidence. For a teacher with thirty scripts to read, or a parent whose child has been accused, that is both the appeal and the danger. This article sets out what the research actually shows, what a score can responsibly be used for, and what to do when work is flagged.

What a detector does, and does not, measure

A detector does not find a watermark or a record of who typed what. It estimates how predictable the text is. GPTZero, for example, is described as scoring text on "perplexity" (how predictable it is) and "burstiness" (how much sentence length and structure vary), with sentence-level highlighting. Turnitin pairs text-similarity matching with an AI-writing detection feature that flags passages statistically likely to have been generated by a language model. Brisk Teaching says it can detect AI-written work by inspecting edit history. Those are the tools' own descriptions, as summarised in our Learning Tools Index.

The common point is that all of them infer. Plain, regular, careful writing is more predictable than loose, varied writing, and that is close to how many students are taught to write.

What the research found

False positives, and who they hit

Researchers tested seven widely used detectors on 91 essays written for the TOEFL English test by non-native English speakers, and on essays by US eighth-graders. The detectors judged the US essays almost perfectly. On the TOEFL essays, the average false-positive rate was 61.22%. 18 of the 91 were flagged as AI-written by all seven detectors, and 89 of the 91 were flagged by at least one. Every essay was written by a person. The authors "strongly caution against the use of GPT detectors in evaluative or educational settings", particularly where they affect non-native speakers (Liang et al., Patterns, 2023).

Human-written essays that detectors called AI

Flagged by all seven detectors19.78%
Average false-positive rate per detector61.22%
Flagged by at least one detector97.8%
0%100%

Every essay was written by a person. The authors report near-perfect accuracy on US eighth-grade essays.

Source: Liang et al., “GPT detectors are biased against non-native English writers”, Patterns (2023); 91 TOEFL essays, seven detectors

This matters for the markets Learner First covers. Many learners in Commonwealth schools write in English as an additional language. The Liang study suggests that group faces a higher risk of false flags, though it tested TOEFL essays, not school pupils in these markets.

Accuracy in general

A separate team tested 12 publicly available detection tools and two commercial systems, Turnitin and PlagiarismCheck. They concluded the tools "are neither accurate nor reliable", with a main bias toward classifying text as human-written, and found that performance worsened when text was obfuscated (Weber-Wulff et al., International Journal for Educational Integrity, 2023). The bias toward "human" means detectors miss AI text as well as flag human text, so a clean score proves nothing either.

What the vendors say

Turnitin has reported a false-positive rate of less than 1%, in figures relayed by Purdue University in January 2024. Purdue's own page, relaying Turnitin's guidance, says the AI percentage "should not be used as the sole basis for action or a definitive grading measure by instructors" (Purdue Online, 2024). Vendors update their models and figures, so check the current ones directly. Note that the vendor figure is about the share of human text flagged across a set, and says little about how it behaves on one particular student's essay.

What a detector score can and cannot do

  • Can: Give a statistical guess about a passage.Cannot: Prove who wrote it.
  • Can: Prompt a conversation about the work.Cannot: Be the only evidence for a penalty.
  • Can: Flag work worth a closer look.Cannot: Tell help from copying, or a draft from a final.
Our editorial summary of the limits of a detector score.

Why a flag is not a finding

A score is a reason to look, not a finding.

Three problems stack up.

  1. A score is a probability. Even a tool with a low false-positive rate will flag some real work when it is used across a large cohort. For the student whose work is flagged, the rate of error across everyone else is no comfort.
  2. The errors are not random. The Liang study found the errors cluster on writers using simpler, more regular English. That is bias, not noise.
  3. The consequences are not symmetrical. A wrong flag can cost a student a grade, a reputation and their trust in the school. In our view, a missed case costs less than a wrong accusation.

Exam bodies are clear about misconduct, but we found nothing in these pages that tells a school how to weigh a detector score. SQA states that submitting generative AI outputs as a learner's own work is malpractice (SQA). England's Department for Education points schools to the Joint Council for Qualifications' guidance on AI misuse and detection and asks teachers and leaders to use professional judgement (DfE). For coursework that counts toward a qualification, follow your board's published procedure.

If work is flagged

If work is flagged as AI-written

If you are a teacher

  1. 1Treat the score as a reason to look, not a finding.
  2. 2Compare with the pupil's other work and drafts.
  3. 3Talk to the pupil first and let them explain the work.

If you are a parent or student

  1. 1Ask what evidence exists beyond the score.
  2. 2Gather drafts, notes, version history and sources.
  3. 3Offer to explain the work aloud, and ask for the school's appeal route.
Process points, not legal advice. Exam coursework has its own formal route through the exam board.

For teachers. Treat the score as a reason to look. Compare the work with the pupil's other writing. Ask for drafts, notes or version history. Then talk to the pupil and ask them to explain a passage, define a word they used or extend an argument aloud. A pupil who wrote it can often do that. Nerves, special educational needs or English as an additional language can make it hard for a pupil who did write it, so a halting answer is not proof either. Follow your school's procedure before any finding. Write down what you asked and what they said.

For parents and students. Ask the school what evidence exists beyond the score. Gather drafts, handwritten notes, browser or document history, and the sources you used. Offer to explain the work aloud. Ask for the school's appeal route in writing. If the work is an exam-board qualification, ask about the board's procedure. Stay calm and factual; a defensive tone makes a hard conversation harder.

What to do instead of relying on a detector

Design tasks so the question is easier to answer. Ask for drafts, planning notes or a short spoken follow-up. Use some in-class writing for a baseline of each pupil's voice. State what AI use is allowed in each task, which we cover in Is using AI for homework cheating?. Teach pupils to acknowledge permitted AI use. These reduce the problem at the source, and they do not depend on a tool being right.

Frequently asked questions

Can AI detectors be trusted? Not as proof. Studies find them unreliable, and the vendor's own guidance says the score should not be used alone. They can prompt a closer look.

Why do detectors flag students who write English as a second language? The tools estimate how predictable text is. Careful, simple, regular English is more predictable. The study found average false-positive rates of about 61% on essays written by non-native English speakers.

My child's work was flagged but they wrote it. What should I do? Ask what evidence exists beyond the score, gather drafts and notes, and offer to have your child explain the work. Ask for the appeal process in writing.

Does a low score mean the work is human? No. Research found a bias toward calling text human-written, and tools performed worse when text was obfuscated.

Should schools ban AI detectors? That is the school's decision. Our view, from the evidence, is that a detector should never be the only basis for a penalty.

Sources

  1. Liang et al., "GPT detectors are biased against non-native English writers," Patterns, 2023 (arXiv:2304.02819v3): https://arxiv.org/html/2304.02819v3
  2. Weber-Wulff et al., "Testing of detection tools for AI-generated text," International Journal for Educational Integrity 19, 2023 (repository abstract): https://eprints.whiterose.ac.uk/207396/
  3. Purdue Online, "Turnitin adding AI writing detection, but instructors should use it with caution" (January 2024): https://www.purdue.edu/online/turnitin-adding-ai-writing-detection-but-instructors-should-use-it-with-caution
  4. SQA, "Generative artificial intelligence (AI) in assessments": https://www.sqa.org.uk/sqa/107607.html
  5. UK Department for Education, "Generative artificial intelligence (AI) in education": https://www.gov.uk/government/publications/generative-artificial-intelligence-in-education/generative-artificial-intelligence-ai-in-education