INDEPENDENT EVIDENCE LIBRARY

What third-party research says about AI detection.

A source-linked review of independent studies on detector accuracy, false positives, edited AI text, and fair interpretation.

Explore the studies How to read the evidence

Evidence snapshot

6independent sources
reviewed below

BenchmarksFalse-positive riskFairness & context

Literature reviewed through September 2026

Research disclosure These publications study AI-text detection generally or named third-party products. They do not validate GramProof, which has not yet been evaluated in an independent peer-reviewed study. We link the original sources and describe their limits so you can assess the evidence directly.

EVIDENCE AT A GLANCE

The numbers behind the headlines.

These figures come from different studies and cannot be combined into one GramProof accuracy score. They show why sample design and error type matter.

Comparative scale2,016

detector classifications across 126 essays in Walters’ 2023 comparison.

Study 01

Editing challenge26%

accuracy on machine-paraphrased AI text in Weber-Wulff et al.’s test.

Study 02

Fairness signal61.3%

average false-positive rate on the human-written TOEFL essays tested by Liang et al.

Study 03

Newer design160

full-length papers split evenly across human, AI, hybrid, and humanised conditions.

Study 06

ONE RESULT, TWO POSSIBLE ERRORS

A useful test reports what it catches and what it gets wrong.

False positives label human writing as AI. False negatives miss AI-generated writing. Lowering a decision threshold may catch more AI text while also flagging more human text, so both rates belong beside every accuracy claim.

Human writingFalse-positive risk

AI-generated writingFalse-negative risk

Illustrative bars — not GramProof measurements

READING THE RESEARCH

No single accuracy number tells the whole story.

Detector performance changes with the generator model, detector version, text length, writing domain, editing method, language background, and decision threshold.

Ground truth

Look for a known mix of human, AI, hybrid, and edited texts—not only obvious AI samples.

Both error types

False positives can harm human writers; false negatives can miss AI text. A useful study reports both.

Test date & versions

Models and detectors change quickly. Treat every comparison as a dated snapshot.

Real writing contexts

Full documents, multilingual writers, mixed authorship, and normal editing better reflect actual use.

STUDY COMPARISON

What each paper actually tested

Study	Evidence type	Scope	Primary focus	Read with caution
01 · 2023	Peer-reviewed	16 detectors · 126 essays · 2,016 classifications	Comparative accuracy	The corpus used fully human or fully AI essays. It did not test hybrid authorship, paraphrasing, multilingual writing, or current detector versions.
02 · 2023	Peer-reviewed · Open access	14 detectors · 54 documents · 756 tests	Robustness to editing	There were nine documents per condition and products have changed since testing. The paper is strongest as evidence about failure modes, not today’s ranking.
03 · 2023	Peer-reviewed · Open access	7 detectors · 91 TOEFL essays · 88 US student essays	Fairness & false positives	This is a false-positive and fairness study. It does not measure how accurately those tools detected AI-generated documents overall.
04 · 2024	Peer-reviewed	4 detectors · 2 ChatGPT-4 texts · 1 human text	Cross-tool agreement	Three texts are far too few for a general accuracy estimate. The study demonstrates disagreement, not a stable leaderboard.
05 · 2025	University working paper	4 detectors · multiple topics, lengths, and AI models	Policy tradeoffs	This is a university working paper rather than a peer-reviewed journal article, and its results apply to the tested product versions and corpus.
06 · 2026	Peer-reviewed · Open access	4 detectors · 160 full-length academic papers	Full-length hybrid writing	The known-ground-truth set came from one faculty and used synthetic AI conditions. The 1,163 real submissions cannot establish accuracy or AI prevalence.

THE SOURCE LIBRARY

Six studies worth reading.

Each summary distinguishes what the researchers tested from what their findings can reasonably support.

Peer-reviewed2023

The Effectiveness of Software Designed to Detect AI-Generated Writing

William H. Walters · Open Information Science, 7(1)

16 detectors · 126 essays · 2,016 classifications

Research question

Can widely available detectors reliably distinguish undergraduate essays from essays generated by GPT-3.5 and GPT-4?

Study details

Corpus: 126 essays, split evenly across three authorship groups
AI conditions: 42 GPT-3.5 essays and 42 GPT-4 essays
Human comparator: 42 first-year composition essays
Evaluation: Every document run through 16 detectors; 2,016 outputs coded as AI, human, or uncertain

Key findings

Compared 42 GPT-3.5 essays, 42 GPT-4 essays, and 42 human undergraduate essays.
Three products performed strongly across this 2023 test set, while most tools struggled more with GPT-4 than GPT-3.5.
The paper warns that results depend on the model, document type, detector version, and the threshold used to interpret a score.

Balanced benchmark designThe three ground-truth groups contained the same number of documents.

Ground-truth group	Documents	Purpose
Human undergraduate essays	42	Measure correct human classifications and false positives
GPT-3.5 essays	42	Measure detection of an earlier model
GPT-4 essays	42	Measure detection of a more capable model

Limitations to keep in view

Tests were performed between June 25 and July 12, 2023, so the results describe those product versions.
The study used unmodified AI essays; it did not test hybrid writing or user-edited AI passages.
A detector’s numeric output did not always mean the same thing across products, requiring a study-specific coding rule.

Read the original source 10.1515/opis-2022-0158

Peer-reviewed · Open access2023

Testing of Detection Tools for AI-Generated Text

Debora Weber-Wulff et al. · International Journal for Educational Integrity, 19

14 detectors · 54 documents · 756 tests

Research question

How do 14 detection tools perform on human, translated, AI-generated, manually edited, and machine-paraphrased documents?

Study details

Corpus: 54 documents across six controlled conditions
Tools: 12 public tools plus Turnitin and PlagiarismCheck
Testing volume: 756 document–detector tests
Evaluation: Binary, semi-binary, and logarithmic scoring plus false-positive and false-negative analysis

Key findings

Tested human, translated, AI-generated, manually edited, and machine-paraphrased writing.
All tools scored below 80% overall accuracy in the authors’ evaluation; only five exceeded 70%.
Accuracy on machine-paraphrased AI text fell to 26%, showing how strongly editing conditions can change results.

Average accuracy by document conditionValues reported by the authors across their accuracy approaches. Machine-translated accuracy is derived from the reported 20-point drop from the 96% human baseline.

Human-written96%

Machine-translated human76%

Unedited AI-generated74%

Human-edited AI42%

Machine-paraphrased AI26%

Source: values reported in the linked publication. Bars are a GramProof visualization of those results.

Limitations to keep in view

The tests were conducted from March through May 2023 and cannot represent later detector updates.
Each condition contained nine documents; condition-level percentages should be read with that sample size in mind.
The paper evaluated specific editing and paraphrasing methods, not every way a person might revise AI text.

Read the original source 10.1007/s40979-023-00146-z

Peer-reviewed · Open access2023

GPT Detectors Are Biased Against Non-Native English Writers

Weixin Liang et al. · Patterns, 4(7)

7 detectors · 91 TOEFL essays · 88 US student essays

Research question

Do commonly used GPT detectors classify non-native English writing differently from native English writing?

Study details

Detectors: Seven widely used GPT detectors
Non-native corpus: 91 TOEFL essays from a Chinese forum
Comparison corpus: 88 US eighth-grade essays from the ASAP data set
Additional analysis: Text perplexity, vocabulary changes, and first-author language patterns in ICLR papers

Key findings

The seven tested detectors had an average 61.3% false-positive rate on the TOEFL essays in this data set.
The same detectors were much more accurate on the US eighth-grade essays used for comparison.
The researchers connect the disparity to predictable language patterns and lower measured perplexity.

False-positive exposure in the TOEFL corpusAll three values refer to the 91 human-written TOEFL essays and the seven detectors tested in the paper.

Average false-positive rate61.3%

Flagged unanimously by all detectors19.8%

Flagged by at least one detector97.8%

Source: values reported in the linked publication. Bars are a GramProof visualization of those results.

Limitations to keep in view

The paper tested early-generation detectors and two particular essay corpora.
Differences between the TOEFL and US essay sets may include age, topic, and assessment context as well as language background.
The findings establish risk in the tested setting; they do not imply every detector or multilingual writer will produce the same rates.

Read the original source 10.1016/j.patter.2023.100779

Peer-reviewed2024

Between Human and AI: Assessing the Reliability of AI Text Detection Tools

Valentina Bellini et al. · Current Medical Research and Opinion, 40(3)

4 detectors · 2 ChatGPT-4 texts · 1 human text

Research question

Do four online detectors agree when judging two ChatGPT-4 texts and one human-written medical text?

Study details

Corpus: Two ChatGPT-4 texts generated with different prompts and one human text
Tools: GPTZero, ZeroGPT, Writer ACD, and Originality
Domain: Medical and scientific writing
Outcome: Comparison of each tool’s authorship assessment on the same samples

Key findings

The four tools produced notably different assessments on the same medical-writing samples.
One detector consistently recognized both ChatGPT-4 samples while others were inconsistent or leaned toward human classification.
The very small sample makes this an illustrative comparison rather than a general performance benchmark.

What was comparedA deliberately small, same-text comparison across four online tools.

Element	Count	Interpretation
ChatGPT-4 samples	2	Tests whether the tools recognize generated text
Human-authored samples	1	A minimal check against false classification
Online detectors	4	Shows whether outputs agree on identical inputs

Limitations to keep in view

Only three texts were tested, so the study cannot estimate a general accuracy or false-positive rate.
A single writing domain and one AI-model family were represented.
The value of this study is the visible disagreement among tools, not a product-wide ranking.

Read the original source 10.1080/03007995.2024.2310086

University working paper2025

Artificial Writing and Automated Detection

Brian Jabarian and Alex Imas · Becker Friedman Institute, University of Chicago

4 detectors · multiple topics, lengths, and AI models

Research question

How should detector performance be evaluated when institutions care about both catching AI text and avoiding false accusations?

Study details

Tools: Pangram, OriginalityAI, GPTZero, and RoBERTa
Corpus: Human and AI writing across topics, lengths, and generator models
Stress tests: Short passages and text processed by humanizer tools
Policy analysis: False-positive and false-negative tradeoffs under policy caps

Key findings

Evaluated false-positive and false-negative tradeoffs across commercial and open-source detectors.
Commercial systems outperformed the tested open-source baseline in the authors’ corpus.
Only one tested detector met the paper’s stringent false-positive policy cap without sacrificing AI-text detection.

Policy-centered evaluationThe paper treats detection as a decision problem with two different harms.

Metric	Meaning	Policy concern
False-positive rate	Human text labeled as AI	Risk of a false accusation
False-negative rate	AI text labeled as human	AI use goes undetected
Policy cap	Maximum tolerated error rate	Makes the tradeoff explicit before choosing a threshold

Limitations to keep in view

This is a University of Chicago working paper rather than a peer-reviewed journal article.
Commercial detectors can change after a benchmark is run.
Results from the paper’s stimulus set and policy cap should not be generalized to every institution without local validation.

Read the original source BFI Working Paper 2025-116

Peer-reviewed · Open access2026

Who Wrote This? Evaluating the Reliability of AI Detection Tools in Higher Education

Mohammad Hadra, Karleen Cambridge, and Mostefa Mesbah · International Journal for Educational Integrity, 22

4 detectors · 160 full-length academic papers

Research question

How accurately do four current detectors measure fully human, fully AI, hybrid, and humanised full-length academic papers?

Study details

Controlled corpus: 160 papers of at least 4,000 words; 40 in each condition
Tools: Pangram, Turnitin, Copyleaks, and GPTZero
Ground truth: Known AI share for human, AI, hybrid, and humanised groups
Additional corpus: 1,163 real master’s theses scanned without known ground truth

Key findings

Compared 40 human, 40 fully AI, 40 hybrid, and 40 humanised documents with known ground truth.
Performance varied sharply by tool and paper type; several tools substantially underestimated AI content in advanced-model text.
All four tools correctly identified the fully human papers in this specific corpus, while one tool best tracked hybrid and humanised content.

Strict classification accuracy by paper typePercent of controlled papers classified within the authors’ strict correct range. Each condition contained 40 papers.

Detector	Fully AI	Hybrid	Humanised	Fully human
Pangram	65%	92.5%	92.5%	100%
Turnitin	0%	60%	50%	100%
Copyleaks	0%	30%	22.5%	100%
GPTZero	0%	0%	2.5%	100%

Limitations to keep in view

The controlled human papers came from one faculty, limiting disciplinary variation.
The real thesis corpus had no ground truth, so its 45.5% flagged rate is not a verified prevalence estimate.
Detector and generator versions will continue to change; the authors frame the results as time-sensitive.

Read the original source 10.1007/s40979-026-00226-w

METRIC GLOSSARY

Know what the score does—and does not—mean.

Accuracy

The share of all test cases classified correctly. It can hide an unacceptable false-positive rate when the test set is unbalanced.

False-positive rate

The share of human-written samples incorrectly flagged as AI-generated. This is the error most likely to create a false accusation.

False-negative rate

The share of AI-generated samples incorrectly classified as human. Low false positives do not automatically mean low false negatives.

Ground truth

The verified origin and composition of every test document. Without it, a study can describe scores but cannot establish accuracy.

RESPONSIBLE REVIEW

Five checks before acting on a score.

01
Confirm the text is long enough and in a supported language.
02
Check which detector and model version produced the result.
03
Review highlighted passages instead of relying on the document score alone.
04
Compare drafts, sources, revision history, and prior writing.
05
Talk with the writer before making any consequential decision.

GRAMPROOF’S STANDARD

Use detection as a signal, never a verdict.

GramProof results should prompt closer review, not an automatic accusation. Consider source notes, revision history, assignment rules, prior writing, and a conversation with the author.

When GramProof completes independent evaluation, the test design, data conditions, model version, thresholds, and full error rates should be published here.

Try the AI Checker