DetectGPT zero-shot AI detector
Vendor Mitchell, Lee, Khazatsky, Manning and Finn (Stanford University) Official site github.com/eric-mitchell/detect-gpt How this card is sourced
DetectGPT classified 25 of 54 documents correctly in Weber-Wulff 2023, a 46% binary accuracy that placed it 11th of 14 tools, with 0 false accusations in 18 human-written documents (arXiv 2306.15666, Tables 5 and 9, read 9 November 2026). DetectGPT is a research method and its code, not a product: the 2023 ICML paper by Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning and Chelsea Finn of Stanford University describes a zero-shot test that asks whether a passage sits in a region of negative curvature of a language model's log probability function, and the official implementation is on GitHub under the MIT License (arXiv 2301.11305 and github.com/eric-mitchell/detect-gpt, read 9 November 2026). No vendor hosts it: the interactive demo the README links at detectgpt.ericmitchell.ai returned 404 on 9 November 2026, and the Wayback Machine index records its last 200 capture on 20 August 2023 and 404 responses from 26 November 2023. The card joins the detector index as the second open-source research tool after GLTR, and how this card is sourced explains why the paper stands in for a product page.
What DetectGPT detects
| Field | Value | Source |
|---|---|---|
| Content types detected | Text | arxiv.org/abs/2301.11305 |
| Generators covered | Any language model whose token log probabilities can be evaluated (white-box setting); the paper scores a passage with the model suspected of writing it, Evaluated in the paper on GPT-2 (1.5B), OPT-2.7B, GPT-Neo-2.7B, GPT-J (6B) and GPT-NeoX (20B), and on GPT-3 and Jurassic-2 Jumbo (175B) via API log probabilities, Studies ran it on GPT-2 XL, OPT-13B, GPT-2 Medium and GPT-3.5 text-davinci-003 (Krishna 2023, Sadasivan 2023) and on ChatGPT documents (Weber-Wulff 2023) | arxiv.org/pdf/2301.11305 |
| Input limits | No product limits: the code runs the paper's experiments from the command line (python scripts in paper_scripts/) and needs a source model to score log probabilities plus a mask-filling model (T5-3B in most experiments, T5-11B for the largest models, mT5-3B for the German WMT16 experiment). Each passage is perturbed about 100 times, masking 2-word spans until 15% of the words are replaced, and the normalized perturbation discrepancy is compared with a threshold; the paper reports that a threshold of slightly below 0.1 separates human and model texts across its datasets and that accuracy converges at 100 perturbations. Passages are roughly paragraph length (500 samples per dataset in Table 1). | arxiv.org/pdf/2301.11305 |
| Free tier | Free and open source (MIT License): the full implementation is on GitHub and runs locally with a Python environment, the source model and T5. The hosted demo at detectgpt.ericmitchell.ai is gone (404 on 9 November 2026) | github.com/eric-mitchell/detect-gpt |
| Launch year | 2023 | arxiv.org/abs/2301.11305 |
| Ownership | Open-source research method and code, not a product: the official implementation of the experiments in the paper DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature by Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning and Chelsea Finn of Stanford University (arXiv 2301.11305, submitted 26 January 2023, v2 23 July 2023; Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 2023). The GitHub repository eric-mitchell/detect-gpt is under the MIT License (Copyright 2023 Eric Mitchell), was created on February 3, 2023 and last pushed on March 4, 2023 (commit Create LICENSE), with 479 stars. The README links an interactive demo at detectgpt.ericmitchell.ai, which returned 404 on 9 November 2026; the Wayback Machine index shows its last 200 capture on 20 August 2023 and 404 responses from 26 November 2023 onward. No company hosts DetectGPT; the three indexed studies that scored it ran the method or the 2023 demo themselves. | github.com/eric-mitchell/detect-gpt |
| Independent studies | 4 records | Studies index |
DetectGPT analyses text and needs the model suspected of writing it. The paper states its hypothesis in one sentence: "minor rewrites of model-generated text tend to have lower log probability under the model than the original sample, while minor rewrites of human-written text may have higher or lower log probability than the original sample" (arXiv 2301.11305, Section 1, read 9 November 2026). The method perturbs a passage with a mask-filling model, T5-3B in most experiments, masking 2-word spans until 15% of the words are replaced, scores the original and about 100 perturbations under the source model, and compares the normalized gap with a threshold; the authors report that "a threshold of slightly below 0.1 separates human and model texts across data distributions" and that accuracy "continues to improve until 100 perturbations, where it converges" (Sections 4 and 5, read 9 November 2026). The method is "zero-shot" because it trains no classifier and needs no labelled examples.
The paper evaluated five open models, GPT-2 at 1.5B parameters, OPT-2.7B, GPT-Neo-2.7B, GPT-J at 6B and GPT-NeoX at 20B, on 500 samples each from XSum news, SQuAD Wikipedia contexts and WritingPrompts stories, and two 175B API models, GPT-3 and Jurassic-2 Jumbo (arXiv 2301.11305, Tables 1 and 2, read 9 November 2026). The authors name the white-box assumption as the method's limit: it needs the model's log probabilities, which "for models behind APIs that do provide probabilities (such as GPT-3)" cost money, and ChatGPT is named as an exception that does not return them (Sections 3 and 6). Every dataset is English except one German WMT16 experiment with mT5 as the perturbation model, so the languages field is empty.
The repository publishes the paper's experiment scripts and nothing else. It was created on February 3, 2023, last pushed on March 4, 2023 with the commit "Create LICENSE", and held 479 stars on 9 November 2026 (GitHub API, read 9 November 2026). Two cards share DetectGPT's lineage: GLTR, the 2019 open-source display that colours words by GPT-2 rank, and the GPT-2 Output Detector, OpenAI's supervised RoBERTa classifier, which the DetectGPT paper compares itself against in Figure 4 and which Sadasivan 2023 tested alongside DetectGPT.
Limits and plans
DetectGPT publishes no character limit, no plan and no price. The README's quickstart creates a Python environment, installs requirements and runs "any of the scripts (or just individual commands) in paper_scripts/"; the WritingPrompts experiments need a Kaggle download (GitHub README, read 9 November 2026). Because the code is free software under the MIT License, there is no plans table, and because no server is shipped, the API field on this card is marked unavailable with the repository as its documentation.
The cost is compute rather than money. The authors write that "DetectGPT is more compute-intensive than other methods for detection, as it requires sampling and scoring the set of perturbations for each candidate passage, rather than just the candidate passage" (arXiv 2301.11305, Section 6, read 9 November 2026). The paper and repository publish no privacy policy and no retention statement; a reader who runs the scripts keeps the text on their own machine, and the hosted demo that would have raised the question is gone.
What the vendor claims
We find DetectGPT is more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. (Abstract.) Table 1 reports average AUROC of 0.97 on XSum news, 0.92 on SQuAD Wikipedia contexts and 0.97 on WritingPrompts across five open models, 500 samples each; Table 2 reports 0.84 on GPT-3 and 0.77 on Jurassic-2 Jumbo for PubMedQA, 0.84 and 0.84 for XSum, 0.87 and 0.84 for WritingPrompts. The paper publishes no accuracy percentage and no false positive rate; AUROC is a threshold-free measure.
The paper's abstract carries the method's one headline figure: DetectGPT is "more discriminative than existing zero-shot methods for model sample detection, notably improving detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT" (arXiv 2301.11305, read 9 November 2026). Table 1 reports average AUROC of 0.97 on XSum, 0.92 on SQuAD and 0.97 on WritingPrompts across the five open models, the best in 14 of 15 dataset and model combinations by "0.06 AUROC improvement on average", and Table 2 reports 0.84 on GPT-3 and 0.77 on Jurassic-2 for PubMedQA, 0.84 and 0.84 for XSum, 0.87 and 0.84 for WritingPrompts (read 9 November 2026). AUROC is a threshold-free measure, so the paper states no accuracy percentage and no false positive rate, and the claim gap ranking takes nothing from it.
On edited text the authors simulated revision by replacing 5-word spans with T5 output and report that "DetectGPT maintains detection AUROC above 0.8 even when nearly a quarter of the text in model samples has been replaced" (arXiv 2301.11305, Figure 5, read 9 November 2026). They also cite the two paraphrase papers indexed below as showing "the effectiveness of paraphrasing as a tool for evading detection, suggesting an important area of study for future work" (Section 2). The limitations section names the white-box assumption, the dependence on "a reasonable perturbation function" and the compute cost, and the discussion frames the goal as tools for "teachers, readers, and consumers" who "are likely to demand tools for verifying the human origin of certain content".
What independent studies found
- AIDI-S-0001Weber-Wulff 20232023
DetectGPT: 25 of 54 documents correct (46% binary accuracy, rank 11 of 14; per class 9 of 9 human, 5 of 9 machine-translated, 2 of 9 and 8 of 9 on the two AI classes, 0 of 9 manually edited, 1 of 9 QuillBot-paraphrased), 543 of 864 points (63%, rank 10) on the logarithmic method; 0 false accusations in 18 human-written documents; 13 of 36 AI documents undetected (36.1%). Tested March 7 to 28, 2023 through the public demo.
- AIDI-S-0010Sadasivan 20232023
DetectGPT AUROC fell from 96.5% to 59.8% after one T5 paraphrase round on 1,000 GPT-2 Medium XSum passages (Section 2.4) and to 25.2% under recursive paraphrasing (introduction); the authors ran the method themselves in four variants.
- AIDI-S-0011Krishna 20232023
One DIPPER paraphrase at a fixed 1% false positive rate cut DetectGPT's detection accuracy on GPT-2 XL text from 70.3% to 4.6% on 3,000 Wikipedia prompts (Table 1) and from 74.9% to 7.6% on 3,000 long-form QA pairs (Figure 3); OPT-13B 14.3% to 0.3%; on GPT-3.5 text-davinci-003 the method scored 0% at 1% FPR, so its cells are given at 20% FPR: 26.5% to 4.5% on 200 samples (Table 1) and 67.0% to 14.0% on 100 samples (Figure 3).
- AIDI-S-0018Pegoraro 20232023
Run as the Mitchell et al. zero-shot method: true positive rate 18.1% on ChatGPT responses and true negative rate 80.0% on human responses on a ChatGPT and human Q&A benchmark, March 2023 (Table I).
Three records in the studies index scored DetectGPT. Weber-Wulff 2023 ran the public demo on 54 documents between March 7 and 28, 2023 and reports 25 of 54 correct (46%, rank 11 of 14) on the binary count and 543 of 864 points (63%, rank 10) on the logarithmic method, with 0 false accusations in 18 human documents and 13 of 36 AI documents undetected (36.1%), of which 5 of 9 were manually edited and 7 of 9 QuillBot-paraphrased ChatGPT texts (arXiv 2306.15666, Tables 5 and 8 to 10, read 9 November 2026). Per class the tool scored 9 of 9 human, 5 of 9 machine-translated, 2 of 9 and 8 of 9 on the two unedited AI classes, 0 of 9 edited and 1 of 9 paraphrased (Table 5).
Krishna 2023 measured the paraphrase question directly. One pass through the DIPPER paraphraser at a fixed 1% false positive rate cut DetectGPT's detection accuracy on GPT-2 XL text from 70.3% to 4.6% on 3,000 Wikipedia prompts and from 74.9% to 7.6% on 3,000 long-form QA pairs, and on OPT-13B from 14.3% to 0.3% (arXiv 2303.13408, Table 1 and Figure 3, read 9 November 2026). On GPT-3.5 text-davinci-003 the method "scored 0% at 1% FPR", so the paper gives those cells at 20% FPR: 26.5% to 4.5% on 200 samples and 67.0% to 14.0% on 100 samples. The 7.6% at 1% FPR is the value the robustness ranking takes; the 14.0% sits at a different operating point and is quoted in the card's base.
Sadasivan 2023 ran four DetectGPT variants on 1,000 GPT-2 Medium XSum passages and reports AUROC falling from 96.5% to 59.8% after one T5 paraphrase round and to 25.2% under recursive paraphrasing (arXiv 2303.11156, Section 2.4 and introduction, read 9 November 2026). An AUROC is not a detection share, so the ranking quotes it without ranking it. The three records agree on one point the paper itself flagged: the curvature signal is strong on unedited model text and weak once the text is rewritten.
Pegoraro 2023, added 9 November 2026 as the fourth record, ran the Mitchell et al. zero-shot method among 20 detection methods on a ChatGPT and human question-and-answer benchmark in March 2023 and records an 18.1% true positive rate on ChatGPT responses and an 80.0% true negative rate on human responses (arXiv 2304.01487, Table I, read 9 November 2026); no ranking field takes a value from the printed rates.
Where DetectGPT appears in the index
Rankings
- AIDI-R-01AI detectors ranked by independent testing Two records added in the fifth wave on 9 November 2026, Pegoraro 2023 and Fleckenstein 2024, changed nine counts and none: Pegoraro 2023 tested ZeroGPT, GPTZero, Copyleaks, Originality.ai, Writer, Content at Scale (BrandWell), DetectGPT, the GPT-2 Output Detector and GLTR, lifting GPTZero to 12, the GPT-2 Output Detector to 10, Copyleaks to 7 and Originality.ai and ZeroGPT to 6; Fleckenstein 2024 tested 289 teachers and no software, so it counts for no card ([Fleckenstein 2024 record](/studies/fleckenstein-2024-teachers/)). Illuminarty, one of four cards added in the same wave, joined with its one image record from Ha and Passananti 2024, so 18 of 72 cards now qualify.
- AIDI-R-02AI detector false positive rates, ranked
- AIDI-R-08AI detectors ranked by published robustness to paraphrased and edited text
- AIDI-R-09AI detector false negative rates, ranked
- AIDI-R-17AI detectors ranked by the generator models the vendor names
- AIDI-R-18AI detectors ranked by study records on non-English and non-native text
- AIDI-R-20AI detectors ranked by launch year, with every change of hands and withdrawal the cards record
Fits these use cases
- Not listed in a use-case view at this index version.
DetectGPT qualifies for four rankings, read 9 November 2026 in the data file. In AI detectors ranked by independent studies the card counts 3 records; in false positive rates it ranks on 0 of 18 in Weber-Wulff 2023; in false negative rates on 13 of 36 (36.1%) in the same study; and in robustness to paraphrase on 7.6% of 3,000 passages still detected after a DIPPER paraphrase in Krishna 2023, the lowest ranked figure on that page. It is absent from the free tier, language, retention and claim gap rankings, which need a word allowance, a language count, a policy and an accuracy percentage the method does not publish.
The card also appears on the open-source view with the MIT License and the last push of March 4, 2023, and in the question page on how AI detectors work it belongs with GLTR as a statistical test rather than a trained classifier. No audience view lists it, because it states no API, no integrations and no plans. Every detector card on the register shows the same fourteen fields, so the empty price, retention and language cells here can be read beside the commercial tools that fill them.
Questions about DetectGPT
Is DetectGPT a tool I can use?
Only as code: the MIT-licensed implementation runs the paper's scripts locally with a source model and T5, and the hosted demo at detectgpt.ericmitchell.ai returned 404 on 9 November 2026, with the Wayback Machine index showing 404s since 26 November 2023. No vendor hosts DetectGPT as a product.
How accurate is DetectGPT?
The paper reports AUROC, not accuracy: 0.95 on GPT-NeoX news articles against 0.81 for the strongest baseline, and 0.97 average on XSum across five open models, read 9 November 2026. Weber-Wulff 2023 recorded 46% of 54 documents correct through the demo, and Krishna 2023 recorded 70.3% detection at a 1% false positive rate on 3,000 GPT-2 XL passages before paraphrase.
Does DetectGPT survive paraphrasing?
Not in the indexed record: detection fell from 70.3% to 4.6% on Wikipedia prompts and from 74.9% to 7.6% on long-form QA after one DIPPER paraphrase at 1% FPR in Krishna 2023, and AUROC fell from 96.5% to 59.8% after one T5 round in Sadasivan 2023, both read 9 November 2026. The paper's own test kept AUROC above 0.8 with a quarter of the words replaced.
Does DetectGPT work on ChatGPT or GPT-4 text?
The method needs the source model's log probabilities, and the paper names ChatGPT as an API that does not return them, read 9 November 2026. Weber-Wulff 2023 ran ChatGPT documents through the public demo in March 2023 and recorded 13 of 36 undetected.
Sources and revisions
Sources
- arxiv.org/abs/2301.11305Read 9 November 2026
- arxiv.org/pdf/2301.11305Read 9 November 2026
- github.com/eric-mitchell/detect-gptRead 9 November 2026
- detectgpt.ericmitchell.aiRead 8 November 2026
- arxiv.org/pdf/2306.15666Read 9 November 2026
- arxiv.org/pdf/2303.13408Read 9 November 2026
- arxiv.org/pdf/2303.11156Read 9 November 2026
Revision log
- Card created2026-11-09: pegoraro 2023 record added to the study list in the fifth wave
). Illuminarty, one of four cards added in the same wave, joined with its one image record from Ha and Passananti 2024, so 18 of 72 cards now qualify.: register plate](/img/independently-tested-480.webp)
