Pegoraro et al. 2023: to ChatGPT or not to ChatGPT, 20 detection methods and no true positive rate above 47.3%

This record joins the studies index as the widest early sweep of ChatGPT detection: 20 tools and methods in one table, measured on the same question-and-answer benchmark. The authors write that "none of the existing methods can effectively detect ChatGPT-generated content" and that most detectors "are prone to classifying any text as human-written, with a general high TNR of 90% and low TPR" (arXiv 2304.01487, Abstract and Conclusion, read 9 November 2026).

Pegoraro 2023 scored 20 approaches in Table I: 12 online tools and 8 methods from the research literature, all run against the same benchmark. Ten have a card in this index (the Writefull card joined on 10 November 2026 with the tenth wave); the OpenAI classifier has its own record (arXiv 2304.01487, Section II-E and Table I, read 9 November 2026).

Detectors as named in Pegoraro et al. 2023, arXiv 2304.01487, Table I. Online tools were accessed between 23 and 28 March 2023 per the reference list.

The benchmark was built from the question prompts of Guo et al. 2023, put through the OpenAI API to produce 58,546 human responses and 72,966 ChatGPT responses, 131,512 samples answering 24,322 distinct questions from medicine, open-domain question answering, finance and social networking platforms. The authors scored similarity between human and ChatGPT answers with the sentence transformer all-MiniLM-L6-v2, kept the most and least similar pairs, and reduced the set to about 10% of the full dataset; the exact count is not printed (arXiv 2304.01487, Section III-A, read 9 November 2026).

Sample as described in Pegoraro et al. 2023, arXiv 2304.01487, Section III-A.

Two metrics were reported: the true positive rate, the share of ChatGPT responses a tool labelled as generated, and the true negative rate, the share of human responses it labelled as human, each at the tool's own verdict with no common threshold stated (arXiv 2304.01487, Section III-B, read 9 November 2026).

Every figure below is a row of Table I in Pegoraro 2023; the base is the reduced benchmark of ChatGPT responses for TPR and human responses for TNR (arXiv 2304.01487, Table I, read 9 November 2026).

Values as printed in Pegoraro et al. 2023, arXiv 2304.01487, Table I; the five rows not shown (Draft and Goal, Kumarage, Fröhling, Kushnareva, AICheatCheck) range from 13.4% to 27.8% TPR.

The pattern across the table is the one the authors name in their conclusion: high true negative rates and low true positive rates. Sixteen of the 20 scored methods kept a TNR of 90% or more, while the highest TPR in the table is 47.3% and ten rows sit below 25% (arXiv 2304.01487, Table I and Section IV, read 9 November 2026). In the authors' words, "the most effective online tool for detecting generated text can only achieve a success rate of less than 50%".

The three lowest true negative rates belong to the Hugging Face ChatGPT-Detection space at 62.9%, Content at Scale at 79.8% and DetectGPT at 80.0%, the only rows under 89% (arXiv 2304.01487, Table I, read 9 November 2026). The Section II-E tool notes add that the Hugging Face space "tends to over-classify text as being ChatGPT-written" and that Originality.ai "is prone to classify ChatGPT-generated content as real". Three further papers (Cotton et al., Khalil and Er, Mitrovic et al.) appear in Table I without scores because they published no reusable detector.

The paper is a six-page security-lab report, not a peer-reviewed article, and its Table I prints two rates per tool with no confidence intervals, no per-domain breakdown and no count for the reduced benchmark beyond "approximately 10%" of 131,512 samples (arXiv 2304.01487, Section III-A, read 9 November 2026). The authors say they evaluated the non-ChatGPT tools "to the best of our abilities", and each tool's input limits (250 to 5,000 characters for GPTZero, 1,000 or more for the OpenAI classifier, 100 words or more for Originality.ai) applied. The benchmark is question-and-answer text generated in March 2023, so every figure describes the ChatGPT of that month.

What this record does not say: the Table I rates are single-run readings on an unnamed number of short answers at each tool's own default verdict, so they are not a false positive rate or a detection rate for any product on essays, articles or later models.

Ten detectors in Pegoraro 2023 have cards in this index; the Writefull card joined on 10 November 2026 and records its 21.6% true positive rate and 99.3% true negative rate as printed, neither inverted. Each card records its Table I true positive rate on ChatGPT responses and true negative rate on human responses (arXiv 2304.01487, Table I, read 9 November 2026).

All ten appear in the ranking of detectors by test record. The true negative and true positive rates stay in the record and on each card's study list: the false positive rate ranking sorts only rates a study prints on human text, and the false negative rate ranking states that a complement of a sensitivity figure is not taken, so 100 minus a Pegoraro rate enters neither field (PAGE-MAP ruling of 9 November 2026). The home page lists which AI detectors exist and who has tested them.

Related records: Weber-Wulff 2023 ran 14 tools on 54 documents later the same year; Orenstrakh 2023 tested eight of the same tools on computing coursework; the OpenAI classifier record carries the vendor's own 26% detection figure for the tool that scored 31.9% here (arXiv 2304.01487, Table I, read 9 November 2026).

Responses to questions are the nearest published analogue to support replies, and the customer support QA view cites this record as the closest evidence for a QA team scoring short agent replies, read 10 November 2026.