Jabarian and Imas 2025: artificial writing and automated detection, four detectors on 1,992 passages

This record joins the studies index as the largest text sample indexed so far. NBER states on the cover that its working papers are circulated for discussion and have not been peer reviewed (NBER w34223 PDF, page 1, read 9 November 2026).

Jabarian and Imas 2025 ran three commercial detectors and one open-source baseline through their APIs. All four have cards in this index (NBER w34223 PDF, Section 2.2 and Appendix D, read 9 November 2026).

Detectors as named in Jabarian and Imas 2025, NBER w34223, Appendix D.

The human corpus holds 1,992 passages written before 2020: 200 Amazon reviews, 200 blog posts, 300 news articles, 1,000 novel excerpts, 100 restaurant reviews and 192 resume passages from six public datasets. For each of four generators, GPT-4.1, Claude Opus 4, Claude Sonnet 4 and Gemini 2.0 Flash, the authors generated a matching 1,992 AI passages, 7,968 in all (NBER w34223 PDF, Section 2.1 and Table A.1, read 9 November 2026).

Sample as described in Jabarian and Imas 2025, NBER w34223, Sections 2.1 and 4.1.

The paper reports AUROC, then false positive and false negative rates at five fixed thresholds from 0.1 to 0.9, then the same rates at a Youden-optimised threshold per detector and genre, and finally false negative rates with each detector held under a false positive "policy cap". Every AI passage was also rewritten by the StealthGPT default endpoint and rescored (NBER w34223 PDF, Sections 2.2, 3, 4.2 and 5.2, read 9 November 2026).

The three commercial tools are read side by side on the GPTZero vs Pangram vs Originality.ai comparison, the only three-way page in the index because this is the only record that ran all three. Table 3 of Jabarian and Imas 2025 gives false positive rates on the 1,992 human passages at each fixed threshold; Table 7 gives false negative rates by genre and model at Youden-optimised thresholds; Table B.4 gives false negative rates after StealthGPT (NBER w34223 PDF, read 9 November 2026).

Values as printed in Jabarian and Imas 2025, NBER w34223, Tables 3, 4, 6, 7, B.4, B.6 and C.2.

The authors summarise Table 3 by writing that Pangram's false positive rate "is essentially zero across all thresholds 0.5 and greater", that GPTZero's "does not change across the threshold range used, staying at around 0.007", and that RoBERTa labels more than 90% of human passages as AI even at the strictest threshold (NBER w34223 PDF, Section 3.2, read 9 November 2026). On false negatives at fixed thresholds, Table 4 shows GPTZero between 0.0020 and 0.0296 and Pangram between 0.0045 and 0.0382, while Originality.ai reaches 0.4242 on Claude Opus 4 text at the strictest.

On the humanizer test, Jabarian and Imas 2025 states that GPTZero "largely loses its capacity to detect AI-generated text, with FNR scores around 0.50 and above across most genres and LLM models", while Pangram detects nearly 100% of longer rewritten passages (NBER w34223 PDF, Section 4.2 and Table B.4, read 9 November 2026). On stubs under 50 words, Pangram's false positive rate reached at most 0.0250 and GPTZero's 0.0333 (Table B.1); Originality.ai was excluded from that test because its API returned a length filter error on some stubs. The GPT-4.1 row at threshold 0.5 is the one the false negative rate ranking and the GPT-4 question cite, and the stub row is the one the minimum word count question cites; the repeatability question reads the authors' call for routine audits.

Under the paper's policy caps, each detector's threshold is pushed until its false positive rate meets a target. Table C.2 shows that at a cap of 0.0001 GPTZero misses every AI passage on all four models, while at a cap of 0.01 its false negative rate falls to between 0.0020 and 0.0291; Pangram sits between 0.0005 and 0.0030 at the 0.01 cap (NBER w34223 PDF, Table C.2, read 9 November 2026). The authors conclude that Pangram alone meets a strict cap of 0.005 without sacrificing detection.

The paper is a working paper and carries NBER's standard note that it has not been peer reviewed. The authors write that they do not expect the results to remain constant over time and describe detection as an arms race between detectors, language models, humanizers and user behaviour, recommending routine published audits (NBER w34223 PDF, Section 1, read 9 November 2026).

The humanizer test used one tool, StealthGPT, at its default rewrite setting, and the evaluation treats every passage as either fully human or fully AI; the authors note that rephrasing of a human draft or grammar correction by an AI app is not separated out (NBER w34223 PDF, Sections 1 and 4.2, read 9 November 2026). The disclosure names the Booth Center for Applied Artificial Intelligence, the Becker Friedman Institute and the Google Cloud Research Program as funders and states that the authors have no conflicts of interest.

What this record does not say: the Youden thresholds and policy caps were fitted on this corpus by the authors, not set by the vendors, so a rate in this record is not the rate a user of the product's default verdict would see, and no figure here is a claim about any detector's current build.

All four detectors in Jabarian and Imas 2025 have cards in this index. Each card records the Table 3 false positive rate at threshold 0.5 on 1,992 human passages and the Table B.4 range after StealthGPT (NBER w34223 PDF, read 9 November 2026).

The three commercial tools appear in the ranking of detectors by independent test record and the Table 3 figures feed the false positive rate ranking. How this record is sourced explains why a thresholded research figure sits apart from a vendor's verdict label, and the home page lists which AI detectors exist and who has tested them.

Related records: Van Vlasselaer 2026 tested Pangram, GPTZero, Copyleaks and Turnitin on 160 academic papers; Weber-Wulff 2023 is the multi-tool test the authors cite as out of date (NBER w34223, read 9 November 2026).