Alshammari and Rao 2025: six AI text detectors on 49 DeepSeek-V3 answers, paraphrased and humanized

This record joins the studies index as the first to score detectors on DeepSeek text; the detectors were run in March 2025 and the authors state the figures "were recorded in the same month" (arXiv PDF, Introduction and Section V-E, read 9 November 2026).

Alshammari and Rao 2025 chose six detectors from a Texas Tech University Libraries list and from Orenstrakh et al. 2024, two of them paid subscriptions. Five have a card in this index, the Content Detector AI card added on 10 November 2026 (arXiv PDF, Section IV, read 9 November 2026).

Detectors as named in Alshammari and Rao 2025, arXiv 2507.17944, Section IV and references 22 to 30; the reference list gives access dates between 13 March and 19 May 2025.

The human set is 49 question-and-answer pairs posted on Quora and Academia between 2011 and 2021 (22 academic, 6 business, 7 history, 5 life, 4 entertainment, 5 other). DeepSeek-V3 answered the same 49 questions; those answers were paraphrased with QuillBot; DeepThink mode answered them again; and the DeepThink answers were paraphrased in a standard and a "humanized" mode, 294 samples in all (arXiv PDF, Section III and Tables I and II, read 9 November 2026).

Sample as described in Alshammari and Rao 2025, arXiv 2507.17944, Table II. The authors cut parts of some samples to fit detector token limits.

The headline metric is the arithmetic mean of the AI-content percentage each detector returned across a set, which the paper calls the "average accuracy score"; for the human set the mean human-content percentage is used. Recall at a 50% threshold appears in Figures 2 to 4 but is not tabulated. Low scorers, and AI Text Classifier, which returned nothing for paraphrased text, were dropped before the DeepThink phase (arXiv PDF, Sections III and V-A, read 9 November 2026).

Every figure below is printed in Tables III, IV and V of Alshammari and Rao 2025; the base is 49 samples per cell (arXiv PDF, Tables III to V, read 9 November 2026).

Values as printed in Alshammari and Rao 2025, arXiv 2507.17944, Tables III, IV and V. Table V prints the QuillBot standard-paraphrase cell as "84.1.9%" and the prose as "84.1.90%"; this record carries 84.1%. The abstract gives AI Text Classifier 98.4% on human text where Table III prints 98.14%; the table value is used.

Through the QuillBot paraphrase QuillBot fell from 98.4% to 95.49%, Copyleaks from 98.8% to 93.54% and GPTZero from 100% to 92.61%, while the GPT-2 Output Detector fell from 8.15% to 3.65% (arXiv PDF, Tables III and IV, read 9 November 2026). The humanized paraphrase of DeepThink text was the strongest attack in the paper: the authors write that it "reduced the accuracy to 71% for Copyleaks, 58% for QuillBot, and 52% for GPTZero" (Abstract and Section V-E).

On false positives the paper is qualitative. QuillBot and Copyleaks "did not present false positive cases" on the 49 human answers, and the tool with the most was Content Detector AI, "followed by GPT-2 and GPTZero"; no count is printed, so the average human scores of 98.2% for GPTZero and 93.73% for the GPT-2 Output Detector are the only numbers on human text (arXiv PDF, Table III and Section V-E, read 9 November 2026).

A second part of the paper uses DeepSeek itself as a classifier on 24 human and 25 AI passages. Zero-shot prompting labelled 22 of 24 human and 18 of 25 AI passages correctly, 81.6% (accuracy); two-shot and three-shot got all 49 right; five-shot got 24 of 25 AI passages, 98.4% (accuracy); chain-of-thought prompting got 25 of 27 AI and 20 of 22 human passages, 91.8% (accuracy) (arXiv PDF, Section V-2, read 9 November 2026). The passage counts differ between settings without explanation, and none of these figures describes a product in this index.

The authors name "dataset limitations and commercial detector subscriptions" as the two constraints: no DeepSeek dataset existed, so 49 questions were collected by hand, and the GPTZero and Copyleaks subscriptions hit token limits "before finishing 30 testing samples" (arXiv PDF, Section VI, read 9 November 2026). The figures here come from the arXiv preprint, not the later IEEE workshop version, the "average accuracy score" is a mean of displayed percentages rather than a share of samples classified correctly, portions of some samples were deleted to fit token limits, and the recall figures appear only as bar charts.

Five detectors in Alshammari and Rao 2025 have cards in this index; the Content Detector AI card joined on 10 November 2026 with the ninth wave and records the Table III average AI score of 67.27% on 49 DeepSeek-V3 answers, 59.25% after the QuillBot paraphrase (Table IV) and the average human score of 55.2% on 49 human answers, the lowest of the six, with the authors' Section V-E remark that the tool produced the most false positives, a count the paper does not print. Each card records the Table III average AI score on 49 DeepSeek answers and average human score on 49 human answers, plus the Table V humanized-paraphrase score for the three tools in the final phase (arXiv 2507.17944, Tables III and V, read 9 November 2026).

All five appear in the ranking of detectors by test record. No figure from this record enters the false positive rate ranking, because the paper prints no count or rate of human answers flagged, or the false negative rate ranking, which takes printed rates of AI texts missed and never a complement of an average score. The Table V drop from 94.1% to 52% for GPTZero, 99.7% to 71% for Copyleaks and 95.4% to 58% for QuillBot is noted on each card for the adversarial robustness ranking as a paraphrase result on 49 texts. The home page lists which AI detectors exist and who has tested them.

Related records: Orenstrakh 2023 guided the authors' choice of detectors; Krishna 2023 and Perkins 2024 measure paraphrase attacks with larger samples; Pegoraro 2023 scored the GPT-2 Output Detector at a 7.2% true positive rate on ChatGPT answers, near its 8.15% average here (arXiv PDF, Section II, read 9 November 2026).