DAMAGE 2025: nineteen AI humanizers against GPTZero, Binoculars, RADAR and a Pangram Labs model

The record joins the studies index as the first to audit humanizer tools themselves: the authors sorted 19 paraphrasers and humanizers into three quality tiers and then tested whether a detector can learn to see through them (arXiv PDF, Sections 3 and 4, read 9 November 2026). The authors are the vendor of one of the detectors tested, and this record keeps that fact beside every figure.

Five detection methods appear in the results tables. Two belong to the authors' company: an "LLM Baseline" trained with no humanized text, and DAMAGE, the same architecture (Mistral NeMo, about 12 billion parameters, LoRA adapters, 512-token context) trained with humanized text added. The paper does not state whether DAMAGE is the shipping Pangram product, so the Pangram card is linked for the vendor, not for a product version (arXiv PDF, Sections 5.3 and 6.1, read 9 November 2026).

Detectors as named in Masrour, Emi and Spero 2025, arXiv 2501.03437, Tables 3, 4 and 7. No detector versions or access dates are printed.

The humanizers are the second set of tools under test. Table 1 names 18 (the paraphrasers DIPPER, Grammarly and QuillBot plus 15 humanizers); the Table 9 audit names 19, adding GPTInf and AIHumanizer.com and omitting Humanizer.com, while the text says 19 throughout. Five vendors audited as humanizers also have a detector card in this index; the study scored their rewriting tools, not their detectors (arXiv PDF, Tables 1 and 9, read 9 November 2026).

Tiers as printed in Table 9 of Masrour, Emi and Spero 2025; the tier describes fluency and faithfulness, "not their effectiveness at bypassing AI detectors" (Section 4.5).

The evaluation text is student writing. Seven held-out essay corpora supply the human side (PERSUADE 2.0, PII Detection, CommonLit, ELLIPSE, BAWE, ICNALE and PELIC, 64,391 essays in Table 6), and each human essay gets one "synthetic mirror" written to the same title and length by a GPT-3.5, GPT-4 family, Claude 2 or 3, LLaMA 2 to 3.1, Mistral or Gemini model; the mirrors are then passed through the humanizers. Results are true positive rate at a fixed 5% false positive rate with 1,000 bootstrap iterations (Table 3), the same metric on the RAID paraphrase and synonym splits (Table 4), and rates at each tool's default threshold (Table 7); the number of essays scored for Tables 3 and 7 is not printed (arXiv PDF, Section 5 and Appendix B, read 9 November 2026).

Every figure below is printed in Tables 2 to 5 and 7 of Masrour, Emi and Spero 2025 (arXiv PDF, read 9 November 2026).

Values as printed in Masrour, Emi and Spero 2025, arXiv 2501.03437. Table 8 prints the final model's default-threshold false positive rate as 3.47% where Table 7 prints 3.40%; this record carries Table 7.

The gap between the two commercial-grade tools is the paper's central result: at the same 5% false positive rate GPTZero kept 60.04% of humanized AI essays and DAMAGE kept 98.26%, and at default thresholds GPTZero caught 34.53% of humanized essays against DAMAGE's 97.47% (arXiv PDF, Tables 3 and 7, read 9 November 2026). RADAR's 3.33% in Table 3 follows from its 51.87% default false positive rate: at a 5% false positive operating point it detects almost nothing.

On 25 samples per humanizer, GPT-4o judged the humanized text more fluent than the original 26.0% of the time for L1 tools, 14.67% for L2 and 2.67% for L3 (arXiv PDF, Section 4.6, read 9 November 2026). Humanized text made up about 0.68% of the DAMAGE training set, oversampled 18 times, and training on all three tiers instead of L1 alone raised the default-threshold false positive rate from 3.47% to 6.00% (Table 8).

The paraphrased text question, added 9 November 2026, counts this record among the eleven that scored a card on paraphrased or humanized text, and the false positive studies question counts its 1.47% default-threshold figure for GPTZero with the caveat that the human essay count is not printed.

The authors call the detector-specific attack "a proof of concept experiment" and the humanizer audit "not exhaustive"; humanizer data volumes were capped at "the limit of the basic 1 month subscription on each humanizer website" (arXiv PDF, Sections 4.1, 5.4 and 7.1, read 9 November 2026). Evident from the paper: the authors sell a detector and built the best-scoring model; the essay count behind Tables 3 and 7 and GPTZero's version are not printed; the humanizer list differs between Table 1 and Table 9; and the false positive figures come from student essays only.

Four cards take a line from this record: RADAR, added on 10 November 2026 with the eleventh wave, records the Table 3 pair (3.33% true positive rate on AI essays at 5% FPR, 5.05% after 19 humanizers), the Table 4 RAID splits (62.4% paraphrase, 62.7% synonym) and the Table 7 default-threshold figures (70.67%, 79.33% and a 51.87% false positive rate on human essays, essay count not printed, quoted in its base and not ranked); Binoculars, added on 10 November 2026 with the tenth wave, records the Table 3 pair (94.15% true positive rate on AI essays at 5% FPR, 28.23% after 19 humanizers), the Table 4 RAID paraphrase figure (80.3%, the same value the RAID 2024 paper prints and the one its robustness rank field carries) and the Table 7 default-threshold figures (94.40%, 29.73% and a 5.40% false positive rate on human essays, essay count not printed, quoted in its base and not ranked, the GPTZero treatment); GPTZero records the Table 3 and Table 7 pairs and the 1.47% default-threshold false positive rate, and Pangram records the DAMAGE and LLM Baseline figures as the vendor's own research models (arXiv 2501.03437, Tables 3 and 7, read 9 November 2026). Both cards appear in the ranking of detectors by test record.

The Table 3 and Table 7 drops are paraphrase results for the adversarial robustness ranking, where GPTZero's 60.04% at 5% FPR sits beside the 64.0% on the RAID paraphrase split that RAID 2024 itself reports. The 1.47% GPTZero figure is a printed rate on human essays at the default threshold and qualifies for the false positive rate ranking; DAMAGE's 3.40% belongs to a research model and stays a card note. No figure enters the false negative rate ranking, which never takes a complement of a true positive rate. The home page lists which AI detectors exist and who has tested them.

Related records: Krishna 2023 introduced the DIPPER paraphraser placed here in tier L1; Perkins 2024 and Jabarian and Imas 2025 measure humanizer rewrites on other detectors.