Krishna et al. 2023: paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense

This record joins the studies index as the paper that built DIPPER, an 11-billion-parameter paraphraser that rewrites whole paragraphs with control over lexical diversity and content reordering, and then used it to attack five detectors across three language models before proposing retrieval over an API provider's past outputs as a defence (arXiv 2303.13408v2, Sections 3 to 5, read 9 November 2026).

Krishna 2023 attacked five detectors with their default settings, discarding any instance shorter than a detector's minimum length. One has a card in this index; one is covered by a separate record (arXiv 2303.13408v2, Section 4.2, read 9 November 2026).

Detectors as named in Krishna et al. 2023, arXiv 2303.13408.

Two generation tasks supplied the text. For open-ended generation, 3,000 two-sentence chunks from the WikiText-103 validation split served as prompts, with the next 300 tokens of the article as the human continuation. For long-form question answering, 500 questions from each of six domains of r/explainlikeimfive, posted between July and December 2021, were paired with their longest human answer, for 3,000 pairs. Three models generated 300-token responses: GPT-2 XL with 1.5 billion parameters, OPT-13B and GPT-3.5 text-davinci-003 (arXiv 2303.13408v2, Section 4.2 and Appendix F.2, read 9 November 2026).

Sample as described in Krishna et al. 2023, arXiv 2303.13408.

Each AI response was paraphrased once by DIPPER at four settings, L20, L40, L60 and L60 with O60, where L is lexical diversity and O is content reordering. The detection threshold of every detector was set on the human text so that 1% of human passages were flagged, and detection accuracy is the share of AI passages flagged at that threshold. Semantic preservation was measured with the P-SP similarity model against a human paraphrase score of 0.76 (arXiv 2303.13408v2, Sections 4.1 and 4.2, read 9 November 2026).

Table 1 of Krishna 2023 gives detection accuracy at 1% FPR on the Wikipedia task before paraphrasing and after the strongest DIPPER setting (L60, O60); Figure 3 gives the same for long-form question answering on three detectors (arXiv 2303.13408v2, read 9 November 2026).

Values as printed in Krishna et al. 2023, arXiv 2303.13408, Table 1, Figure 3 and Table 2.

The paper's own summary of Table 1 is that paraphrasing lowered detection accuracy across every model, detector and setting while preserving meaning: 88% to 99% of DIPPER paraphrases scored above the median human paraphrase on P-SP, and Krishna 2023 names watermarking the most resilient detector (arXiv 2303.13408v2, Section 4.3, read 9 November 2026). The one exception is RankGen, which flagged paraphrases more often than originals, rising from 13.5% to 28.5% on GPT-2 XL. DetectGPT on GPT-3.5 scored 0% at 1% FPR, so its cells are given at 20% FPR on 200 samples, 26.5% to 4.5%.

On the commercial detector, Krishna 2023 states that GPTZero and RankGen "perform the worst among the five detectors on all tested LMs", detecting under 15% of unparaphrased AI text, and adds "we recommend against using these detectors" (arXiv 2303.13408v2, Section 4.3, read 9 November 2026). The paper also notes that every non-watermark detector scored below 50% on text from OPT-13B and GPT-3.5 before any paraphrasing.

The retrieval defence in Krishna 2023 searches a database of an API's past generations for a match to the candidate text. Against a corpus of 15 million T5-XXL generations, BM25 retrieval detected 97.3% of paraphrased Project Gutenberg passages and 80.4% of paraphrased Wikipedia passages at 1% FPR, down from 98.3% and 90.0% at one million; detection plateaued once a query reached 50 tokens (arXiv 2303.13408v2, Section 5.3 and Figure 6, read 9 November 2026).

The paraphrased text question, added 9 November 2026, counts this record among the eleven that scored a card on paraphrased AI text.

The attack used a single paraphrase per passage; the authors state that sampling several paraphrases and keeping the one that evades the detector would make the attack stronger, and they did not run it because it assumes access to the detector (arXiv 2303.13408v2, Section 4.4, read 9 November 2026). Footnote 8 records that OpenAI took its classifier down in July 2023 for low accuracy, after the experiments.

The retrieval defence carries limits the authors list themselves: a privacy risk in exposing every LLM response behind a classifier, no way to apply it to open-source models, and the cost of running the infrastructure, which they estimate at 5 terabytes of storage per month and 130 seconds per retrieval on a laptop at ChatGPT scale (arXiv 2303.13408v2, Section 5.4, read 9 November 2026). Appendix F.3 acknowledges the argument that a paraphrased passage is no longer the target model's output, so model-specific detectors are not wrong to miss it.

Two of the five detectors in Krishna 2023 have a card in this index; the DetectGPT card was added on 9 November 2026 with the fourth wave and records the fall from 70.3% to 4.6% on 3,000 Wikipedia prompts and from 74.9% to 7.6% on 3,000 long-form QA pairs at 1% FPR, the 7.6% being the figure the robustness ranking takes. The GPTZero card records the three Table 1 figures on a base of 3,000 prompts at 1% FPR: 13.9% to 1.2% on GPT-2 XL, 8.7% to 1.0% on OPT-13B and 7.1% to 1.8% on GPT-3.5 (arXiv 2303.13408v2, read 9 November 2026).

GPTZero and DetectGPT appear in the ranking of detectors by independent test record and in the adversarial robustness ranking, where this record supplies both values. The OpenAI classifier's figures are carried on its own record, since the tool was withdrawn and has no card. How this record is sourced explains why thresholded research figures sit apart from vendor verdict labels, and the home page lists which AI detectors exist and who has tested them.

Related records: Sadasivan 2023 took the DIPPER paraphraser built here and ran it recursively, reporting that five rounds broke the retrieval defence; the RAID benchmark later measured DIPPER paraphrase attacks on commercial detectors at a 5% false positive rate (arXiv 2303.13408, read 9 November 2026).