NIST AI 700-1, 2025: the GenAI text-to-text pilot, 348 detector submissions on AI and human summaries

This record joins the studies index as the only government-run evaluation indexed here, and the only one in which the detectors are anonymous: NIST identifies each team by a five-character label and names no commercial product among the participants (NIST AI 700-1 PDF, Section 7.2 and Table 2, read 9 November 2026).

The pilot launched in April 2024 with two tracks: generator teams produced summaries of at most 250 words from about 25 newswire documents per topic, and discriminator teams scored each summary from 0 to 1 for the likelihood that a language model wrote it. The report names no participant and lists only team labels, plus two NIST baseline detectors (NIST AI 700-1 PDF, Sections 4.2.5, 5 and 7.2, read 9 November 2026).

Detectors as named in NIST AI 700-1, DOI 10.6028/NIST.AI.700-1, Table 2 and Section 4.2.5.

Of 172 registered entities, 6 generator teams and 11 discriminator teams from 14 organisations took part. Generators sent 48 valid submissions and discriminators 348, split 50, 137 and 161 across three rounds, the last including one NIST baseline run. Of the 11 discriminator teams, 8 stayed for round two and 6 for round three (NIST AI 700-1 PDF, Section 7.2 and Table 2, read 9 November 2026).

Sample as described in NIST AI 700-1, DOI 10.6028/NIST.AI.700-1, Section 7.1.

The primary metric is AUC. Because AUC ignores calibration, the report adds two Brier terms: BrierT, the mean squared distance of scores on AI summaries from 1, and BrierN, the same distance of scores on human summaries from 0, both ideally near zero. The report treats a BrierN above 0.25 as poorly calibrated (NIST AI 700-1 PDF, Sections 6.1.5 and 8.1, read 9 November 2026).

Tables 4 to 8 of NIST AI 700-1 give results for selected generator and detector pairs; the report prints no league table of teams (NIST AI 700-1 PDF, read 9 November 2026).

Values as printed in NIST AI 700-1, DOI 10.6028/NIST.AI.700-1, Tables 3 to 8 and Figure 18.

In the first round some submissions returned a BrierT of 1, the maximum: submission 15 from team 6655b scored every GPT-3.5 summary at or near zero, treating all of them as human, while other submissions returned a BrierN near 1, treating human summaries as AI with high confidence (NIST AI 700-1 PDF, Section 8.1 and Figure 9, read 9 November 2026). The authors write that reading a poorly calibrated score as a probability "would be highly misleading and cause many false positives" at a 0.5 cutoff, even when AUC is close to 1.

The report's central finding is asymmetry: "there are some generators that could deceive most discriminators, and there are discriminators that could detect AI-generated content from almost all generators". In round three, 27 generator and detector pairs met the report's definition of a generator win, with three generator submissions from teams 804fe and 87a8c (IDs 53, 95 and 110) recurring, while detector submission 186 from team 29d48 and submission 409 from team b3cd9 recur among the detector wins (NIST AI 700-1 PDF, Abstract, Tables 7 and 8, read 9 November 2026).

On change over time, the cumulative distributions of AUC in Figure 18 show more values near 1 in each later round, which the authors read as detectors improving, while cautioning that the three rounds used different test sets (NIST AI 700-1 PDF, Section 8.4, read 9 November 2026).

The report lists its scope as summarisation only, which may not carry over to conversation, creative or technical writing; a strict binary of AI or human authorship that ignores co-authored text; heavy human and computational cost that limits scale; and the possibility that some detectors are biased toward linguistic patterns of the human summaries. It also notes that generators are improving faster than detectors and that the study does not address societal or policy questions (NIST AI 700-1 PDF, Sections 9.2 and 9.3, read 9 November 2026).

NIST's disclaimer states that identifying any software or material in the report does not imply recommendation or endorsement, and the report defers per-submission and per-topic analysis to a later publication (NIST AI 700-1 PDF, front matter and Section 8.4, read 9 November 2026).

What this record does not say: no figure in NIST AI 700-1 is attributable to a named commercial detector, so nothing here feeds a card's false positive or accuracy field, and the RoBERTa baseline's own scores are not printed under its name.

Two detectors named in NIST AI 700-1, the RoBERTa-base-openai-detector and RADAR-Vicuna-7B baselines, have a card in this index, the second since 10 November 2026; the report prints no result under either name, so the GPT-2 Output Detector card records only that the detector served as a NIST baseline and the RADAR card carries the naming as a note outside its study count (NIST AI 700-1 PDF, Section 4.2.5, read 9 November 2026).

No team in the pilot is linked to a product on the ranking of detectors by independent test record, and no figure here enters the false positive rate ranking. How this record is sourced explains why an anonymised evaluation is indexed but does not score a card, and the home page lists which AI detectors exist and who has tested them.

Related records: the RAID benchmark is the shared academic benchmark that does name commercial detectors; Jabarian and Imas 2025 ran the same RoBERTa GPT-2 detector as an open-source baseline and reports its false positive rate at 0.9503 on 1,992 passages (DOI 10.6028/NIST.AI.700-1, read 9 November 2026).