Binoculars zero-shot AI text detector
Vendor Hans, Schwarzschild, Cherepanova, Kazemi, Saha, Goldblum, Geiping and Goldstein (University of Maryland, Carnegie Mellon University, New York University, ELLIS Institute Tübingen and MPI for Intelligent Systems) Official site github.com/ahans30/Binoculars How this card is sourced
Updated · AI Detector Index editors
Binoculars flagged 0.00% of RAID's human-written documents as machine-generated at a naive threshold of 0.5, and 0.07% at 0.25, in Dugan et al. 2024 (arXiv 2405.07940, Table 4, read 10 November 2026). Binoculars is a research method with published code, not a company's product: the ICML 2024 paper by Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping and Tom Goldstein scores a passage by contrasting two closely related language models, and the implementation is on GitHub under the BSD 3-Clause License (arXiv 2401.12070 and github.com/ahans30/Binoculars, read 10 November 2026). One hosted demo exists: the Hugging Face Space tomg-group-umd/Binoculars, a Gradio app the Maryland group runs on ZeroGPU hardware, which was in the RUNNING state on 10 November 2026 (huggingface.co, read 10 November 2026). The card joins the detector index beside GLTR, DetectGPT and Fast-DetectGPT as an open research tool, and how this card is sourced explains why the paper and the repository stand in for a product page.
What Binoculars detects
| Field | Value | Source |
|---|---|---|
| Content types detected | Text | arxiv.org/abs/2401.12070 |
| Generators covered | ChatGPT (the Ghostbuster News, Creative Writing and Student Essay sets), LLaMA-2-7B and LLaMA-2-13B (paper experiments), Falcon-7B generations (paper experiments), open models in the M4 dataset experiments (paper, Section 5.1) | arxiv.org/pdf/2401.12070 |
| Input limits | Research code: Binoculars() scores text with two Falcon-7B models and truncates each input at max_token_observed = 512 tokens by default; a list of strings is scored as a batch. The hosted Hugging Face Space refuses text under 64 tokens (Too short length. Need minimum 64 tokens to run Binoculars) and its disclaimer states: We recommend using 200-300 words of text at a time. Fewer words make detection difficult, as can using more than 1000 words. Paste only; no upload format is published. Two modes: Low False Positive Rate (threshold 0.8536, chosen at 0.01% FPR on the validation set) and High Accuracy (threshold 0.9015, F1-optimised). | github.com/ahans30/Binoculars |
| Free tier | Free and open source (BSD 3-Clause License): the code is on GitHub and the Hugging Face Space demo runs without a published quota. No plan, price or credit is sold | github.com/ahans30/Binoculars |
| Integrations | GitHub repository (pip-installable Python package with compute_score and predict methods), Gradio demo script (python app.py) and the hosted Hugging Face Space tomg-group-umd/Binoculars | github.com/ahans30/Binoculars |
| Data retention | The paper and repository publish no privacy policy and the code keeps text on the machine that runs it. The Hugging Face Space runs on Hugging Face's platform under its terms; the demo's own disclaimer states only that the provided prediction is for demonstration purposes only and that the authors assume no liability for any use. No retention period is stated anywhere. | huggingface.co/spaces/tomg-group-umd/Bino... |
| Launch year | 2024 | github.com/ahans30/Binoculars |
| Ownership | A research method published at ICML 2024 (Proceedings of the 41st International Conference on Machine Learning, PMLR 235) by Abhimanyu Hans, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha and Tom Goldstein (University of Maryland), Avi Schwarzschild (Carnegie Mellon University), Micah Goldblum (New York University) and Jonas Geiping (ELLIS Institute Tübingen, MPI Intelligent Systems); arXiv 2401.12070, first posted 22 January 2024 and revised to v3 on 13 October 2024. The code is on GitHub under the BSD 3-Clause License (LICENSE.md); the repository was created on 10 December 2023, last pushed on 14 May 2024 (merge of a config import fix) and held 434 stars and 69 forks on 10 November 2026. A hosted Gradio demo runs as the Hugging Face Space tomg-group-umd/Binoculars (Tom Goldstein's group at Maryland), last modified 18 August 2024 and in the RUNNING state on ZeroGPU hardware on 10 November 2026. No company sells the method. | arxiv.org/abs/2401.12070 |
| Independent studies | 2 records | Studies index |
Binoculars reads text through two lenses. The README states that the method "is based on a simple idea: most decoder-only, causal language models have a huge overlap in pretraining datasets", and the RAID authors describe the score as "perplexity divided by cross-entropy between two very similar language models" (README and arXiv 2405.07940, Appendix, read 10 November 2026). The default pair is Falcon-7B as the observer and Falcon-7B-Instruct as the performer, and the code ships two fixed thresholds chosen on that pair: 0.8536 for the low false positive mode, "chosen at 0.01%", and 0.9015 for the accuracy mode, "optimized for f1-score" (detector.py in the repository, read 10 November 2026). The paper's abstract carries the headline claim: "Over a wide range of document types, Binoculars detects over 90% of generated samples from ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being trained on any ChatGPT data" (arXiv 2401.12070, read 10 November 2026).
The paper's generators are ChatGPT on the Ghostbuster News, Creative Writing and Student Essay sets, LLaMA-2-7B and LLaMA-2-13B, Falcon-7B and the open models of the M4 dataset, and its text sources include CCNews, PubMed, CNN, Open-Orca prompts and the M4 samples in Urdu, Russian, Bulgarian and Arabic (arXiv 2401.12070, Sections 3 and 5.1, read 10 November 2026). No supported-language list is published, so the languages field is empty; Section 5.2 reports that on languages "not well represented in Common Crawl data" false positive rates "remain low" while "machine text in these low-resource languages is often classified as human". Section 5.2 also reports an accuracy of 99.67% on both the original and the grammar-corrected essays of non-native English speakers in the EssayForum set, a result the authors set against the commercial detectors Liang 2023 examined.
The repository was created on 10 December 2023, last pushed on 14 May 2024 with a merge that fixed a config import, and held 434 stars and 69 forks on 10 November 2026 (GitHub API, read 10 November 2026). Two cards share its lineage in RAID 2024's grouping of open metric-based detectors: Fast-DetectGPT, the ICLR 2024 curvature method, and GLTR, the 2019 display tool; the question on how AI detectors work places all three beside the trained classifiers.
Limits and plans
The research code publishes no character limit, no plan and no price. Binoculars() truncates each input at 512 tokens by default (max_token_observed) and scores a Python list of strings as one batch, which is the only bulk surface the card records (detector.py and README, read 10 November 2026). The hosted Space refuses short text with the warning "Too short length. Need minimum 64 tokens to run Binoculars" and its disclaimer states: "We recommend using 200-300 words of text at a time. Fewer words make detection difficult, as can using more than 1000 words" (demo.py in the Space, read 10 November 2026). The 64-token refusal is a code guard rather than a vendor statement of a minimum, and the 200 to 300 words are advice, so the minimum input field is left empty. The demo takes pasted text only, offers the two modes as "Low False Positive Rate" and "High Accuracy", and publishes no quota.
No privacy policy exists for the method. The paper and repository publish none, the Space runs under Hugging Face's platform terms, and the demo's own text says only that "the provided prediction is for demonstration purposes only" and that the authors "assume no liability for any use" (Space disclaimer, read 10 November 2026). The retention field is therefore empty, the reading the index gave DetectGPT, and the card sits in the second table of the data retention ranking. The README's limitations section states the authors' own boundary: "This implementation is for academic purposes only and should not be considered as a consumer product. We also strongly caution against using Binoculars (or any detector) without human supervision" (README, read 10 November 2026).
What the vendor claims
No accuracy percentage is published. Paper abstract: Over a wide range of document types, Binoculars detects over 90% of generated samples from ChatGPT (and other LLMs) at a false positive rate of 0.01%, despite not being trained on any ChatGPT data. Figure 1 caption: Binoculars, based on open-source Falcon models with no finetuning, outperforms commercial detection systems, such as GPTZero, as well as open-source detectors. Section 5.2: Binoculars attains equal accuracy at 99.67% for both corrected and uncorrected essay datasets written by non-native English speakers (EssayForum).
What independent studies found
- AIDI-S-0006RAID 20242024
Run from the official repository with the default Falcon-7B and Falcon-7B-Instruct pair: false positive rate on RAID human-written documents of 0.07% at threshold 0.25 and 0.00% at 0.5, 0.75 and 0.95 (Table 4); accuracy at a fixed 5% FPR of 99.9 on open chat models with greedy decoding, 72.4 on open non-chat models under sampling and 0.6 under sampling with a repetition penalty, 99.2 on closed chat models (Table 5); 79.6 overall on non-adversarial text, the highest of the four open metric-based detectors (Table 15); 80.3 under paraphrase, 43.5 under synonym swap, 37.7 under homoglyphs and 99.1 under zero-width spaces against 79.6 with no attack (Table 16); 59.9 on Python code, 67.0 on Czech news and 76.7 on German news (Table 7); the paper states that Binoculars was particularly strong at low FPR (Section 5) and performed impressively well across models even at extremely low false positive rates (Conclusion), and that metric-based methods such as Binoculars degraded by as much as 36.1% under synonym swaps (Finding 3).
- AIDI-S-0025DAMAGE 20252025
Run as the open-source zero-shot baseline by Pangram Labs' authors: true positive rate at a fixed 5% false positive rate of 94.15% on AI-written student essays and 28.23% after the essays passed through 19 commercial humanizers (Table 3, 1,000 bootstrap iterations, essay count not printed); 80.3% on RAID's paraphrase split and 43.5% on its synonym split (Table 4); at the default threshold 94.40% on AI essays, 29.73% after humanizing and a 5.40% false positive rate on human essays (Table 7).
Two records in the studies index scored Binoculars. RAID 2024 ran the official repository with the default Falcon pair and reports false positive rates on human-written documents of 0.07% at threshold 0.25 and 0.00% at 0.5, 0.75 and 0.95 (arXiv 2405.07940, Table 4, read 10 November 2026). At a threshold calibrated to 5% false positives it scored 99.9 on open chat models with greedy decoding, 72.4 on open non-chat models under sampling and 0.6 once a repetition penalty was added (Table 5), 79.6 overall on non-adversarial text, the highest of the four open metric-based detectors (Table 15), and 59.9 on Python code, 67.0 on Czech news and 76.7 on German news (Table 7). The paper's Section 5 states that Binoculars "was particularly strong at low FPR", and its conclusion that it "performed impressively well across models even at extremely low false positive rates".
Under attack the same record prints 80.3 for paraphrase against 79.6 with none, 43.5 under synonym swap, 37.7 under homoglyphs and 99.1 under zero-width spaces (arXiv 2405.07940, Table 16, read 10 November 2026). Finding 3 of RAID 2024 names the weakness: "metric-based methods such as Binoculars degraded by as much as 36.1%" when a few words were swapped for synonyms. DAMAGE 2025, written by Pangram Labs, ran Binoculars as its open-source baseline on AI-written student essays and reports a true positive rate at a fixed 5% false positive rate of 94.15% before and 28.23% after 19 commercial humanizers (Table 3, 1,000 bootstrap iterations, essay count not printed), 80.3% on RAID's paraphrase split and 43.5% on its synonym split (Table 4), and at the default threshold 94.40% on AI essays, 29.73% after humanizing and a 5.40% false positive rate on human essays (arXiv 2501.03437, Tables 3, 4 and 7, read 10 November 2026).
The fields follow the index rules. The false positive value is 0.00% at threshold 0.5 from RAID 2024 Table 4, the GLTR and Fast-DetectGPT treatment; DAMAGE 2025's 5.40% carries no printed essay count and stays in the base text. Robustness is 80.3, the paraphrase column at 5% false positives, the highest printed post-attack figure, with the 43.5 synonym result and the 28.23% humanizer result quoted beside it. No record prints a miss count, so the false negative ranking carries no value: 79.6 and 94.15% are percent-correct figures whose complements the rule does not take.
Where Binoculars appears in the index
Rankings
- AIDI-R-01AI detectors ranked by independent testing Two records added in the fifth wave on 9 November 2026, Pegoraro 2023 and Fleckenstein 2024, changed nine counts and none: Pegoraro 2023 tested ZeroGPT, GPTZero, Copyleaks, Originality.ai, Writer, Content at Scale (BrandWell), DetectGPT, the GPT-2 Output Detector and GLTR, lifting GPTZero to 12, the GPT-2 Output Detector to 10, Copyleaks to 7 and Originality.ai and ZeroGPT to 6; Fleckenstein 2024 tested 289 teachers and no software, so it counts for no card ([Fleckenstein 2024 record](/studies/fleckenstein-2024-teachers/)). Illuminarty, one of four cards added in the same wave, joined with its one image record from Ha and Passananti 2024, so 18 of 72 cards now qualify.
- AIDI-R-02AI detector false positive rates, ranked
- AIDI-R-08AI detectors ranked by published robustness to paraphrased and edited text
- AIDI-R-20AI detectors ranked by launch year, with every change of hands and withdrawal the cards record
Fits these use cases, questions and comparisons
Binoculars qualifies for three study rankings, read 10 November 2026 in the data file: independently tested detectors with 2 records, false positive rates at 0.00% of RAID's human documents at threshold 0.5, and adversarial robustness at 80.3 under paraphrase. On stated values it qualifies for the open-source view with the BSD 3-Clause License and the last push of 14 May 2024; the README's batch scoring of a Python list is a code argument rather than a product batch feature, so the bulk flag is off and the card is absent from the bulk and batch view (ruling of 10 November 2026, PAGE-MAP 1q). It is absent from the free tier, language, price, API and claim gap rankings, which need a word allowance, a language count, a price, an endpoint and an accuracy percentage the method does not publish.
The RAID leaderboard carries a Binoculars entry dated 22 January 2024 and marked "Submitted by RAID Authors", with the Space as its website (leaderboard metadata, read 10 November 2026), the kind of entry the question on public leaderboard entries sets apart from a study. The humanizer results read beside the paraphrased text question, where GPTZero's DAMAGE 2025 figure fell from 99.73% to 60.04% on the same essays (arXiv 2501.03437, Table 3, read 10 November 2026). Every detector card on the register shows the same fourteen fields, so the empty price and language cells here can be read beside the commercial tools that fill them.
Questions about Binoculars
Is Binoculars a tool I can use?
Yes, two ways: the BSD-licensed code runs locally with Falcon-7B and Falcon-7B-Instruct after pip install, and the Hugging Face Space tomg-group-umd/Binoculars scores pasted text of 64 tokens or more without a price, read 10 November 2026. The README calls the implementation academic, not a consumer product.
How accurate is Binoculars?
The paper reports detection at a fixed false positive rate, not accuracy: over 90% of ChatGPT samples at 0.01% false positives, read 10 November 2026. RAID 2024 measured 79.6 accuracy at a fixed 5% false positive rate on non-adversarial text and 99.9 on open chat models with greedy decoding.
What false positive rate did RAID find for Binoculars?
0.00% of human-written documents at thresholds 0.5, 0.75 and 0.95 and 0.07% at 0.25, in Dugan et al. 2024 Table 4, read 10 November 2026; DAMAGE 2025 Table 7 prints 5.40% on human student essays at the default threshold, with no essay count.
Does Binoculars survive paraphrasing and humanizers?
Paraphrase yes, humanizers no, in the two indexed records: RAID 2024 reports 80.3 accuracy at 5% false positives under paraphrase against 79.6 with none, while DAMAGE 2025 reports the true positive rate on student essays falling from 94.15% to 28.23% after 19 commercial humanizers, read 10 November 2026.
Does Binoculars store the text you check?
The research code keeps text on the machine that runs it, and no privacy policy is published for the method or its Hugging Face Space, read 10 November 2026. The demo's disclaimer states only that its prediction is for demonstration purposes.
Sources and revisions
Sources
- arxiv.org/abs/2401.12070Read 10 November 2026
- arxiv.org/pdf/2401.12070Read 10 November 2026
- github.com/ahans30/BinocularsRead 10 November 2026
- huggingface.co/spaces/tomg-group-umd/Bino...Read 10 November 2026
- github.com/liamdugan/raid/tree/main/leade...Read 10 November 2026
- arxiv.org/pdf/2405.07940Read 10 November 2026
- arxiv.org/pdf/2501.03437Read 10 November 2026
Revision log
- Card created2026-11-10: bulk flag set to off at the merge (a readme list argument is not a product batch feature)
). Illuminarty, one of four cards added in the same wave, joined with its one image record from Ha and Passananti 2024, so 18 of 72 cards now qualify.: register plate](/img/independently-tested-480.webp)
