Ibrahim 2023: GPT-2 Output Detector and Crossplag on 120 ESL student essays and 120 ChatGPT essays

This record joins the studies index as a detector test on essays by second-language writers, received on 1 August 2023, four months after Liang 2023 reported seven detectors flagging a majority of TOEFL essays by non-native writers (Springer PDF, Declarations, read 9 November 2026). The author concludes that both detectors reached "an average accuracy of 89%" and that educators "should not base their decisions about academic integrity solely on detectors' analysis" (Conclusion and Practical implications).

Ibrahim 2023 ran two free, publicly available detectors, both RoBERTa classifiers fine-tuned on GPT-2 output. One has a card in this index; Crossplag does not (DOI 10.1186/s40468-023-00260-2, Introduction and Literature review, read 9 November 2026).

Detectors as named in Ibrahim 2023, DOI 10.1186/s40468-023-00260-2, Methodology. Both were accessed with their public web interfaces; no version or access date is printed.

The human set is a convenience sample of 120 research-argument essays from an advanced first-year composition course at an American university in Kuwait, written in Fall 2021, Spring 2022 and Summer 2022, before ChatGPT was released. The machine set is 120 essays that the January 2023 version of ChatGPT (GPT-3.5) wrote from the same project description, with topics matched to the student essays and titles and reference lists removed (Springer PDF, Data collection, read 9 November 2026).

Sample as described in Ibrahim 2023, DOI 10.1186/s40468-023-00260-2, Data collection.

The dependent variable is an "originality score", the percentage probability that a text is human-written. GPT-2 Output Detector reports that figure directly; Crossplag reports an AI probability, so the author subtracted it from 100. A human essay counted as correctly placed if it scored 95% or above, and a ChatGPT essay if it scored 5% or below; scores were compared with Mann-Whitney U tests in SPSS 28 because the distributions were skewed (Springer PDF, Data analysis, read 9 November 2026).

Every figure below is printed in the Results or Discussion of Ibrahim 2023; the base is 120 essays per set (Springer PDF, Results and Discussion, read 9 November 2026).

Values as printed in Ibrahim 2023, DOI 10.1186/s40468-023-00260-2, Results sections "GPT Output Detector Demo" and "Crossplag Detector" and Discussion RQ1 to RQ3.

One point of wording matters. Ibrahim calls a human essay scoring below 95% originality a "false negative" and a ChatGPT essay scoring above 5% a "false positive", because the construct measured is originality, not AI authorship. In the terms used across this index, the 13.3% and 15% figures are human essays treated as less than fully human, and the 8.3% and 7.5% figures are ChatGPT essays not treated as fully machine-written (Springer PDF, Data analysis and Discussion RQ1, read 9 November 2026). The record keeps the author's counts and states the direction in plain words.

The two detectors differed in which text type they handled better. Scores for the ChatGPT essays were lower on Crossplag than on GPT-2 Output Detector (mean ranks 72.66 against 168.34, U = 1459, P = 0.001), and scores for the human essays were higher on GPT-2 Output Detector than on Crossplag (mean ranks 155.68 against 85.33, U = 2979, P = 0.001) (Springer PDF, Performance Comparison, read 9 November 2026). Both detectors misjudged the same essays in each set, which the author attributes to their shared GPT-2 training data and RoBERTa architecture (Discussion RQ3).

The paper's summary sentences do not all agree. The RQ3 discussion and the Conclusion say GPT-2 Output Detector was more sensitive to human text and Crossplag to machine text, while the RQ4 summary states the reverse, and the RQ3 text prints a mean rank of 158.34 where the Results print 168.34 (Springer PDF, Discussion RQ3 and RQ4, Conclusion, read 9 November 2026). This record follows the Results figures.

The human sample was a convenience sample from one course at one university and "cannot be generalized to the population of ESL writing students"; the sample "was small"; only two detectors with the same training protocol and architecture were compared; and the ChatGPT essays were generated in January 2023, so later outputs may differ (Springer PDF, Limitations and future research directions, read 9 November 2026). The 95% and 5% cut points are the author's choice, not the detectors' verdict labels; no confidence intervals, AUROC or per-essay scores are printed. A correction published on 31 October 2023 changed only the Acknowledgements, adding a Gulf University for Science and Technology grant that covered the article processing charge (DOI 10.1186/s40468-023-00268-8, read 9 November 2026).

One detector in Ibrahim 2023 has a card in this index. The GPT-2 Output Detector card records the 13.3% of human essays below 95% originality, the 8.3% of ChatGPT essays above 5% and the 89.2% average accuracy the author computed (DOI 10.1186/s40468-023-00260-2, Discussion RQ1, read 9 November 2026). Crossplag has no card, so its 15%, 7.5% and 88.75% stay in this record.

The card appears in the ranking of detectors by test record. The false positive rate ranking sorts rates a study prints on human text, and the 13.3% figure is printed on human essays under the author's own label and cut point; the false negative rate ranking takes printed rates on AI text and never a complement of a sensitivity figure, and the 8.3% figure is a printed count of ChatGPT essays above the cut. Whether each enters its ranking under the author's inverted labels is noted on the card. The home page lists which AI detectors exist and who has tested them.

Related records: Liang 2023 found seven detectors flagged an average 61.22% of 91 TOEFL essays by non-native writers, against the 13.3% and 15% here; Gao 2023 ran the same GPT-2 Output Detector on 50 human and 50 ChatGPT abstracts, the one earlier test Ibrahim cites; Pegoraro 2023 scored it at a 7.2% true positive rate on ChatGPT answers (DOI 10.1186/s40468-023-00260-2, Literature review, read 9 November 2026).