Ha and Passananti et al. 2024: organic or diffused? Hive, Optic and Illuminarty on 280 human artworks and 350 AI images

This record joins the studies index as its first image study, and the only one to test detectors on human art rather than photographs or deepfakes. Six of the seven authors are at the University of Chicago, whose team also built the Glaze protection tool the paper tests against (arXiv 2402.03214v3, Sections 1 and 4.3, read 9 November 2026).

Ha and Passananti 2024 ran three commercial image detectors and two research detectors, then compared them with three groups of human judges. One detector has a card in this index (arXiv 2402.03214v3, Section 3.2, read 9 November 2026).

Detectors as named in Ha and Passananti et al. 2024, arXiv 2402.03214.

The human set came from 53 artists recruited through the Cara portfolio platform: 280 artworks, 40 in each of seven styles (anime, cartoon, fantasy, oil or acrylic, photography, sketch, watercolor), with signatures cropped out. For each style the authors captioned 10 artworks with BLIP and prompted five generators, CIVITAI fine-tunes of SDXL, DALL-E 3, Adobe Firefly, Midjourney v6 and Stable Diffusion XL, for 10 images each, 350 AI images in all (arXiv 2402.03214v3, Section 4, read 9 November 2026).

Sample as described in Ha and Passananti et al. 2024, arXiv 2402.03214, Sections 4 and 7.2.

Each detector returns a 0 to 100% score. The authors mapped scores under 40% to a human decision and over 60% to an AI decision and dropped the 40% to 60% band as "not sure", which affected 0.54% of Hive's decisions and 4.29% of Optic's. They reran the AI images after JPEG compression, Gaussian noise, a CLIP-based adversarial perturbation and Glaze at medium and high intensity (arXiv 2402.03214v3, Sections 4.3 and 5.1, read 9 November 2026).

Table 1 of Ha and Passananti 2024 gives results on the unperturbed set, Table 2 by generator, Table 3 under perturbation and Table 4 with Glaze on both human and AI images (arXiv 2402.03214v3, read 9 November 2026).

Values as printed in Ha and Passananti et al. 2024, arXiv 2402.03214, Tables 1 to 5 and 11.

The authors call Hive "the clear winner among all five detectors" and note that it never misclassified a human artwork in the test, while Optic's lower false negative rate "comes at the expense of a high 24.47% FPR" and Illuminarty showed "even harsher treatment towards human art" (arXiv 2402.03214v3, Section 5.2, read 9 November 2026). An ensemble of the three commercial tools lifted accuracy to 98.75% at a 0.48% false positive rate.

Generator choice mattered more than art style. Firefly images were the hardest: Hive marked 6 of 70 as human and 3 as not sure, which the authors attribute to Firefly being the newest model with the least training data available (arXiv 2402.03214v3, Section 5.3, read 9 November 2026). Glaze, a feature-space perturbation built to stop style mimicry, cut Hive's detection of AI images to under 70%, while Glaze on human art left those decisions largely unchanged.

The human studies ran the same images past 177 crowdworkers, 3,803 professional artists and 13 expert artists. On unperturbed images the order was Hive, Optic, expert artist, professional artist, Illuminarty, general user. On Glazed images the experts' false negative rate of 8.97% beat Hive's 20.62%, and a Hive plus expert team with confidence as tiebreak reached 92.54% accuracy (arXiv 2402.03214v3, Section 7 and Tables 8, 9 and 11, read 9 November 2026). A follow-up after acceptance found that overlaying a white wall image flipped Hive's verdict on 17.5% of 105 AI images at or below 50% overlay intensity in April 2024.

The text and image question, added 9 November 2026, counts this record as the image record of Hive, AI or Not and Illuminarty, and the false positive studies question tables its 24.47% and 67.40% rates on 280 human artworks as image rows.

The authors list three limits: the dataset covers a fixed set of seven styles; cropping was applied to a small number of human artworks; and "not sure" responses were discarded from both machine and human results. They also note that DIRE and DE-FAKE were trained on bedrooms, faces and ImageNet or on MSCOCO captions, so their near-chance results reflect poor transfer to art (arXiv 2402.03214v3, Sections 5.2 and 10, read 9 November 2026). Illuminarty's creator told the authors in November 2023 that the model had not been updated for six months, and Hive's model coverage grew from nine generators to 27 during the study.

Two of the five detectors in Ha and Passananti 2024 had a card in this index on the morning of 9 November 2026, when the AI or Not card, the tool the paper names Optic, was added; the Illuminarty card followed later the same day, so three of the five now have one. The card records the Table 1 figures on 280 human artworks and 350 AI images, and the Glaze result from Table 4 (arXiv 2402.03214v3, read 9 November 2026).

Hive, AI or Not and Illuminarty appear in the ranking of detectors by independent test record, and this study is the image entry on the false positive rate ranking; the content type matrix shows which detectors in the index cover images at all. How this record is sourced explains why an image result is kept apart from text results on the same card, and the home page lists which AI detectors exist and who has tested them.

Related records: Perkins 2024 and the RAID benchmark measure adversarial edits against text detectors, the text counterpart to the Glaze and overlay tests here (arXiv 2402.03214, read 9 November 2026).