{"id":1435,"date":"2024-09-10T20:13:26","date_gmt":"2024-09-10T20:13:26","guid":{"rendered":"https:\/\/thehive.ai\/blog\/?p=1435"},"modified":"2025-07-29T20:52:00","modified_gmt":"2025-07-29T20:52:00","slug":"clear-winner-study-shows-hives-ai-generated-image-detection-api-is-best-in-class","status":"publish","type":"post","link":"https:\/\/thehive.ai\/blog\/clear-winner-study-shows-hives-ai-generated-image-detection-api-is-best-in-class","title":{"rendered":"&#8220;Clear Winner&#8221;: Study Shows Hive&#8217;s AI-Generated Image Detection API is Best-in-Class"},"content":{"rendered":"\n<h5 class=\"at-a-glance-heading\">Contents<\/h5>\n\n\n\n<ul class=\"at-a-glance\"><li><a href=\"#anchor1\">Navigating an Increasingly Generative World<\/a><\/li><li><a href=\"#anchor2\">Structuring the Study<\/a><\/li><li><a href=\"#anchor3\">Evaluation Methods and Findings<\/a><\/li><li><a href=\"#anchor4\">Final Thoughts Moving Forward<\/a><\/li><\/ul>\n\n\n\n<h2 id=\"anchor1\">Navigating an Increasingly Generative World<\/h2>\n\n\n\n<p>To the untrained eye, distinguishing human-created art from AI-generated content can be difficult. Hive\u2019s commitment to providing customers with API-accessible solutions for challenging problems led to the creation of our <strong>AI-Generated Image and Video Detection API,<\/strong> which classifies images as human-created or AI-generated. Our model was evaluated in an independent <a href=\"https:\/\/arxiv.org\/pdf\/2402.03214\">study<\/a> conducted by Anna Yoo Jeong Ha and Josephine Passananti from the University of Chicago, which sought to determine who was more effective at classifying images as AI-generated: humans or automated detectors.<\/p>\n\n\n\n<p>Ha and Passananti\u2019s study addresses a growing problem within the generative AI space: As generative AI models become more advanced, the boundary between human-created art and AI-generated images has become increasingly indistinguishable. With such powerful tools being accessible to the general public, various legal and ethical concerns have been raised regarding the misuse of said technology.<\/p>\n\n\n\n<p>Such concerns are pertinent to address because the misuse of generative AI models negatively impacts both society at large and the AI models themselves. Bad actors have used AI-generated images for harmful purposes, such as spreading misinformation, committing fraud, or scamming individuals and organizations. As only human-created art is eligible for copyright, businesses may attempt to bypass the law by passing off AI-generated images as human-created. Moreover, multiple studies (on both generative <a href=\"https:\/\/arxiv.org\/abs\/2307.01850\">image<\/a> and <a href=\"https:\/\/arxiv.org\/abs\/2305.17493\">text<\/a> models) have shown evidence that AI models will deteriorate if their training data solely consists of AI-generated content\u2014which is where Hive\u2019s classifier comes in handy.<\/p>\n\n\n\n<p>The study\u2019s results show that Hive\u2019s model outperforms both its automated peers and highly-trained human experts in differentiating between human-created art versus AI-generated images across most scenarios. This post examines the study\u2019s methodologies and findings, in addition to highlighting our model\u2019s consistent performance across various inputs.<\/p>\n\n\n\n<h2 id=\"anchor2\">Structuring the Study<\/h2>\n\n\n\n<p>In the experiment, researchers evaluated the performance of five automated detectors (three of which are commercially available, including Hive\u2019s model) and humans against a dataset containing both human-created and AI-generated images across various art styles. Humans were categorized into three subgroups: non-artists, professional artists, and expert artists. Expert artists are the only subgroup with prior experience in identifying AI-generated images.<\/p>\n\n\n\n<p>The dataset consists of four different image groups: human-created art, AI-generated images, \u201chybrid images\u201d which combine generative AI and human effort, and perturbed versions of human-created art. A perturbation is defined as a minor change to the model input aimed at detecting vulnerabilities in the model\u2019s structure. Four perturbation methods are used in the study: JPEG compression, Gaussian noise, CLIP-based Adversarial Perturbation (which performs perturbations at the pixel level), and Glaze (a tool used to protect human artists from mimicry by introducing imperceptible perturbations on the artwork).<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" width=\"1024\" height=\"439\" src=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1024x439.png\" alt=\"\" class=\"wp-image-1450\" srcset=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1024x439.png 1024w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-300x129.png 300w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-768x330.png 768w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1536x659.png 1536w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" width=\"1024\" height=\"439\" src=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1-1024x439.png\" alt=\"\" class=\"wp-image-1451\" srcset=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1-1024x439.png 1024w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1-300x129.png 300w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1-768x330.png 768w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1-1536x659.png 1536w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/In-Blog-Graphic-1.png 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>After evaluating the model on unperturbed imagery, the researchers proceeded to more advanced scenarios with perturbed imagery.<\/p>\n\n\n\n<h2 id=\"anchor3\">Evaluation Methods and Findings<\/h2>\n\n\n\n<p>The researchers evaluated the automated detectors on four metrics: overall accuracy (ratio of training data classified correctly to the entire dataset), false positive rate (ratio of human-created art misclassified as AI-generated), false negative rate (ratio of AI-generated images misclassified as human-created), and AI detection success rate (ratio of AI-generated images correctly classified as AI-generated to the total amount of AI-generated images).<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" width=\"1024\" height=\"438\" src=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Table-min-1024x438.jpg\" alt=\"\" class=\"wp-image-1438\" srcset=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Table-min-1024x438.jpg 1024w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Table-min-300x128.jpg 300w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Table-min-768x329.jpg 768w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Table-min-1536x658.jpg 1536w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Table-min.jpg 1920w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Among automated detectors, Hive\u2019s model emerged as the \u201cclear winner\u201d (Ha and Passananti 2024, 6). Not only does it boast a near-perfect 98.03% accuracy rate, but it also has a 0% false positive rate (i.e., it never misclassifies human art) and a low 3.17% false negative rate (i.e., it rarely misclassifies AI-generated images). According to the authors, this could be attributed to Hive\u2019s rich collection of generative AI datasets, with high quantities of diverse training data compared to its competitors.<\/p>\n\n\n\n<p>Additionally, Hive\u2019s model proved to be resistant against most perturbation methods, but faced some challenges classifying AI-generated images processed with Glaze. However, it should be noted that Glaze\u2019s primary purpose is as a protection tool for human artwork. Glazing AI-generated images is a non-traditional use case with minimal training data available as a result. Thus, Hive\u2019s model\u2019s performance with Glazed AI-generated images has little bearing on its overall quality.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" width=\"949\" height=\"1024\" src=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min-949x1024.jpg\" alt=\"\" class=\"wp-image-1439\" srcset=\"https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min-949x1024.jpg 949w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min-278x300.jpg 278w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min-768x829.jpg 768w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min-1423x1536.jpg 1423w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min-1898x2048.jpg 1898w, https:\/\/staticblog.thehive.ai\/uploads\/2024\/09\/Figure_Vertical-min.jpg 1920w\" sizes=\"(max-width: 949px) 100vw, 949px\" \/><\/figure>\n\n\n\n<h2 id=\"anchor4\">Final Thoughts Moving Forward<\/h2>\n\n\n\n<p>When it comes to automated detectors and humans alike, Hive\u2019s model is unparalleled. Even compared to human expert artists, Hive\u2019s model classifies images with higher levels of confidence and accuracy.<\/p>\n\n\n\n<p>While the study considers the model\u2019s potential areas for improvement, it is important to note that the study was published in February 2024. In the months following the study\u2019s publication, Hive\u2019s model has vastly improved and continues to expand its capabilities, with 12+ model architectures added since.<\/p>\n\n\n\n<p>If you\u2019d like to learn more about Hive\u2019s AI-Generated Image and Video Detection API, a demo of the service can be accessed <a href=\"https:\/\/hivemoderation.com\/ai-generated-content-detection\">here<\/a>, with additional documentation provided <a href=\"https:\/\/docs.thehive.ai\/docs\/ai-image-and-video-detection\">here<\/a>. However, don\u2019t just trust us, test us: reach out to <a href=\"mailto:sales@thehive.ai\">sales@thehive.ai<\/a> or contact us <a href=\"https:\/\/thehive.ai\/contact-us?source=blog\">here<\/a>, and our team can share API keys and credentials for your new endpoints.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>An independent study shows that Hive&#8217;s AI-Generated Image and Video Detection API outperforms its competitors and human experts in classifying art as human-created or AI-generated.<\/p>\n","protected":false},"author":7,"featured_media":1923,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"kia_subtitle":""},"categories":[16,8,4],"tags":[],"_links":{"self":[{"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/posts\/1435"}],"collection":[{"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/comments?post=1435"}],"version-history":[{"count":5,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/posts\/1435\/revisions"}],"predecessor-version":[{"id":1454,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/posts\/1435\/revisions\/1454"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/media\/1923"}],"wp:attachment":[{"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/media?parent=1435"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/categories?post=1435"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thehive.ai\/blog\/wp-json\/wp\/v2\/tags?post=1435"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}