BACK TO ALL BLOGS

“Clear Winner”: Study Shows Hive’s AI-Generated Image Detection API is Best-in-Class

Contents

Navigating an Increasingly Generative World

To the untrained eye, distinguishing human-created art from AI-generated content can be difficult. Hive’s commitment to providing customers with API-accessible solutions for challenging problems led to the creation of our AI-Generated Image and Video Detection API, which classifies images as human-created or AI-generated. Our model was evaluated in an independent study conducted by Anna Yoo Jeong Ha and Josephine Passananti from the University of Chicago, which sought to determine who was more effective at classifying images as AI-generated: humans or automated detectors.

Ha and Passananti’s study addresses a growing problem within the generative AI space: As generative AI models become more advanced, the boundary between human-created art and AI-generated images has become increasingly indistinguishable. With such powerful tools being accessible to the general public, various legal and ethical concerns have been raised regarding the misuse of said technology.

Such concerns are pertinent to address because the misuse of generative AI models negatively impacts both society at large and the AI models themselves. Bad actors have used AI-generated images for harmful purposes, such as spreading misinformation, committing fraud, or scamming individuals and organizations. As only human-created art is eligible for copyright, businesses may attempt to bypass the law by passing off AI-generated images as human-created. Moreover, multiple studies (on both generative image and text models) have shown evidence that AI models will deteriorate if their training data solely consists of AI-generated content—which is where Hive’s classifier comes in handy.

The study’s results show that Hive’s model outperforms both its automated peers and highly-trained human experts in differentiating between human-created art versus AI-generated images across most scenarios. This post examines the study’s methodologies and findings, in addition to highlighting our model’s consistent performance across various inputs.

Structuring the Study

In the experiment, researchers evaluated the performance of five automated detectors (three of which are commercially available, including Hive’s model) and humans against a dataset containing both human-created and AI-generated images across various art styles. Humans were categorized into three subgroups: non-artists, professional artists, and expert artists. Expert artists are the only subgroup with prior experience in identifying AI-generated images.

The dataset consists of four different image groups: human-created art, AI-generated images, “hybrid images” which combine generative AI and human effort, and perturbed versions of human-created art. A perturbation is defined as a minor change to the model input aimed at detecting vulnerabilities in the model’s structure. Four perturbation methods are used in the study: JPEG compression, Gaussian noise, CLIP-based Adversarial Perturbation (which performs perturbations at the pixel level), and Glaze (a tool used to protect human artists from mimicry by introducing imperceptible perturbations on the artwork).

After evaluating the model on unperturbed imagery, the researchers proceeded to more advanced scenarios with perturbed imagery.

Evaluation Methods and Findings

The researchers evaluated the automated detectors on four metrics: overall accuracy (ratio of training data classified correctly to the entire dataset), false positive rate (ratio of human-created art misclassified as AI-generated), false negative rate (ratio of AI-generated images misclassified as human-created), and AI detection success rate (ratio of AI-generated images correctly classified as AI-generated to the total amount of AI-generated images).

Among automated detectors, Hive’s model emerged as the “clear winner” (Ha and Passananti 2024, 6). Not only does it boast a near-perfect 98.03% accuracy rate, but it also has a 0% false positive rate (i.e., it never misclassifies human art) and a low 3.17% false negative rate (i.e., it rarely misclassifies AI-generated images). According to the authors, this could be attributed to Hive’s rich collection of generative AI datasets, with high quantities of diverse training data compared to its competitors.

Additionally, Hive’s model proved to be resistant against most perturbation methods, but faced some challenges classifying AI-generated images processed with Glaze. However, it should be noted that Glaze’s primary purpose is as a protection tool for human artwork. Glazing AI-generated images is a non-traditional use case with minimal training data available as a result. Thus, Hive’s model’s performance with Glazed AI-generated images has little bearing on its overall quality.

Final Thoughts Moving Forward

When it comes to automated detectors and humans alike, Hive’s model is unparalleled. Even compared to human expert artists, Hive’s model classifies images with higher levels of confidence and accuracy.

While the study considers the model’s potential areas for improvement, it is important to note that the study was published in February 2024. In the months following the study’s publication, Hive’s model has vastly improved and continues to expand its capabilities, with 12+ model architectures added since.

If you’d like to learn more about Hive’s AI-Generated Image and Video Detection API, a demo of the service can be accessed here, with additional documentation provided here. However, don’t just trust us, test us: reach out to sales@thehive.ai or contact us here, and our team can share API keys and credentials for your new endpoints.

BACK TO ALL BLOGS

Introducing Hive’s Intellectual Property & Publicity Detection Product Suite

Three complementary APIs to understand and protect proprietary content

Blog banner for Hive's IP & Publicity Detection launch

We are excited to launch a new product suite that is purpose-built to empower our customers to protect their own IP or proactively monitor digital platforms for the potential misuse of others’ IP. 

Hive’s Intellectual Property and Publicity Detection suite consists of three complementary cloud-based APIs:

  • Media Search API: identifies when copies and variants of content from thousands of movies and TV shows are being used.
  • Likeness Detection API: identifies the “likeness” of the most popular characters or artworks in images across a wide breadth of IP domains, based on their defining characteristics.
  • Celebrity Recognition API: detects the presence of well-known figures in images. It’s powered by our face detection and face similarity models and a curated and constantly updated Custom Search Index.

All three of these APIs boast comprehensive indexes that are proactively updated. Each API is seamless to integrate and can be built into any application with just a few lines of code. Importantly, with Hive, our customers can achieve speed at scale, as we serve real-time responses to billions of API calls each month.

Media Search API

Hive’s Media Search API automates human-like visual analysis to catch reposts of movies and TV shows. Our Media Search API is a powerful tool for both digital platforms who want to avoid hosting copyright-protected media, as well as content providers and streaming sites looking to be alerted to unauthorized reposts of their proprietary content on digital platforms. 

Our Media Search API detects not only exact duplicates, but also modified versions, leveraging our Image Similarity Model. This includes manual image manipulations like rotations and text overlays, as well as more subtle augmentations such as introduction of noise, filters, and other pixel-level changes.

Additionally, for each query, the Media Search API response includes valuable metadata such as IMDB ID, content type (movie or TV show), title, relevant timestamps, and season and episode numbers (if applicable). This metadata empowers our customers to have the full context surrounding this APIs match results. 

Finally, Hive’s Media Search API brings to bear a comprehensive search index that is regularly and proactively updated, so our matches are always up-to-date. You can learn more about Hive’s Media Search API on our documentation page

Likeness Detection API 

To complement our Media Search API, we are launching our Likeness Detection API, which identifies a comprehensive set of characters and artworks across the most well-known intellectual property domains. 

Hive’s Likeness Detection API is trained on thousands of images per character or artwork across a wide breadth of domains in which that particular subject may have appeared. As a result, our Likeness Detection API is able to identify the “likeness” of well-known characters in any context, based on their defining characteristics. For example, our Likeness Detection API understands that blue costume + red cape + “S” emblem represents the likeness of a certain Kryptonian superhero, whether that subject appears in a live action film, cartoon, halloween costume, or AI-generated image.

Like our Media Search API, our Likeness Detection API is a powerful tool for digital platforms to proactively avoid hosting copyright-protected content, as well as for content creators and streaming platforms to monitor for the misuse of their proprietary content.

However, Hive’s Likeness Detection API also empowers Generative AI platforms to proactively filter and remove potentially copyright-protected characters buried in their datasets, before training text-to-image models. Of course, Likeness Detection API is also capable of detecting the likeness of characters within AI-generated images themselves, which may be highly stylized. 

Finally, beyond monitoring for the potential misuse of proprietary content, digital platforms can leverage our Likeness Detection API to more deeply understand the content that their users are engaging with. Understanding the popular IP that users are posting and sharing is a valuable tool for contextual ad-targeting and improving content recommendation systems. Visit our documentation page to learn more about Hive’s Likeness Detection API. 

Celebrity Recognition API 

Rounding out Hive’s IP and Publicity Detection suite is our Celebrity Recognition API, which enables our customers to identify thousands of celebrities, politicians, athletes, and other well-known public figures in images and videos.

Hive’s Celebrity Recognition API automates human-like perceptual comparisons to identify any public figures visible in an image or video. Our Celebrity Recognition API is powered by our face detection and face similarity models and a curated and constantly updated Custom Search Index. Given an input image, Hive detects all faces present and returns a bounding box and a match for each, as well as a confidence score. When the face does not belong to a celebrity, the string returned is “No Match” and no confidence score is returned.

Paired with Hive’s AI-Generated Content Classification APIs, social platforms can use our Celebrity Recognition API to prevent the proliferation of political or personal misinformation by filtering content for specific well known figures, as well as screening for deepfakes or AI-generated content.

Additionally, digital platforms can use our Celebrity Recognition API to easily sort and tag large media libraries by automatically detecting which celebrities are present. Similarly, streaming platforms and online media databases can quickly identify which actors appear in any frame of films, TV shows, interviews, and more in order to highlight specific actor details to enrich their user experiences. 

Finally, Hive’s Celebrity Recognition API can equip celebrities themselves, or the agencies who represent them, to monitor digital platforms for potential misuse of their likeness, enabling proactive brand protection for well-known public figures. To learn more, check out our documentation page for Celebrity Recognition API

How you can Use IP and Publicity Detection Products Today

With our launch of Hive’s IP and Publicity Detection Products, Hive is bringing to market a comprehensive suite of AI models for understanding and protecting content. However, don’t just trust us, test us: reach out to sales@thehive.ai and our team can share API keys and credentials for your new endpoints. 

BACK TO ALL BLOGS

Matching Against CSAM: Hive’s Innovative Integration with Thorn’s Safer Match

Hive's Innovative Integration with Thorn's Safer Match

An image of the Hive and Thorn logos side by side

We are excited to announce that Hive’s Partnership with Thorn is now live! Our current and prospective customers can now easily integrate Thorn’s Safer Match, a CSAM (child sexual abuse material) detection solution, using Hive’s APIs.

The Danger of CSAM

The threat of CSAM involves the production, distribution, and possession of explicit images and videos depicting minors. Every platform with an upload button or messaging capabilities is at risk of hosting child sexual abuse material (CSAM). In fact, in 2023 alone, there were over 104 million reports of potential CSAM reported to the National Center of Missing and Exploited Children.

The current state-of-the-art approach is to use an encrypting function to “hash” the content and then “match” it against a database aggregating 57+ million verified CSAM hashes. If the content hash matches against the database, then the content can be flagged as CSAM.

How the Integration Works 

When presented with visual content, we first hash it, then match it against known instances of CSAM.

  1. Hashing: We take the submitted image or video, and convert it into one or more hashes.
  2. Deletion: We then immediately delete the submitted content ensuring nothing stays on Hive’s servers.
  3. Matching: We match the hashes against the CSAM database and return whether the hashes match or not to you.

Hive’s partnership with Thorn allows our customers to easily incorporate Thorn’s Safer Match into their detection toolset. Safer Match provides programmatic identification of known CSAM with cryptographic and perceptual hash matching for images and for videos, through proprietary scene-sensitive video hashing (SSVH).

How you can use this API today:

First, talk to your Hive sales rep, and get an API key and credentials for your new endpoint.

Image

For an image, simply send the image to us, and we will hash it using MD5 and Safer encryption algorithms. Once the image is hashed, we return the results in our output JSON.

Video

You can also send videos into the API. We use MD5 hashes and Safer’s proprietary perceptual hashing  for videos as well. However, they have different use cases. MD5 will return exact match videos and will only indicate whether the whole video is a known CSAM video.

Additionally, Safer will hash different scenes within the video and will flag those which are known to be violating. Safer scenes are demarcated by a start and end timestamp as shown in the response below. 

Note: For the Safer SSVH, videos are sampled at 1FPS.

How to Hive processes media to match against Thorn's classifier and the format of the response

For more information, you can reference our documents.

Teaming Up For a Safer Internet

CSAM is one of the most pervasive and harmful issues on the internet today. Legal requirements make this problem even harder to tackle, and previous technical solutions required significant integration efforts. But, together with Thorn’s proactive technology, we can respond to this challenge and help make the internet a safer place for everyone.

BACK TO ALL BLOGS

Digiday

BACK TO ALL BLOGS

Forbes

BACK TO ALL BLOGS

NBC Bay Area