Hive is excited to participate in Advertising Week 2020, kicking off Tuesday, September 29th. Watch our promotional video to learn more about how Hive is working with media owners, brands, agencies, and others in the media ecosystem.
Social media platforms increasingly play a pivotal role in both spreading and combating hate speech and discrimination today. Now integrated into Hive’s content moderation suite, Hive’s hate model enables more proactive and comprehensive visual and textual moderation of hate speech online.
Year over year, our content moderation suite has emerged as the preeminent AI-powered solution to both help platforms keep their environments protected from harmful content, and to dramatically reduce the exposure of human moderators to sensitive content. Hive’s content moderation models have consistently and significantly outperformed comparable models, and we are proud to currently work with more than 30 of the world’s largest and fastest-growing social networks and digital video platforms.
Today we are excited to officially integrate our hate model into our content moderation product suite, helping our current and future clients combat racism and hate speech online. We believe that blending our best-in-class models with the significant scale of our clients’ platforms can result in real step-change impact.
Detecting hate speech is a unique challenge that is dynamic and evolving rapidly. Context and subtle nuances vary widely across cultures, languages, and regions. Additionally, hate speech itself isn’t always explicit. Models must be able to recognize subtleties quickly and proactively. Hive is committed to taking on that challenge and, over the past months, we have partnered with several of our clients to ready our hate model for today’s launch.
Hate speech can occur both visually and textually with a large percentage occurring in photos and videos. Powered by our distributed global workforce of more than 2 million registered contributors, Hive’s hate model is trained on more than 25 million human judgments and supports both visual classification models and text moderation models.
Our visual classification models classify entire images into different categories by assigning a confidence score for each class. These models can be multi-headed, where each group of mutually exclusive model classes belongs to a single model head. Within our hate model, some examples of heads include the Nazi and KKK symbols, and other terrorist or white supremacist propaganda. Results from our model are actioned according to platform rules. Many posts are automatically actioned as safe or restricted; others are routed for manual review of edge cases where a symbol may be present but not in a prohibited use. Our visual hate models will typically achieve >98% recall and a <0.1% false positive rate. View our full documentation here.
Our text content moderation model is a multi-head classifier that will now include hate speech. This model automatically detects “hateful language” – defined, with input from our clients, as any language, expression, writing, or speech that expresses / incites violence against, attacks, degrades, or insults a particular group or an individual in a particular group. These specific groups are based on protected attributes such as race, ethnicity, national origin, gender, sex, sexual orientation, disability, and religion. Hateful language includes but is not limited to hate speech, hateful ideology, racial / ethnic slurs, and racism. View our full documentation here.
We are also breaking ground on solving the particularly challenging problem of multimodal relationships between the visual and textual content, and expect to be adding multi-modal capabilities over the next weeks. Multimodal learning allows our models to understand the relationship between both text and visual content in the same setting. This type of learning is important to better understand the meaning of language and the context in which it is used. Accurate multimodal systems can avoid flagging cases where the visual content on its own may be considered hateful, but the presence of counterspeech text — where individuals speak out against the hateful content — negates the hateful signal in the visual content. Similarly, multimodal systems can help flag cases where the visual and textual content independently are not considered to be hateful, but in the context of one another are in fact hateful, such as hateful memes. Over time, we expect this capability to further reduce the need for human reviews of edge cases.
Today’s release is a milestone we are proud of, but merely the first step in a multi-year commitment to helping platforms filter hate speech from their environments. We will continue to expand and enhance model classification with further input from additional moderation clients and industry groups.
TV advertising has been arguably more dynamic than ever during the COVID-19 pandemic. How will the lingering uncertainty around the 2020-2021 school year impact how marketers approach Back-to-School campaigns? Research using Mensio, a media analytics platform developed by Hive and Bain Media Lab, highlights how advertisers approached Back-to-School campaigns in 2019 and what to expect in 2020.
This year’s NFL Draft stole headlines with record ratings. Viewers saw not only a new format with cameras in homes, but also significant changes in the brands who advertised.
With reported year-over-year viewership gains of roughly forty percent, the 2020 NFL Draft was an unqualified success for viewers, brands, and the airing networks – all starved for live sports content as the COVID-19 disruption persists.
For those who watched in 2019, this year’s event looked different. Notably, more than 600 camera feeds enabled a “virtual draft” format featuring teams, players, and commentators connecting from their homes. Beyond the altered format, viewers were also exposed to a much different set of brands.
Of the 150 advertisers who ran spots across ESPN, ABC, and NFL Network during this year’s event, only 61 also advertised in the 2019 NFL Draft (see Figure 1). This overlapping group was led by brands including Verizon, Lowe’s, Pizza Hut, Taco Bell, and State Farm.
Several major automakers including GMC, Nissan, and Honda were among the 96 brands sitting out this year’s draft after being active in the 2019 event. Filling the gaps were 89 brands not active in last year’s event, with IBM, BMW, John Deere, and DoorDash among the new-to-Draft brands with heavy presence.
The carousel of brands impacted category-level share of voice as well. Last year, ads from travel & leisure, automotive, media & entertainment, and retail advertisers made up 43 percent of airings across ESPN, ABC, and NFL Network; this year, that number decreased to 27 percent (see Figure 2). Growing share of voice year-over-year were quick service restaurants, insurers, tech companies, and streaming video platforms.
Over the past month, Hive and Bain Media Lab have monitored the continued increase in TV ad campaigns related to COVID-19, tracking the flighting and messaging of the more than 160 brands who have released bespoke campaigns. Across the TV universe, these campaigns made up 19 percent of all airings during the past week. This concentration was significantly higher during the NFL Draft, with 35 percent of all spots featuring coronavirus-related campaigns (see Figure 3). This peaked at 41 percent of national TV ad airings on the NFL Network feeds, 34 percent on ABC, and 29 percent on ESPN.
A host of league and broadcast sponsors achieved additional exposure in-content with logo presence in the telecast. In total across the three days, 33 brands received 15 or more seconds of in-content logo exposure, excluding league, team, and network logos (see Figure 4). This analysis was completed using Hive’s logo model, using a computer vision model trained with more than two million manhours of human-labeled training data and able to automatically detect and value the presence of logos from more than 5,000 brands.
Visible brands included Lowe’s as the presenting sponsor and other broadcast sponsors scattered into the broadcasts (e.g. “Autotrader Trade Alerts”). League sponsors including Microsoft and Gatorade had visible product placement in the telecast, as did Bose which monopolized the headphones and earphones used by players and teams during the event (albeit without camera-visible branding).
Interestingly, many of the top brands visible within the 2020 NFL Draft programming earned their exposure primarily through highlight footage from past events. This included Nike, which had the second-most total exposure, as well as four other on-field apparel brands which ranked among the top 15. College bowl sponsors including Chick-Fil-A and Allstate also made the list, making the case for always-on measurement of sponsorship exposures. While the NFL Draft is one example of this, separate research from Hive has found that shoulder programming and highlights consistently amplify valuations for sports sponsorships – sometimes by as much as twice the value of whistle-to-whistle measurement.
With the intrigue of draft selections passed, the key question in the sports world now returns to when and how games will resume. If this weekend was any indication, changes to the format shouldn’t have any negative impact on the demand from viewers or brands.
Note: Published Bain Media Lab research relies solely on third-party data sources and is independent of any data or input from clients of Bain & Company
Two weeks ago, we published a set of TV advertising trends in the context of the COVID-19 pandemic. As stay-at-home mandates have expanded and extended as the calendar turned to April, brands are continuing to evolve how they are messaging to consumers during the crisis.
The number of brands releasing TV ad campaigns specific to the coronavirus has continued to grow, starting with Verizon on Sunday, March 15, and reaching 123 brands and counting four weeks later. COVID-19 campaigns composed just 2.5 percent of all national TV ad airings on Sunday, March 22; this increased to 13 percent by Sunday, March 29, and just above 15 percent by Sunday, April 5. For now, the mix has stabilized around 15 percent of all national TV ad airings even as 21 additional brands released new campaigns in the past week (see Figure 1).
While automakers and telecom providers grabbed two-thirds of all airings in the week ending March 22, the first week of COVID-19 campaigns, they represented just over 20 percent of airings in the week ending April 12 – dwarfed by a surge in airings from restaurants and retailers that now make up more than half of the week’s coronavirus-related ad airings (see Figure 2).
While a diverse set of brands have released COVID-19 campaigns, industry verticals are not uniform in if and how brands are choosing to message about COVID-19 on television.COVID-19 campaigns now make up more than half of all national TV ad airings within the restaurant, automotive, and telecom categories; conversely, coronavirus-related airings still make up less than five percent of total CPG category airings (see Figure 3).
While the difference across categories is significant, the brands that have chosen to release COVID-19 campaigns tend to make them the majority of their messaging. Coronavirus-related spots have exceeded 80 percent of total airings for more than half of the brands who have released COVID-19 campaigns to date (see Figure 4).
What brands are saying continues to vary across categories and, increasingly, within them.
More than 88 percent of restaurant airings message product and offering changes, such as “contactless” delivery and pickup options. Conversely, more than 83 percent of airings from financial services & insurance companies communicate general support and empathy.
As more brands join the conversation, messages within some categories are starting to become more diverse.
Airings from retailers, primarily driven by “big box” brands, are roughly split between messages of general support, communication of product and offering changes, and thematic marketing (i.e. messaging existing offerings in the context of the COVID-19).
Stay-at-home orders impacted how Americans celebrated Easter this year, and COVID-19 also appeared to impact how much TV advertising is focused on Easter-related messaging. In 2019, 16 brands released Easter-themed campaigns – led by eight retailers and six chocolate manufacturers.
The number of brands with Easter ad campaigns on TV dropped to 11 brands this year. Four of the eight retailers with Easter campaigns on TV in 2019 were not at all active with TV advertising in the two weeks leading up to Easter this past Sunday, and two of the remaining four decreased national ad airings by more than 80 percent compared to 2019. In aggregate, this resulted in a year-over-year decrease of almost 50 percent in total Easter campaign airings by retailers.
Conversely, the environment did not reduce campaigns from chocolate makers. All six brands were active across years, with airings roughly equal year-over-year Easter-themed airings (See Figure 5).
Earlier this month, as an investment in industry relationships during the disruption, Bain & Company and Hive announced that a no-cost trial version of Mensio will be made available upon request to any national TV brand, full-service media agency, or U.S. TV ad sales team. The trial version of Mensio will include:
Access can be requested at: https://mensio.com/covid-19.
Note: Ongoing analysis and perspectives will be shared throughout the Bain & Company and Hive LinkedIn pages. Please follow for notification of additional releases:
Note: Published Bain Media Lab research relies solely on third-party data sources and is independent of any data or input from clients of Bain & Company
The scope of disruption from COVID-19 has included brands and agencies. Here’s how different brands and industries have adapted their TV advertising during the crisis.
While the first U.S. coronavirus case was reported on January 21, the impact on most Americans wasn’t material until mid-March when restrictions on travel and public gatherings and an increasing number of stay-at-home orders set in. Despite many working from home since, brands and agencies have been quick to adapt their messaging to acknowledge the coronavirus outbreak.
National public service announcements from the Center for Disease Control began on March 13 and have since aired almost 1,500 times across more than 40 networks.
Verizon was the first brand to acknowledge the environment in its TV commercials, launching a campaign during the Democratic Debate on March 15 focusing on network stability so that “during times like this, Americans can stay connected to work, school and, most importantly, to each other.” Versions of the creative have since aired more than 2,500 times across 54 networks, in addition to airings from Verizon’s other coronavirus-related campaigns since launched.
During the week of March 16, 22 brands across industries aired national TV ad spots explicitly or implicitly addressing the crisis. During the following week, that number increased to 62 brands.
There has been significant variability across categories in terms of how many brands have released messages, and how quickly those campaigns were released (see Figure 1). Both weeks, restaurants and automakers had the highest count of brands with active coronavirus-related TV ad campaigns. Across categories, there were at least twice as many brands active during the week of March 23 compared to the week of March 16.
Initial coronavirus-related creatives fell into four broad buckets of messaging: 1) product and offering changes, 2) general support, 3) thematic marketing, and 4) virus-related information and calls-to-action.
Through March 29, 52 percent of campaigns and 64 percent of airings addressed product and offering changes. 15 restaurant brands composed the plurality of this group. Several quick service restaurants introduced “contactless” drive-thru, pickup, and delivery experiences; casual dining brands such as Chili’s and Denny’s announced waived delivery fees. Automotive brands were the next largest cohort in messaging product or offering changes, with nine brands releasing campaigns. This list included General Motors’ brands, which announced free OnStar Crisis Assist services and in-vehicle Wi-Fi data for existing Chevrolet, Buick, Cadillac, and GMC owners as well as zero percent financing with deferred payments and at-home delivery options for new buyers.
26 percent of campaigns and 20 percent of airings conveyed a diverse set of messages broadly aiming to show empathy and convey support. Quilted Northern and Angel Soft affirmed their commitment to restocking shelves with toilet paper. Anheuser-Busch announced that Budweiser would redirect its sports investments toward hosting American Red Cross blood drives at stadiums across the country. Walmart thanked its employees, many still working in-store to serve customers’ needs through the crisis.
Eight brands launched campaigns featuring existing products and services in the context of the crisis. Food delivery service DoorDash was an example, messaging that its network of restaurants was open for delivery through the crisis.
Six brands launched campaigns with informative messages, in addition to a series of public services announcements from parties including the CDC and American Red Cross. Among the brands, Clorox shared a brand-relevant informational message providing instruction on how best to kill germs in the home.
While consumer surveys to date have generally found positive receptivity to brands acknowledging COVID-19 in their marketing messages, advertisers will face a challenge to be differentiated over time. Even among the initial set of brands airing coronavirus-related creatives, the concentration of message themes has been relatively consistent within categories. 98% of restaurant airings have highlighted product or offering changes, as have 89% of automotive ad airings (see Figure 2).
“Even if stores are closed or products are sold out, TV will remain a valuable brand-building channel for marketers. However, as the pandemic continues, brands will need to continue to evolve their messages,” said Laura Beaudin, a partner at Bain & Company, who leads the firm’s Marketing Excellence practice. “Consumers won’t want to see a full commercial break with each advertisement telling them how to wash their hands.”
Restaurants have been among the hardest hit industries during the COVID-19 outbreak, with many closing dining rooms at the request of local officials. While this has resulted in a growing number of independent restaurants closing their doors during the disruption, it has pushed quick service restaurants and casual dining chains to change their messaging.
While several restaurants have released new campaigns specific to the outbreak, including those promoting “contactless” transactions, the broader category has shifted its mix of national TV ads towards promoting off-premise dining using both new and preexisting creatives. During the four weeks ending March 15, 24 percent of restaurant airings highlighted pickup or delivery options, either as the focal point of the message or with an end card (including those promoting partnerships with delivery aggregators such as DoorDash and GrubHub). That mix of restaurant airings promoting delivery and pickup options increased to 28 percent during the week of March 16 and surged to 52 percent during the week of March 23.
Bain & Company and Hive also announced today that a version of Mensio will be made available upon request to any national TV brand, media agency, or TV ad sales team, providing users platform access through at least April 2020 including:
Eligible users can request access by registering at thehive.ai/mensio-covid-19.
“Brands and agencies face uncertainty over if and how to maintain TV advertising investments during the COVID-19 crisis, what to message, and how competitive brands are responding,” said Dan Calpin, president of Hive Media and a senior advisor to Bain & Company. “The playbook on how to do this right is still being written, but it’s safe to say that no brand wants to be remembered for saying the wrong thing or nothing at all.”
Calpin added, “While the need for real-time competitive intelligence exists now more than ever, we know many companies face contract freezes preventing access to new tools to help understand how the landscape is changing. We view this offer as an opportunity to invest in the industry through the disruption.”
Note: Ongoing analysis and perspectives will be shared throughout the Bain & Company and Hive LinkedIn pages. Please follow for notification of additional releases:
Note: Published Bain Media Lab research relies solely on third-party data sources and is independent of any data or input from clients of Bain & Company
Social platforms face significant PR and revenue risks during the coronavirus crisis, challenged to maintain safe environments in the face of constrained human content moderation and insufficient in-house AI; Hive is using AI and its distributed workforce of 2 million contributors to help
SAN FRANCISCO, CA (March 23, 2020) – The extraordinary measures taken worldwide to limit the spread of the coronavirus disease have disrupted the global economy, as businesses across industries scramble to adapt to a reality few were prepared for. In many cases, companies have stalled operations – with notable examples including airlines, movie theaters, theme parks, and restaurants among others.
The disruption facing consumer technology companies like Google, Facebook, Twitter, and others is different. Engagement on social media platforms is unaffected, if not boosted, by the outbreak. However, underneath user trends are significant public relations and revenue risks if content moderation cannot keep up with the volume of user-generated content uploads.
Hive, a San Francisco-based AI company, has emerged as a leader in helping platforms navigate the disruption through a combination of data labeling services at scale and production-ready automated content moderation models.
Hive operates the world’s largest distributed workforce of humans labeling data, now more than 2 million contributors from more than 100 countries, and has been able to step in to support emergent content moderation data labeling needs as contract workforces of business process outsourcers (BPOs) have been forced to go on hiatus given their inability to work from home. Further, Hive’s suite of automated content moderation models have consistently and significantly outperformed capable models from top public clouds, and are being used by more than 15 leading platforms to reduce the volume of content required for human review.
It is no secret that major social platforms employ tens of thousands of human content moderators to police uploaded content. These massive investments are made to maintain a brand safe environment and protect billions of dollars of ad revenue from marketers who are fast to act when things go wrong.
Most of this moderation is done by contract workers, often secured through outsourced labor from firms like Cognizant and Accenture. Work from home mandates spurred by COVID-19 have disrupted this model, as most of the moderators are not allowed to work from home. Platforms have suggested that they will use automated tools to help fill the gap during the disruption, but they have also acknowledged that this is likely to reduce effectiveness and to result in slower response times than normal.
Hive has emerged in a unique position to meet emergent needs from social media platforms.
As BPOs have been forced to stand down onsite content moderation services, significant demand for data labeling has arisen. Hive has been able to meet these needs on short notice, mobilizing the world’s largest distributed workforce of humans labeling data, now more than 2 million contributors sourced from more than 100 countries. Hive’s workforce is paid to complete data labeling tasks through a consensus-driven workflow that yields high quality ground truth data.
“As more people worldwide stay close to home during the crisis and face unemployment or furloughs, our global workforce has seen significant daily growth and unprecedented capacity,” says Kevin Guo, Co-Founder and CEO of Hive.
Among data labeling service providers, Hive brings differentiated expertise to content moderation use cases. To date, Hive’s workforce has labeled more than 80 million human annotations for “not safe for work” (NSFW) content and more than 40 million human annotations for violent content (e.g. guns, knives, blood). Those preexisting job designs and workforce familiarity has enabled negligible job setup for new clients signed already this week.
Platforms are also relying on Hive to reduce the volume of content required for human review through use of Hive’s automated content moderation product suite. Hive’s models – which span visual, audio, and text solutions – have consistently and significantly outperformed comparable models from top public clouds, and are currently helping to power content moderation solutions for more than fifteen of the top social platforms.
Guo adds, “We have ample capacity for labeling and model deployment and are prepared to support the industry in helping to keep digital environments safe for consumers and brands as we all navigate the disruption caused by COVID-19.”
For press inquiries, contact Kevin Guo, Co-Founder and CEO, at kevin.guo@thehive.ai.
Hive has been named to Fast Company’s prestigious annual list of the World’s Most Innovative Companies for 2020
SAN FRANCISCO, CA (March 10, 2020) – Hive has been named to Fast Company’s prestigious annual list of the World’s Most Innovative Companies for 2020.
The list honors the businesses making the most profound impact on both industry and culture, showcasing a variety of ways to thrive in today’s fast-changing world. This year’s MIC list features 434 businesses from 39 countries.
“It’s an honor to be featured in Fast Company’s list of the Most Innovative Companies for 2020,” said Kevin Guo, Co-Founder and CEO of Hive. “This recognition follows a year of step-change growth in Hive’s business and team, and symbolizes our progress in powering practical AI solutions for enterprise customers across industries.”
Hive is a full-stack AI company specialized in computer vision and deep learning, serving clients across industries with data labeling, model licensing, and subscription data products. During 2019, Hive grew to more than 100 clients, including 10 companies with market capitalizations exceeding $100 billion.
At the core of Hive’s business, the company operates the world’s largest distributed workforce of humans labeling data – now boasting nearly 2 million registered contributors globally. Hive’s workforce hand-labeled more than 1.3 billion pieces of training data in 2019, inputs to a consensus-driven workflow that powers deep learning models with unparalleled accuracy compared to similar offerings from the largest public cloud providers.
The company’s core models serve use cases including automated content moderation, logo and object detection, optical character recognition, voice transcription, and context classification. Across its models, Hive processed nearly 20 billion API calls in 2019.
The company also operates Mensio, a media analytics platform developed in partnership with Bain & Company that integrates Hive’s proprietary TV content metadata on commercial airings and camera-visible sponsorship placements with third-party viewership and outcome datasets. Mensio is currently in use by leading TV network owners, brands, and agencies for competitive intelligence, media planning, and optimization.
Fast Company’s editors and writers sought out the most groundbreaking businesses on the planet and across myriad industries. They also judged nominations received through their application process.
The World’s Most Innovative Companies is Fast Company’s signature franchise and one of its most highly anticipated editorial efforts of the year. It provides both a snapshot and a road map for the future of innovation across the most dynamic sectors of the economy.
“At a time of increasing global volatility, this year’s list showcases the resilience and optimism of businesses across the world. These companies are applying creativity to solve challenges within their industries and far beyond,” said Fast Company senior editor Amy Farley, who oversaw the issue with deputy editor David Lidsky.
Fast Company’s Most Innovative Companies issue (March/April 2020) is now available online at fastcompany.com/most-innovative-companies/2020, as well as in app form via iTunes and on newsstands beginning March 17, 2020. The hashtag is #FCMostInnovative.
Hive is an AI company specialized in computer vision and deep learning, focused on powering innovators across industries with practical AI solutions and data labeling, grounded in the world’s highest quality visual and audio metadata. For more information, visit thehive.ai.
Fast Company is the only media brand fully dedicated to the vital intersection of business, innovation, and design, engaging the most influential leaders, companies, and thinkers on the future of business. Since 2011, Fast Company has received some of the most prestigious editorial and design accolades, including the American Society of Magazine Editors (ASME) National Magazine Award for “Magazine of the Year,” Adweek’s Hot List for “Hottest Business Publication,” and six gold medals and 10 silver medals from the Society of Publication Designers. The editor-in-chief is Stephanie Mehta and the publisher is Amanda Smith. Headquartered in New York City, Fast Company is published by Mansueto Ventures LLC, along with our sister publication Inc., and can be found online at www.fastcompany.com.
Improved content moderation suite with additional subclasses; now performs better than human moderators
The gold standard for content moderation has always been human moderators. Facebook alone reportedly employs more than 15,000 human moderators. There are critical problems with this manual approach – namely cost, effectiveness, and scalability. Headlines over recent months and years are scattered with high-profile quality issues – and, increasingly, press has covered significant mental health issues affecting full-time content moderators (View article from The Verge).
Here at Hive, we believe AI can transform industries and business processes. Content moderation is a perfect example: there is an obligation on platforms to do this better, and we believe Hive’s role is to power the ecosystem in better addressing the challenge.
We are excited to announce the general release of our enhanced content moderation product suite, featuring significantly improved NSFW and violence detections. Our NSFW model now achieves 97% accuracy and our violence model achieves 95% accuracy, considerably better than typical outsourced moderators (~80%), and even better than an individual Hive annotator (~93%).
Deep learning models are only as good as the data they are trained on, and Hive operates the world’s largest distributed workforce of humans labeling data – now nearly 2 million contributors globally (our data labeling platform is described in further detail in an earlier article).
In our new release, we have more than tripled the training data, built off of a diverse set of user-generated content sourced from the largest content platforms in the world. Our NSFW model is now trained on more than 80 million human annotations and our violence model trained on more than 40 million human annotations.
We were selective in our construction of the training dataset, and strategically added the most impactful training examples. For instance, we utilized active learning to select training images where the existing model results were the most uncertain. Deep learning models produce a confidence score on input images which ranges from 0 (very confident the image is not in the class) to 1.0 (very confident the image is in the class). By focusing our labeling efforts on those images in the middle range (0.4 – 0.6), we were able to improve model performance specifically on edge cases.
As part of this release, we also focused on lessening ambiguity in our ‘suggestive’ class in the NSFW model. We conducted a large manual inspection of images where either Hive annotators tended to disagree, or even more crucially, when our model results disagreed with consented Hive annotations. When examining images in certain ground truth sets, we noticed that up to 25% of disagreements between model prediction and human labels were due to erroneous labels, with the model prediction being accurate. Fixing these ground truth images was critical for improving model accuracy. For instance, in the NSFW model, we discovered that moderators disagreed on niche cases, such as which class leggings, contextually implied intercourse, or sheer clothing fell into. By carefully defining boundaries and relabeling data accordingly, we were able to teach the model the distinction in these classes, improving accuracy by as much as 20%.
Classified as clean:
Classified as suggestive:
For our violence model, we noticed from client feedback that the classes of knives and guns included instances of these weapons that wouldn’t be considered cause for alarm. For example, we would flag the presence of guns during video games and the presence of knives when cooking. It’s important to note that companies like Facebook have publicly stated the challenge of differentiating between animated and real guns (View article on TechCrunch). In this release, the model now distinguishes between culinary knives and violent knives, and animated guns and real guns, by the introduction of two brand new classes to provide real, actionable alerts on weapons.
Hive can now distinguish between animated guns and real guns:
The following knife picture is not considered violent anymore:
The improvement of our new models compared to our old models is significant.
Our NSFW model was the first and most mature model we built, but after increasing training annotations from 58M to 80M, the model still improved dramatically. At 95% recall, our new model’s error rate is 2%, while our old model’s error rate was 4.2% – a decrease of more than 50%.
Our new violence model was trained on over 40M annotations – a more than 100% increase over the previous training set size of 16M annotations. Performance also improved significantly across all classes. At 90% recall, our new model’s error rate decreased from 27% to 10% (a 63% decrease) for guns, 23% to 10% (a 57% decrease) for knives, and 34% to 20% (a 41% decrease) for blood.
Over the past year, we’ve conducted numerous head-to-head comparisons vs. other market solutions, using both our held-out test sets as well as evaluations using data from some of our largest clients. In all of these studies, Hive’s models came out well ahead of all the other models tested.
Figures 6 and 7 show data in a recent study conducted with one of our most prominent clients, Reddit. For this study, Hive processed 15,000 randomly selected images through our new model, as well as the top three public cloud players: Amazon Rekognition, Microsoft Azure, and Google Cloud’s Vision API.
At recall 90%, Hive precision is 99%; public clouds range between 68 and 78%. This implies that our relative error rate is between 22x and 32x lower!
The outperformance of our violence model is similarly significant.
For guns, at recall 90%, Hive precision is 90%; public clouds achieve about 8%. This implies that our relative error rate is about 9.2x lower!
For knives, at recall 90%, Hive precision is 89%; public clouds achieve about 13%. This implies that our relative error rate is about 7.9x lower!
For blood, at recall 90%, Hive precision is 80%; public clouds range between 4 and 8%. This implies that our relative error rate is between 4.8x and 4.6x lower!
This latest model release raises the bar on what is possible from automated content moderation solutions. Solutions like this will considerably reduce the costs of protecting digital environments and limit the need for harmful human moderation jobs across the world. Over the next few months, stay tuned for similar model releases in other relevant moderation classes such as drugs, hate speech and symbols, and propaganda.
For press or inquires, please contact Kevin Guo, Co-Founder and CEO (kevin.guo@thehive.ai)
Next-day analysis highlights trends in measured exposure during Hollywood’s biggest night
Hollywood’s biggest night, the Academy Awards, wrapped up this year’s awards season in style. Red carpet fashion started the night and Parasite stole headlines after the Korean-language film claimed four awards including Best Picture.
While awards are permanent markers of achievement, exposure is a broader prize shared by winners, nominees, performers, and presenters. Hive’s Celebrity Model, used by agencies to measure endorsement value and by media companies to enrich metadata in their video libraries, measured the screen time earned by the stars during the 2020 Oscars.
Bong Joon Ho – who took the stage as a winner four times for Parasite – earned the most time on screen during last night’s telecast of the Academy Awards, according to data from Hive’s Celebrity Model (see Figure 1). The remainder of the top 10 was made up of winners (Joaquin Phoenix, Brad Pitt, Laura Dern), presenters (Steve Martin, Chris Rock, Kristin Wiig, Maya Rudolph), and some celebs wearing multiple hats (Elton John as a winner and performer; Taika Waititi as a winner and presenter).
Much was said leading up to the event about another host-less award show short on diversity. A diverse mix of presenters and performers aimed to compensate for nominees that skewed white and male. However, while these themes stole headlines leading up to the event and were scattered across acceptance speeches during the night, most of what was said during the show was relatively consistent year-over-year (see Figure 2).
Hive’s Speech-to-Text model, with commercial uses including transcription of audio and monitoring of brand mentions in TV, radio, and digital video, was used to track mentions and keywords from within the Oscars’ telecast. Insights from what was said across award presentations and acceptance speeches included:
Hive’s Celebrity Model is trained to identify more than 80,000 public figures in media content and uniquely leverages Hive’s distributed workforce of more than 1.5 million registered contributors to efficiently optimize the precision and recall of low confidence results. Commercial uses of the model include measurement of endorsement value by agencies and enrichment of metadata in media companies’ video libraries.
Hive’s Speech-to-Text Model parses and transcribes speech data from video and audio content, and can be accessed via an API or on device. The model is trained by tens of thousands of hours of paired audio and speech data. Commercial uses of the model include transcription of audio and monitoring of brand mentions in TV, radio, and digital video.
Kevin Guo is the cofounder and CEO of Hive and is based in San Francisco. Dan Calpin is President of Hive Media and a Senior Advisor with Bain & Company based in Los Angeles; he was a founding partner of Bain Media Lab. Laura Beaudin is a Bain partner in San Francisco and leads Bain’s Global Marketing Excellence practice. Andre James is a Bain partner in Los Angeles and leads Bain’s Global Media & Entertainment practice; he was a founding partner of Bain Media Lab.