Measuring inclusion in your work

It’s hard to look back at work you’re proud of and admit that it’s not good enough. But the first step in making real progress towards authentically inclusive work is understanding where you currently stand.

Our learnings

Each year, we apply these principles to complete a creative audit of our own work. Here are some of the top findings from our most recent audits.

2020 Creative Audit

Google Marketing
2020 Creative Audit

A few years ago, we began auditing Google’s creative work to understand how we’re doing on inclusion and representation in our marketing. The resulting insights have helped us celebrate progress, set goals, and highlight opportunities for work that authentically represents the world.

In 2020, with the help of the Geena Davis Institute on Gender in Media, external evaluators, and machine learning tools, we conducted over 10,000 evaluations of Google’s 2019 marketing assets that launched in the U.S. Together, these evaluations formed the basis of our creative audit.

A graphic illustration showing that 66% of our ads included women. Text stating that 50.8% of the U.S. population identifies as Female.

While we include women, we still give men the spotlight.

66% of our ads portray women, but they have less than half of the speaking and screen time. Additionally, the women featured were less likely to have darker skin tones than men.
Three bar graphs showing that in 2018: 8% of our ads included Asian people, in 2019: 12% of our ads included Asian people, and in 2019: 0.14% of our ads included Native Hawaiian and Pacifiic Islander people. Text stating that 5.9% of the U.S. population identifies as Asian.

Our representation of Asian people is on the rise.

We included Asians and Asian Americans above U.S. population demographic levels, and Native Hawaiian and Pacific Islander representation is near benchmark in our ads.
Two bar graphs showing that in 2018: 23% of our ads included Black people, and in 2019: 32% of our ads included Black people. Text stating that 13.4% of the U.S. population identifies as Black or African American.

We’re making progress on including Black people in thoughtful ways.

Black stories are increasingly thoughtful, expanding beyond stereotypical portrayals of dancing, music, and sports.
A graphic illustration showing two people, a younger man and an older woman with a cane, and that 0.14% of our included Indigenous people. Text stating that 1.3% of the U.S. population identifies as American Indian/Native American and Alaska Native.

We must strengthen Indigenous representation.

This starts with prioritizing more stories, more prominent featuring, and more thoughtful portrayals of Indigeneous people to reach U.S. population demographic levels.
A pie chart showing that 6% of our ads portrayed Latinos/Latinx people compared to 18.5% of the U.S. identifies as Hispanic or Latino

We need to amplify
 Latino/Latinx stories.

When it comes to representing Latinos/Latinx people at U.S. population demographic levels, we’re only a third of the way there.
A graphic illustration showing three people holding hands, one wearing a skirt and two wearing pants, and that 1% of our ads included people perceived to be a part of the LGBTQ+ community. Text stating that 12% of the U.S. population identified as LGBTQ.

We must tell more LGBTQ+ stories and go beyond Pride to tell them.

We underrepresented members of the LGBTQ+ community and relied too heavily on Pride associations to tell their stories.
A graphic illustration showing two people, one with a cane and one with a prosthetic arm and glasses, and that 1% of people portrayed in our work have an apparent disability. Text stating that 26% of adults in the U.S. have some type of disability.

People with disabilities are missing from our creative work.

In our representation of the disabled community, we overly relied on wheelchairs and other stereotypical signals of apparent disabilities.

Learn more about our audit rubric and methodology.

Create a rubric

A simple rubric can help you measure and understand who's being represented and how well they're being portrayed in your work. Currently, the stereotypes we look for and demographic benchmarks we use are U.S.-centric, so our audits cover creative that ran in the U.S. only. However, we’ve tested and used this methodology at a smaller scale in other regions.

  • Prioritize key dimensions of inclusion relevant to your market. Our audits consider age, disability, gender, LGBTQ+ identity, military status, race/ethnicity, skin tone, and socioeconomic status. See below for more details.
  • Identify room for improvement using both qualitative and quantitative measurements (e.g., what percentage of our assets contain portrayals of Latinos, and which stereotypes of Latinos show up in our work?).
  • Plan to assess how different dimensions intersect with each other (e.g., skin tone and gender, or age and race).
  • Review your rubric with diverse audiences with different perspectives to make sure you haven’t overlooked anything.

Dimensions we look for in our creative audits

  • Note that these dimensions are not intended to be fully inclusive of all possible identities. A limited set of dimensions were chosen to support measuring representation at scale. We also acknowledge that many dimensions of identity are not visible to others. Our goal is to gain a clearer understanding of whether our intent to be inclusive is perceived by others through the work we create.
  • Age: Perceived presence of people in the age groups of children, teens, young adults, adults, and older adults.
  • Disability: Perceived presence and representation of people with various types of apparent and non-apparent disabilities.
  • Gender: Perceived presence and representation of women, men, and gender expansive individuals.
  • LGBTQ+: Perceived presence and representation of LGBTQ+ people (with the understanding that LGBTQ+ stands for lesbian, gay, bisexual, transgender, and questioning or queer).
  • Military status: Perceived presence of veterans or people who are serving in the military.
  • Race and ethnicity: Perceived presence and representation of Asian, Black, Indigenous, Latino/Latinx, or white people.
  • Skin tone: Perceived presence of a diverse spectrum of skin tones from dark to light. At a minimum, we recommend covering the spectrum of dark skin tones to light skin tones on a 6-point scale.
  • Socioeconomic status: Perceived presence of low-income, middle-income, and high-income socioeconomic categories.

Review your work

When it comes to evaluating work, choosing the right reviewers is as important as establishing the set of criteria.

  • Partner with trained evaluators who will review and rate your existing and recent creative work based on your rubric.
  • Partner with expert researchers from institutes or non-profits. We’ve partnered with the Geena Davis Institute on Gender in Media to evaluate assets from many campaigns. You can also look into ANA’s #SeeHer Gender Equality Measure (GEM) and AIMM’s Cultural Insights Impact Measure (CIIM™). GEM is the global industry standard for measuring gender bias in ads and programming, while CIIM™ provides measurement and data on the impact and effectiveness of cultural insights in ads and programming.
  • To better understand four of the leading measurement systems for culture/gender portrayals in creative content, check out this Diversity Measurement Matrix created by AIMM, Geena Davis Institute, Unstereotype Alliance, and SeeHer.
  • Use public crowdsourcing tools to evaluate large sets of assets.
A grid of YouTube videos with different images of women

In 2017, we started to use Google’s machine learning tools to evaluate perceived gender and age representation in our own advertising. Recently, we partnered with the Geena Davis Institute on Gender in Media to publish The YouTube Ads Gender Representation Report that used machine learning to analyze millions of videos.

Take action

Once you've reviewed your creative, here are a few ways to take action:

  • Create new annual goals for representation and inclusion.
  • Update materials used in trainings and workshops with data and learnings from your audit so people are aware of any issues.
  • Inform creative reviews of upcoming campaigns and launches, and strive to fill gaps with future projects.