Deepfake Incident Spotlight: The Pope in Balenciaga

In 2017, deepfakes made national headlines when an AI-generated recording of then-president Barack Obama giving a speech on fake news appeared. Ironically, the video itself was fake news. It was actor and director Jordan Peele’s attempt to show the dangers of disinformation following a polarizing election largely defined by conspiracy theories and falsehoods. Since then, there’s been an ever-present concern about the dangers of AI to truth and the media, but, since 2016, it’s largely resided on the periphery. However, the emergence of ChatGPT has renewed concerns about AI and misinformation—concerns that are not unfounded.

On March 26th, an image of the Pope sporting a white Balenciaga puffer jacket captivated the internet, leaving countless Twitter users in awe of the Pope’s fashion sense.

And it was all fake. 

The Pope/Balenciaga saga offers a case study into how fake narratives can spread--and a sneak peak at how AI-generated falsehoods may impact brands in future.

A Fake News Story Emerges - The Beginnings of the Pope’s Puffer Jacket

Most false narratives begin with a single post--often on “alternative” or less “mainstream platforms.” Think fringe blogs and sites like Reddit. From there, often through cross-posting or mainstream pick-up, they make their way to mainstream news sites and social media platforms. This is exactly what happened with Pope’s supposed puffer jacket. 

Midjourney describes itself as an “independent research lab exploring new mediums of thought and expanding the imaginative powers of the human species.” The company has a subreddit with 189,000 followers, and it primarily consists of AI-generated images. In the afternoon on Friday, March 25, a construction worker from Chicago used the Midjourney tool to create Pope in a puffer jacket, posting it first to a Facebook group called AI Art Universe and then to the Midjourney subreddit. All the post contained beside the photo was the title: “The Pope Drip.” 

By the next day, the image had made it from the fringes of the subreddit to the Twitterverse.

From Reddit to Twitter: How a Fake Story Makes it to the Mainstream 

At some point on March 25, someone who had seen the Pope in a puffer jacket image on the Midjourney subreddit decided to re-post it on Twitter. From there, the story spiraled. Tweets began to surface en masse on the Pope’s eye-catching coat, and some of them had tens of thousands or even millions of views. 

At around the same time, comments began to emerge on the original Midjourney post by users in disbelief that people actually believed the puffer jacket was real. Of course, many people seeing the photo in their Reddit feed probably didn’t know what Midjourney was, and the post itself didn’t provide any clarifying details. 

The Debunking

A fake story rarely lasts forever--even if some people continue to believe it long after the truth is uncovered. Especially since 2016 and the advent of “fake news,” there have been concerted efforts to debunk false narratives. Sites like NewsGuard rank the reliability of news sources, while websites like PolitFact fact check popular stories. In other cases, brands themselves put out statements to correct misconceptions. 

On March 26, Snopes, the fact-checking website, published an article explaining that the image was, in fact, AI-generated. It was that same day that PeakMetrics’ narrative monitoring detected early signals of a “fake photo of Pope Francis in a puffer jacket ” spreading across social media. By March 27th, there were countless articles in mainstream media about the fake photo and what it means for the future of truth, AI…and fashion. 

Assessing the Fallout

Any false narrative will inevitably have consequences for those involved. While in the case of Pizzagate, the consequences were nearly deadly, the case of the Pope in the puffer jacket was relatively innocuous. Oddly enough, in some ways, the viral deepfake may have been beneficial to Balenciaga in the short term. 

Thanks to the popularity of the Pope’s puffer jacket, other Balenciaga “deepfakes” emerged and went viral. It’s spawned a series of videos and TikTok stories starring popular celebrities and TV and book characters, with the Harry Potter gang joining the Lord of the Rings and Pirates of the Caribbean casts as Balenciaga deepfake stars.

In fact, by March 30th, one of the narratives tracked by PeakMetrics was, “People are fascinated by AI-generated videos featuring Balenciaga.” Other, similar narratives have followed, including “people expressing their love for Balenciaga memes“ and “people are enjoying and obsessed with Balenciaga’s TikTok AI videos.” 

For the brand, while these videos are fake, the videos have breathed new life into a brand that was, just a few months ago, ‘disowned’ by the likes of Kim Kardashian. 

In November, the brand faced widespread criticism and a #cancelBalenciaga campaign on Twitter after debuting photos of children with teddy bears in bondage harnesses and costumes. In a second controversial ad that same month, the brand included a copy of a Supreme Court case involving child pornography. This devolved into yet another “fake” for the brand--that Balenciaga was promoting child pornography and trafficking.  

In the wake of the Pope’s puffer jacket, those controversies have taken a backseat. Kim Kardashian has started promoting the brand again, and Balenciaga is currently most associated with “fun” and “popular” memes and AI videos. 

That said, their ad controversy has not disappeared altogether. In fact, this newfound attention has thrust the November conspiracy theory back into the spotlight, at least among some audiences. By April 1st, Peak Metrics had begun tracking a new narrative, primarily on social media--one involving anger that people appear to have forgotten Balenciaga’s “child pornography” controversery.

What the Future Holds

What happens to Balenciaga’s brand is arguably not as important as what the Pope deepfake says about the future of information and truth. As Fast Company reporter Ryan Broderick said, “I think Balenciaga pope might be the first real mass-level AI misinformation case.” Countless individuals shared the image without a thought--and never considered that it might not be real. Fortunately, no one appears to have doubted the truth once it came out.

This is just the beginning. As AI becomes more advanced, it will be harder to distinguish the fakes from the real images, particularly given how fast social media moves. The Pope’s fake puffer jacket went viral and was then debunked within a mere twenty-four hours. And, while careful examination may be able to help you spot a deepfake, most people don’t think that much before posting.

So, where does that leave consumers--and the brands that are powerfully impacted by what people are saying about them online? For starters, they need to be constantly monitoring what narratives are emerging about their brand from all corners of the web. You can’t debunk a fake unless you know it exists, and when it comes to misinformation, every hour matters. 

As AI advances forwards, hyper-vigilance will be key--for both brands and consumers.