Early reports find Google’s Privacy Sandbox isn’t ready for prime time

Now that Google’s Privacy Sandbox initiative is starting to take shape, ad-tech companies have a few notes for the digital giant.

So far, there are a few kinks to be worked out, ranging from latency issues to falling prices, according to three recent reports from platforms on both the supply and demand sides, together representing the digital plumbing of the internet.

The findings build on early data that has shown the effects of Privacy Sandbox, a collection of tools designed for ad-tech companies to maintain the monetization of online advertising without invading user privacy. Privacy Sandbox has been touted by Google as a replacement for third-party cookies, which the company has repeatedly announced plans to phase out of its Chrome browser.

In January, Google removed cookies from about 1% of Chrome users, or around 30 million, to allow for testing—and almost immediately, some publishers reported that those users monetized nearly 30% worse than users with cookies. The new reports from industry players Criteo, Index Exchange, and NextRoll, which were released in late June and early July, suggest that nearly six months later, little has changed.

“We’re still seeing that 30% hit to revenue in Chrome cookieless world, and that’s a big amount of revenue for people to lose,” Paul Bannister, chief strategy officer at publishing ad-tech company Raptive, told Marketing Brew. That figure comes with a caveat, he noted, since third-party cookies remain in use.

Because Privacy Sandbox “totally changes the way advertising works today, everyone is at a kindergarten level again,” he said. “We’ve got this really powerful tool, and we don’t fully know how to use it.”

The findings

In late June, ad-tech company Criteo said that in a test involving the majority of its 18,000 advertisers and its 1,200 publishers representing 100 million weekly impressions, it found that publishers could stand to lose on average 60% of their revenue from Google Chrome if third-party cookies were depreciated today and they were to rely on Privacy Sandbox. It also claimed that Google had intended to limit publisher revenue losses to a maximum of 5%, although Google Senior Director of Product Management Victor Wong replied to a post about the report on X, stating that “5% has never been a goal. No idea where that came from in the report.”

Criteo also reported that Privacy Sandbox seemed to favor Google’s own ad-tech business, capturing a majority of the spend in tests. “This demonstrates a significant increase in publisher reliance on Google for ad revenue,” Criteo’s chief product officer and the author of the report, Todd Parsons, wrote in a company blog post detailing the findings—a particularly striking claim, considering that Google will soon head to court to defend itself against an antitrust case brought by the Department of Justice, which argues that Google has monopolized the digital ad market.

Supply-side platform Index Exchange also published similar findings last month: in tests that included data from 10 demand-side platforms, 100 publishers, and “thousands of domains,” Index Exchange found that CPMs fell 33% when advertisers used Privacy Sandbox.

Beyond that, CPMs without cookies or Sandbox tools were 36% lower overall, meaning Privacy Sandbox performed 3% better than nothing at all. “With its current limitations, Privacy Sandbox may not be an effective solution yet for general use, or it may be too costly for tech companies to ready their implementations for general availability,” according to the report. “There are major risks to publishers and the overall programmatic ecosystem that we must address to make it easier and more efficient to scale.”

Not all bad news

NextRoll, a demand-side platform, found that, in some instances, CPMs increased threefold, though Andrew Pascoe, VP of data science engineering at the company, told Marketing Brew that this was largely because the company was placing higher bids to win more impressions and gather more data.

Though concerns about publishers potentially losing revenue are legitimate, early testing doesn’t paint the full picture, Pascoe said. Cookies are still abundant and Privacy Sandbox adoption is limited. “These numbers don’t really reflect a future reality,” Pascoe said, echoing Raptive’s Bannister.

The reports also raise concerns about ad latency, meaning ads didn’t load properly, or in a sufficient amount of time. NextRoll reported that without cookies, average bid response time was five times “above the upper bound of latency that we aim for.”

“When you’re reading a webpage and you’re scrolling past an ad spot that hasn’t loaded, from a publisher perspective that is lost monetization…The person didn’t see the ad,” Bannister said. “From an advertiser perspective, your performance is worse, because the user had no possibility of engaging with that ad. It’s a lose-lose for the advertiser and the publisher in that scenario.”

What it all means

There’s real money at stake. Google Chrome is the most popular browser in the world, and, according to 33Across data cited by eMarketer, more than three-quarters of US programmatic buys relied on cookies as recently as Q3 2023—a hefty chunk of a nearly $180 billion industry.

And so far, the indications of a significant loss of revenue are a “red flag,” Michael McNeeley, SVP of product at Index Exchange, told us, but he wondered, too, about what the results would look like in a cookieless world.

“I think the question is, How indicative is it?” he said. “If the cookie were to go away, [would this] be the results that we would see? That’s kind of the open question, and I don’t know that we have an answer there yet.”

The results of these early tests from ad-tech companies could have wide-ranging effects. Google can’t kill cookies off entirely without the blessing of the UK’s Competition and Markets Authority (CMA), which opened an antitrust investigation into the company’s cookie phase-out in 2021, and the results from some third-party tests will be provided to the CMA.

In each of the three reports, the authors make recommendations that they encourage Google to adopt. Index Exchange asked Google to support different kinds of ad inventory, while Criteo asked Google to deprecate cookies in waves, a position Raptive’s Bannister said he supports. NextRoll, for its part, asked Google to provide a final date to expect the official death of the cookie so ad tech companies can better plan for the future.

Meanwhile, Google is soliciting feedback about Privacy Sandbox.

“We are encouraged to see companies building with the Privacy Sandbox and other privacy enhancing technologies. But it’s not possible to predict publisher performance based on effectiveness of a single buying platform, as publishers typically work with dozens of demand sources,” Google spokesperson Scott Westover told Marketing Brew in an email. “We expect performance numbers to evolve, and they currently don’t reflect how the overall ecosystem will perform in a true marketplace, which won’t exist until adoption expands alongside third-party cookieless traffic.”