Login
A TikTok A/B Test Proves It Is Responsible for What Users See
In 2023, Bloomberg Businessweek asked TikTok for comment as it reported out a story about users who were recommended videos about self-harm and suicide, particularly teenagers like Chase Nasca. It is a difficult read. As a result of this press outreach, TikTok analyzed Nasca’s account and why he was given so much of this material that it likely played a role when he took his own life at just sixteen years old.
Olivia Carville, of Bloomberg Businessweek, recently obtained that internal report:
Chase’s account was stuck in a so-called filter bubble, an online echo chamber that pumped out repetitive content the algorithm predicted he’d want to watch. There he received an “onslaught” of suicide and self-harm content, according to the document. Because Chase was in the control group, it says, “TikTok’s filter bubble prevention strategies did not take effect on this user by design.”
Those two words, “by design,” carry extra weight against the backdrop of litigation over social media addiction that’s playing out in US courts. The crux of the legal argument being wielded against the world’s biggest social media platforms is that their products are dangerous by design. The companies, including TikTok, vehemently deny this and point to safety features they’ve introduced to protect users. But the document shows TikTok intentionally withheld a safety feature from millions of people, and why.
The “why” is simple: TikTok made a change to its recommendations system and wanted to see how it performed. TikTok rolled out this change to 90% of its users but kept 10% as a control group; Nasca’s account was in that group.
That split is odd to me — when Google experiments with new features, it first tries them with a “small” and “narrow” group, only expanding them later. Trying a new feature with nearly every user suggests to me that TikTok had a hunch this was a promising enhancement and it only needed to keep a small number as validation. In the report obtained by Carville, one consideration was “the ability to measure impact on [daily active users] and core metrics”, which suggests an alarming followup question: if a better-moderated feed hurt these metrics, would TikTok have considered rolling it back?
A steelmanned argument in TikTok’s favour is that such metrics are only one consideration in adjusting recommendations, and that depressive material was probably still seen by users who were using the newer system. And, yeah, those two things may be true, but the simple fact is that TikTok differed its recommendations system for different users, and one of them was fed an obsessive number of similar videos advocating suicide. That is a choice the company made about what videos would be seen by which users. TikTok has agency, and it should have responsibility.
