How to Disappear an Account

This is Part Three of The Price of Winning, a four-part Tinsel series on harassment, moderation, and the people who compete in public. Read Part One and Part Two. Part Four, the final installment, publishes Wednesday.

Automated moderation began as a reasonable answer to a real problem. No company operating at this scale could put a human in front of every report, and nobody serious is asking it to. The trouble is narrower and more fixable than the scale makes it sound. These systems are built to act on volume and pattern, and they are poor at weighing context. A surge of reports against one account registers as a surge of reports against one account. The machine does not ask why the surge arrived this week, or who organized it, or what the ninety days before the suddenly circulating clip actually held.

The volume alone guarantees mistakes. Kate Klonick, a law professor who studies how platforms police themselves, put the arithmetic to Vice: "If you moderate posts 40 million times a day, the chance of one of those wrong decisions blowing up in your face is so much higher." At that scale, a wrong call stops being a freak event and becomes a daily certainty, thousands of times over, and most of the people on the receiving end of one never make the news.

The people who study this professionally have been saying so for years. Jillian York, who directs international freedom of expression work at the Electronic Frontier Foundation, puts the baseline plainly: content moderation, she has written, "however it's done, human or automation, comes with fairly high error rates. Offering every user the ability to appeal ensures that those errors can be corrected." The systems are wrong often enough that the real question is what happens after they get it wrong.

The pressure, meanwhile, runs in one direction, toward removing more rather than less. The content-moderation scholar Evelyn Douek, writing in WIRED in 2021, described the ratchet: "Can moderate implies ought to moderate," she wrote, and once a takedown tool exists, "it's hard to put it back in the box." The consequence is that "content moderation is now snowballing, and the collateral damage in its path is too often ignored." The creators wrongly swept up are that collateral damage, and they are close to invisible in the numbers.

Creators and the people who advocate for them have been making a related point about how the errors are triggered, and Moxie Media Marketing, which represents independent creators, says it has watched the same sequence play out across its own roster. An account can be flagged for one heated second while the months of provocation that produced that second stay invisible, because provocation does not generate reports. The person being baited generates the reports. A motivated group learns to use that on purpose, filing complaints in coordinated waves so the sheer count reads to the system as a genuine groundswell. The result is an enforcement logic that quietly rewards whoever organizes first and loudest. A coordinated reporting campaign gets processed as a louder form of community feedback, when in practice it operates as an attack. The plain name for this belongs in the open. Organized mass-reporting is bullying routed through official channels, run by groups who have learned that the report button hits harder than the insult ever did, and an enforcement system that takes their filings at face value ends up deputizing the very people it was built to protect everyone from.

For a creator caught in one of these campaigns, the way back narrows to a single process, the appeal, and the word that fits it is opaque. The original notice rarely names the exact post or the exact rule. The appeal vanishes into a queue with no visible person at the end of it. The experience is common enough that ordinary users describe it in almost identical terms. In a March 2026 CBS News investigation into wrongful automated bans on Facebook and Instagram, a Chicago teacher named Eric Cunningham, locked out of accounts he had done nothing to lose, said what most of them conclude: "The appeal process was clearly not done by a person."

Reaching a human to fix the error can be an ordeal of its own. Amir Hosseini, a Montreal music-label founder whose business accounts were wrongly suspended by Meta's automated systems, has said it took him a month, and a paid subscription, before a person would look at his case. His ask is a modest one. "If big companies like Meta want to use [AI], fine," he told CBC News, "but at least they can leave a channel of human interaction or human support behind us, so these matters could be solved." His point is simple. When the machine gets it wrong, there should be a person within reach to set it right.

And no platform owns more of this problem than TikTok. It sits at the center of the live-creator economy, where a stream is a stage, an audience is a livelihood, and a leaderboard can be a career, and it is a paid partner in every one of those livelihoods. By its own transparency reporting, TikTok removes tens of millions of videos every month, roughly nine in ten of them flagged by automated systems. What it has not built is the narrow thing this entire problem calls for: a guarantee that a human being weighs the full context before an account, and the livelihood inside it, is permanently erased, and a plain-language explanation when it acts. Both are ordinary, buildable things, available to a platform of TikTok's size any day it decides the people earning it money deserve them. Until then, every coordinated campaign that erases a creator there runs on TikTok's own tools, on TikTok's watch, inside the gap between what the platform could do and what it does.

Creators feel the stakes of that more sharply than most, because for them the account is the business. "Criticism comes with the territory and I can take it," says Jolene Burns, the Belfast singer-songwriter who built her audience streaming live. "What scares me is how easy it is for a coordinated group to turn the reporting tools into a weapon, and how little say you have when an account just vanishes and nobody will tell you why. Artists are building their whole livelihoods on these platforms. There should be an actual person in the loop, and a real chance to be heard, before someone's work gets wiped out."

The fix that keeps coming up, from experts and affected users alike, is unglamorous and specific. George Dixon, an IT analyst quoted in the same CBS investigation after his own accounts were wrongly removed, described it as a matter of basic accountability: "I'd like to see a system where, if you're flagged for something this serious, a human double-checks the AI's work." That is close to the whole of it. An enforcement system that can erase a livelihood on the strength of an organized crowd, with no person in the loop and no real way to be heard, is dangerous to lose inside. The people most exposed are the independents, who have the most to gain from the format and the least protection when it turns on them. None of that is weather. Every part of it was built by someone, which means every part of it could be built differently.

Sources

• Kate Klonick, quoted in "The Impossible Job: Inside Facebook's Struggle to Moderate Two Billion People," Vice
• Jillian C. York, Electronic Frontier Foundation, on moderation error rates and appeals, Everything in Moderation, 2021
• Evelyn Douek, "More Content Moderation Is Not Always Better," WIRED, 2021
• Eric Cunningham and George Dixon, quoted in "Facebook, Instagram accounts wrongly banned," CBS News, March 2026
• Amir Hosseini, quoted in "Meta suspended his business's social accounts," CBC News, 2025
• TikTok, Community Guidelines Enforcement Reports, TikTok Transparency Center, 2025