What Happened When Twitter and Other Social Media Platforms Cracked Down on Extremists
In a Q&A with ProPublica reporter AC Thompson, former intelligence officer and data scientist Welton Chang explains how conspiracy theorists and violent racists have fled to smaller platforms. Once there, their words escalate and spread.
Since Donald Trump’s election in 2016, an entire ecosystem of right-wing social media platforms has sprung up — from Gab (where the alleged Pittsburgh synagogue shooter posted hateful screeds) to Parler (a hotspot for insurgent activity as Jan. 6 approaches) at the former president’s own Truth Social (which was frequented by a fan of his who was recently shot after attacking an FBI office in Cincinnati). This new wave of apps and sites follows in the footsteps of 4chan and 8kun, older internet forums that continue to attract large audiences of conspiracy theorists and violent racists.
Welton Chang knows this corner of the digital world well. A former army intelligence officer and human rights activist, Chang runs Pyrra, a small tech startup dedicated to identifying and tracking extremist ideas circulating in these spaces. Pyrra, which launched in early 2022 with $1.3 million in funding, monitors more than 20 alternative social media sites and online forums, scanning some 100 million posts per week.
Chang, a data scientist, says increased moderation of content on major social media platforms – including the ousting of figures ranging from Trump to Alex Jones – has driven a large contingent of users to the spaces Pyrra tracks. , which tend towards an absolutist vision of freedom of expression.
This conversation has been edited for length and clarity.
Can you tell me in simple terms what Pyrra does?
Pyrra is a threat intelligence company. We do three things: We collect content — publicly available information — from alternative social media sites. We use machine learning and advanced algorithms to detect violent threats, hate speech and misinformation that appear on these platforms. And then we display that information for our clients, either through reports or through a platform that we have.
We got our start in the human rights community. We were a project within Human Rights First [a U.S.-based advocacy group]. … We separated from HRF as our own company in December 2021 and launched our platform earlier this year.
In general, what are the big extremist threats that you follow these days? What worries you?
One is just the death of critical thinking and the amount of unsubstantiated speculation that becomes the truth, the little truth, on these platforms. It can range from something as innocuous as something done by a celebrity to things that really impact the health of our democracy.
But it’s also just the general lack of trust in institutions at large. … We are at an all-time low in terms of government confidence, based on all the different metrics that are out there. …
[This brings us to] the inherent inconsistency of conspiracy theories and these really bizarre ideas about how the world actually works. People believe that the government is both utterly incompetent and also all-knowing and all-knowing and capable of a massive effort like helping Bill Gates deliver the COVID vaccine through mind control via 5G technology.
These are diametrically opposed ideas, but people simultaneously believe both and say, “That’s what’s happening in the world today.
I’ve been really into this stuff since 2016, and I’m still regularly appalled, surprised, and surprised by some of the things I read on these platforms. And maybe the day I get used to this stuff will be the day I have to quit the business. But I’m still very shocked by the things I read.
Look at Pew Research polls on how many people believe in QAnon’s fundamentals. I think we’ve entered a new phase in which social media has altered and distorted the way we encounter information, how we process it, how we internalize what counts as truth. This has significant implications for our democracy.
I truly believe that social media is an accelerator. …
An accelerator of societal disintegration?
Yes, yes, exactly.
You had an interesting Twitter thread about the misinformation you see around the January 6 committee. Can you tell me about it?
On these alternative social media platforms, the January 6 narrative began to be pushed on January 7. People started with saying it was antifa that was responsible. This was amplified by more traditional characters, even Tucker Carlson talked about antifa possibly having a role on January 6th.
Right from the start, they were trying to deflect blame. You had card-carrying members of the MAGA community like [Jan. 6 protester] Ray Epps is falsely accused of being FBI informants and responsible for pushing people into the Capitol. He came out and said, “I was one of them [the pro-Trump movement]and they just sort of backfired on me.
All it takes is a single user on one of these platforms to write something weird without any factual basis or proof. They don’t quote anything, they don’t look at any hardcore information, or they take things out of context. And that is simply magnified endlessly by other users. People who aren’t information-savvy consumers see this on these platforms, and they say, “I’m okay with that. It seems plausible. This is now the truth for me.
If you ask people, “Who was responsible for January 6?” a significant number of people will tell you that antifa played a role on January 6th. Several credible investigations have shown that antifa played no role on January 6. …Yet it keeps a cohesive narrative, and that narrative basically started spinning as soon as people were clear of the Capitol building.
In the past, this is the kind of thing that would have happened on Twitter. But now it starts on small platforms. It may eventually migrate to Twitter. But Twitter and the bigger platforms actually do some moderation of content, which makes it harder for these things to gain traction or be picked up.
These smaller places don’t have the resources or the will to do content moderation. They allow these narratives to fester and gain traction and ultimately blow the hosts away.
Of all the alternative social media apps and sites, which seem to be the most successful? Where is the energy?
It’s still 4chan. …One secret about 4chan is that they actually have to do a significant amount of content moderation now – where they remove posts because of how bad and violent they are. There are a lot of people on 4chan regularly, who are frequent travelers on the forums. It’s still crazy over there.
More than Telegram, an instant messaging service?
Telegram is also huge. Right now we follow thousands of Telegram channels but this is just a drop in the bucket.