Twitter on Tuesday announced a significant shift in the type of data it makes available for free to third-party academic researchers interested in studying user behaviors and trends related to online discourse. Now, Twitter says it will no longer require researchers pay for premium or enterprise developer access and will instead make the “full history of public conversation” — what the company refers to as its full-archive search endpoint — available to any researcher or developer who applies as part of the launch of a new academic research track.
The change is part of the company’s ongoing efforts to improve the Twitter API, the set of tools it makes available for those outside the company to build something on top of Twitter or use its mountains of data for research studies. Twitter launched its new and improved API last summer as a way to make amends with the developer and research communities, both of which have often found themselves at odds with the platform’s continuous and sometimes adversarial product development over the years. This contentious relationship began most notably in 2012, when Twitter decided to cut off key developer access to the platform to maintain stricter control over what data it let potential competitors use.
Twitter acknowledges the less-than-ideal relationship it’s had with the developer and research communities. “Our developer platform hasn’t always made it easy for researchers to access the data they need, and many have had to rely on their own resourcefulness to find the right information. Despite this, for over a decade, academic researchers have used Twitter data for discoveries and innovations that help make the world a better place,” write product managers Adam Tornes and Leanne Trujillo in a blog post published today.
Tornes and Trujillo point to steps Twitter has taken in the past year to improve its tools, including the release of a COVID-19 endpoint for free and a beta of the new academic research track to gather proper feedback ahead of today’s public launch. And the company is making a concerted effort to convince researchers it wants to help them, especially now that social media discourse has become more crucial in understanding online misinformation, election interference, hate speech, and other topics that took center stage during and after the 2020 US election.
In addition to opening up its public archive, Twitter says it’s also giving approved applicants a higher monthly tweet volume cap of 10 million tweets, which is 20 times higher than what was available on the standard free track before. It’s also allowing more precise filtering to help researchers pinpoint tweets and other data relevant to what they’re studying and “new technical and methodological guides” for helping researchers find what they’re looking for and better make use of it in studies.
Twitter says interested academics and developers can apply to the new academic research track on the company’s developer website. The track will, however, come with some limitations. For now, the company is now opening up access to independent researchers or journalists. You’ll have to be a student or part of an academic institution. Twitter also says it will be not be providing access to data from accounts that have been suspended or banned, which could complicate efforts to study hate speech, misinformation, and other types of conversations that violate Twitter rules.
This also means that the account @realDonaldTrump is also not accessible through Twitter’s archive following US social media sites’ widespread deplatforming of Trump following the deadly US Capitol attack. Still, Trump’s tweets are archived online through his official presidential library, while independent websites and database tools like The Trump Twitter Archive and ProPublica’s Politwoops have also been preserving the former president’s tweets, included the deleted ones.
“We’ve heard a lot of interest from the academic research community in studying @realDonaldTrump,” Trujillo said during a call with reporters on Monday. “We’re having conversations internally about how we might give thoughtful consideration to the study of this topic.”
Update January 26th, 3:47PM ET: Added more information around access to suspended or banned accounts and former President Donald Trump’s tweets.
