#data: how Twitter is tracking social trends

Originally written for The Student, November 2013

Twitter is inevitably a breeding ground for mediocrity. ‘Falling asleep in lectures #EdUni’. ‘Down to my last chocolate Hobnob #sadtimes’. But despite the abundance of tedium, a new generation of data analysts are using the content of our tweets as a window into a diverse range of social or economic trends. The self-important nature of a tweet makes it an ideal tool for monitoring the thought processes of a vast data set. Shifting 147 characters from conscience to keyboard transforms you into a data point. If you have a Twitter handle, you’re part of the study.

Already, the number of studies using tweets as data is extensive. Dengue fever outbreaks have been tracked in Brazil, the ‘happiest’ spots in New York have been mapped and the excitement surrounding the Higgs boson discovery (or ‘Higgsteria’) has been carefully plotted. The UK Met Office has even started using geo-located tweets to track snowfall.

A study of language used in tweets amongst a large African-American population showed that new terms, such as the use of ‘bruh’ instead of ‘bro’ or ‘brother’, are often used in cities first, before spreading more widely. What is more, cities that share the same new language trends tend to be economically or ethnically similar, rather than geographically close to each other. As linguist John Nerbonne puts it, “Birds of a feather tweet together”.

The University of Bristol runs a Media Patterns project, dedicated to analysing media content on a large scale. One aspect of the program is the Flu Detector, which tracks epidemics of influenza-like illnesses across the UK. When Twitter users are sick, they micro-blog about it.

Another section of the Media Patterns project analysed the public mood in the summer of 2011, when riots swept across London and other cities in the UK. An examination of British tweets revealed that fear and anger were reduced slightly in the lead up to the royal wedding in April, but peaked just before the riots began in August. However, Twitter is not just a social science tool. The social network took considerable blame for distributing information that encouraged the riots to spread.

Twitter also came under fire in early 2011 for allowing rebels to spread messages during the uprising that led Hosni Mubarak to resign as President of Egypt, but Ingmar Weber and colleagues at the Qatar Computing Research Institute used Egyptian tweets to track political tension. Tweets containing hashtags like #Bieber or #iPhone were assumed to be politically neutral, but when the hashtag #tamarrod (which roughly translates as ‘rebel’) was used amongst secular Tweeters, this was taken as an indication of polarity. The group found that when political polarity was high on Twitter, real life tension faithfully followed.

This kind of data mapping could prove invaluable for monitoring, or even predicting, political tension. Countermeasures could be introduced by governments in response to a diverging political polarity of a country. Humanitarian Joseph Bock has also advocated Twitter as a “technology of non-violence” as it allows members of non-violent movements to track outbreaks and alert experts in non-violent conflict resolution.

There is arguably a danger that Twitter will remove any privacy we feel entitled to, surrounding our personal details or daily activities. Satirical newspaper The Onion described social networking as “truly a dream come true for the CIA” and it’s not just the government that are watching you; marketing firms regularly monitor reactions to product launches through Twitter or Facebook.

The phenomenal scale of Twitter data makes it a goldmine for social scientists, investigating trends that could promote wellbeing and manage political tension. The invasive nature of Twitter data is what makes it so invaluable. As long as we don’t just tweet about Hobnobs.