Media intelligence company BrandsEye started analysing the US election process shortly after calling the outcome of the Brexit referendum. It started collecting social media data, particularly from the Twitter network, in July 2016. While early opinions do not greatly influence actual voting behaviour, they do help to fine tune the data collection and tagging. In particular, given the Electoral College system of determining the US president, it was critical to focus on the key battleground or swing states, where the outcome of the election would be decided.
Over the next few months, a large amount of data was collected, geolocated and analysed for sentiment. All social media matching the candidates’ names and mention of the US election itself was collected. Data from outside the US was excluded. In total, over 37.6 million conversations from over four million authors were collected for analysis. The company’s geolocation algorithms identified in which state the author was located and finally the conversations from the key battleground states were sent to the BrandsEye Crowd – an integrated, scaled crowdsourcing platform trained to accurately and quickly understand sentiment.
Despite much publicised recent advances in AI (artificial intelligence), understanding human sentiment accurately remains beyond the reach of machines. In particular, social media is rife with sarcasm, implied and hidden meanings, abbreviated phrases, swearing, innuendo and a host of other complications. Reading humour is certainly beyond the reach of an algorithm. As a result, BrandsEye adopted a crowdsourcing approach in which contributors around the world tackled micro-tasks of correctly assigning sentiment to the candidates (for or against) in return for micropayments - an approach which leads to accuracy levels in the upper nineties.
Once the sentiment had been correctly assigned in all 11 of the key battleground states, the methodology the company adopted was logical and simple. Advocacy for a candidate was counted from all unique authors supporting them or speaking against their opponent. Neutral conversation was excluded. This advocacy was then divided by the total emotive (for or against) conversation to determine a percentage. In each of the key battleground states, if a candidate’s percentage was over 50%, BrandsEye saw the candidate winning that state.
This simple methodology also gave greater weighting to the most recent conversations. If what people say determines how they will act, then it stands to reason that what they say most recently will reflect most closely how they will act in the moment. This is especially true if the comments are unsolicited and spontaneous as is the case in social media. So, BrandsEye collected and analysed data right up to, and including, the day before Election Day, and weighted the data according to its imminence to Election Day itself. This was the final piece to the puzzle, and BrandsEye then called the outcome of the 11 battleground states that would determine the 45th US President.
The results speak for themselves. In contrast to traditional polling, BrandsEye’s weighted average matched the outcome of nine of the 11 states. As with Brexit, accurate social media analysis had once again proven to be the best way to understand the voice of the people. That voice is a human voice and use of trained humans to understand the sentiment of millions of citizens efficiently and effectively was the key to unlocking how they acted.
As the world becomes more connected, differences between decision makers and their stakeholders are becoming more visible and volatile than ever. Traditional methods of understanding a broad group of people are breaking down, because they can measure neither the intensity nor the commitment of the emotions. Sophisticated analysis of social media, however, offers a more reliable understanding of what is happening in today’s world.
