From: 11th May 2019
To: 6th June of 2019
Data size: 3.3 GB
Users: 194k unique
On 23 June 2016, the United Kingdom held a referendum, whether the British people prefer to stay in the European Union or leave. In this referendum, the people voted 51.9% supporting leaving the EU. As a result, the Government invoked Article 50 of the Treaty on European Union, starting a two-year process which was due to conclude with the UK’s exit on 29 March 2019. This process is since referred to as “Brexit”, which is used as a shorthand way of saying the UK leaving the EU – merging the words Britain and exit to get Brexit.
In this project, we are going to study how the public’s views towards Brexit developed before, during and after the European Elections of 26 May 2019, an action that the British Parliament wanted to avoid, but had to attend, nevertheless. The public’s sentiment was studied through the views expressed on Twitter, a famous social network, where people are free to express their views in short sentences. For the prediction of the sentiment, the emotion and the geolocation of the public’s views, a set of machine learning algorithms were employed.
On the sentiment information extraction task our focus is to identify the overall sentiment of the tweets and classify them in one of two main categories: negative and positive. The dataset we used to train our models for this task is the Large Movie Review Dataset v1.0.
As we can see from the chart on the left, the BERT model is the best performing model with an accuracy of 0.8720 followed by the Linear SVM and Naïve Bayes models. Random forest was the worst performing model with a significant difference. The voting classifier, which selects the prediction with the majority votes, decreased the accuracy our best model. Although BERT was the top performing model, the fine-tuning phase of the BERT model takes significantly more time than training the other models.
In the bar chart below, we can see the average sentiment score per day and the number of tweets from each side (positive/negative). It’s clear that there is a trend for positive sentiment to dominate everyday except one single day. The 2nd June 2019. That day, we had a waterfall of events starting with a UK poll bringing Nigel Farage and the Brexit Party leading with 26% winning the elections. Following by Donald Trump’s support to Farage and his claim that UK should be ready to the EU without any deal.
It comes as no surprise to see that Brexit is not a local event that concerns the people of Europe alone, but also the rest of world is aware of the situation and the ongoing events. Apart from Europe, USA is very keen on commenting on the events and especially on the dates that their president visited the UK and also offered support on pro-Brexit politicians. In addition, India, Saudia Arabia and Japan were also very active in commenting the news.
Business Solutions Consultant at Gas Distribution Company
Director of Product Development at Innovation Accelerator Foundation
Data Scientist at Hattrick Ltd