- Polling data move markets, but how reliable are they?
- Respondent age, employment status can sway data
- Bookmakers not necessarily more accurate despite financial interests
With the UK referendum on whether it should stay a member of the European Union fast approaching, there has been a flood of opinion polls hoping to provide some insight into how the Brits will vote next week. But how reliable are they?
The latest YouGov survey released over the weekend showed 43% of voters intended to vote leave the European Union, compared to 42% for Remain with 11% undecided. However, a poll at the end of last week by the ORB placed the leave campaign with 55% of the vote; as opposed to 45% for remain. This week, the Leave campaign had a six point lead over remain at 53 percent to 47% in the latest Guardian/ICM poll.
Poll results are closely watched by investors as well as politicians and can have serious effects on the market. For instance, sterling fell to a two-month low against the dollar on Monday, trading as low as $1.412 before paring some losses, as a result of the weekend's Yougov poll.
One reason for the differing results between polls from different companies comes down to methodology. For example, online surveys often reach very different people compared to polls conducted over the telephone.
"Every method, whether it's telephone, face-to-face or online is trying to do the same thing: it will be trying to represent the wider population. Doing that is pretty difficult; we all have to deal with imperfections in our work", explained Joe Twyman, head of political and social research for EMEA at polling company YouGov in a phone interview with CNBC.
"For instance, in telephone surveys, the response rate is less than 5% and if you don't have a landline you are far less likely to be contacted. [Online surveys] have problems with the fact that fewer people in older age groups are online, although that doesn't meant they aren't there at all."
Another problem for polling data is that in the UK, the industry is having to deal with much more scepticism this year, after many surveys were wrong about how people would vote in the 2015 general election.
"We didn't have a good day at the office," Twyman said in reference to the general election. "But since then, we at YouGov have correctly predicted that Jeremy Corbyn would become Labour leader, despite how everyone said that was impossible. We are getting better… because we have gone back and we have looked at our methods."
Polls that did correctly predict the general election result were the ones that were able to find hard-to-reach people, those who are difficult to contact on the telephone or online, according to Damian Lyons Lowe, founder and CEO of Survation.
"Say you are a 35 year old male and you work a 9-5 job, you are not able to answer your land line because you are at work and you are not able to answer your mobile phone," he told CNBC's Street Signs. "We would try and call that person in the evening, at a time when they might be able to participate in the interview."
Polling data is further complicated by the social media. Data analysts Ontotext released a study last week arguing that there was a clear preference for leaving the EU among Twitter users, based on findings that leave-related hashtags were mentioned 2.6 times more often the remain-related hashtags.
However, Twyman questioned the value of using social media for survey data.
"The reason you get difference from social media polls is social media polls are complete rubbish and are not representative of anything other than the echo chamber of social media," he told CNBC in a phone interview.
"Something conducted on Twitter isn't representative of Twitter users; it's an example of what we call passive polling, where it is a free-for-all and anyone can take part."
Another source for predictions is betting companies. Generally, these companies expect the remain vote to win and have offered much shorter odds for bets on a vote to remain than for a vote to leave. For instance, Reuters reported that bookmaker Betfair had calculated at the weekend a 70 percent probability of a vote to remain (although this was down from a 78% probability earlier in the week).
However, just because bookmakers expect a vote to remain does not mean they are a more reliable measure than polls.
"Why are polls different from the betting markets? Because they are measuring different things; polls are only ever intended as a snapshot of public opinion at the time," said Twyman.
"[Polls] can't be and nor should they be used as a prediction. Betting markets are only interested in that prediction," he added. "At the moment, they are showing overwhelmingly remain… because that's where historical precedent goes."
CNBC is a recognized world leader in business news, providing real-time financial market coverage and business information to more than 340 million homes worldwide. Visit CNBC.com for more financial news and information.