Thursday 30 June 2016

YouGov deserve credit for showing the weakness in telephone polling methods during the EU Referendum

British pollsters face a crisis of confidence. Witness the Sturgis Report (Jan 2016) on their woeful 2015 General Election performance, highlighting response error, sampling error and even herding between the firms.

However, for the (online) pollsters, EU Referendum performance was much better, particularly from the most progressive firm, YouGov.  They courageously and correctly deconstructed Populus' (telephone) polling before the result.  See their excellent critique here.  This was perhaps the most useful bit of academic analysis published by anybody during the entire campaign, and boldest.  And the final result supported their claims. Telephone polling was collectively wrong, often predicting Remain 10+ points ahead, before mysteriously converging on the online firms/ numbers at the last minute.  More herding methinks. The online pollsters had generally seen the race neck-and-neck.

The divergence heaped more harm the industry but the telephone pollsters should take the blame. Rather pathetically, Populus had released a telephone poll in February with 44% graduates, and 46% in March in its weighted sample.  Clearly their telephoned graduates responded in greater number, yet despite 'education' being the top demographic indicator of EU support, more important than age or class, Populus didn't weigh the interviewed graduates to the national average of 27%-33%, (the real number in the U.K. from the 2011 Census and adult population surveys.)  This showed how fallible polling can be to expertly devised weightings and assumptions

The disparity between telephone and online polling was the most pressing quandary for political science during the Referendum.  But no one (to my knowledge) adjudicated between YouGov and Populus, bar Matt Singh who gave weak support for telephone polling, proving all winning streaks come to an end one day.  In truth, the likes of Mike Smithson and co, weren't prepared to stick their necks out on something they struggled to reach a conclusion on.  

Very few got to hear about this major fallout between YouGov and Andrew Cooper of Populus, the retained pollster for Remain, and his new side-kick, Peter Kellner (pictured).




Remarkably, Kellner was President of YouGov until just two months previously, but now began railing against the whole online polling model that is championed by YouGov.  Stunning broadsides flew at YouGov methodology from the ex YouGov figurehead.  Ultimately, YouGov won this tiff on the basis of the final result. They have the best staff (like Joe Twyman, Antony Wells, Stephan Shakespeare and many others) and the strongest reputation - only enhanced by the EU Referendum.  Kellner's status meanwhile, despite his undoubted eloquence, continues to nose-dive.