OVER the last few weeks we have been inundated with headlines based on the results of polls conducted by an organisation called YouGov. These suggest Nigel Farage’s Reform UK party winning by a landslide if an election were to be called, that almost half of the 111,000 Brits surveyed support introduction of Digital ID even as more than 2.6million (and counting) have signed a petition against it, that almost two-thirds of the 7,526 adults polled believe the two-child cap on social welfare benefits should remain, and that in spite of journalists and commentators telling us every day what they believe US President Donald Trump thinks on issues such as Ukraine and Russia, YouGov respondents have little clue about the President’s opinions. In this series of articles we are going to look at YouGov: its scientific processes, its respondents, and some of its recent surveys.
YouGov is an organisation similar to NielsenIQ, Morgan-Gallup and Ipsos Mori in that it undertakes opinion polls, or surveys, of a sample of citizens. While some, such as those on digital ID and voter intention, can be classified as public interest surveys, others are conducted on behalf of commercial entities seeking to understand brand preferences and buyer intention. However, in the case of YouGov, the first issue to note is that the YouGov name creates some degree of confusion regarding who operates the service. Some people wrongly assume it is part of the government.
How YouGov works
While some pollsters operate by approaching people in the street or by phone, YouGov recruits volunteers to participate in surveys and/or comprehensive data collection in return for the promise of significant-sounding cash payments. YouGov describes the sign-up process as collecting a ‘host of socio-demographic information’, yet my own experience when signing up under an assumed name saw it collect only a year of birth, gender, postcode and email address. This was confirmed by the descriptions of YouGov evangelists, like this one.
It highlights one potentially glaring issue with any of the possibly hundreds of thousands or millions of survey-only YouGov participants: that their identity could be completely fictitious and we should have little if any confidence in the veracity of their answers.
YouGov claims to have 2million British respondents, whom they like to call ‘panel members’. I believe this labelling is intended to give the impression that an empanelment, like a jury in a criminal or civil court, is created for each survey.
Respondents are paid for answering questions in ‘points’, with one point valued at £0.01. Points can be redeemed in minimum £50 lots either for cash (paid into your bank) or shopping vouchers.
They are paid extra for sharing their data. This includes giving YouGov unlimited access to their banking, credit card and other financial data, YouTube, Netflix, AppleTV, Disney+ and Amazon viewing data, and internet usage via a VPN app called YouGov Pulse.
The amounts panellists receive for revealing their private information to YouGov include:
Thus, YouGov may receive a considerable amount of data from the respondent before any money changes hands.
(The sign-up process gives the potentially misleading impression that every question you answer is worth five points. I quickly found this was not the case. In one survey they offered about three hours after sign-up, I was offered 20 points to complete eight questions, half of the per-question rate shown at sign-up.)
YouGov sponsors, runs, or in some indirect way supports online groups that promote its service and surveys. These include the Reddit group r/beermoneyuk (here and here) that among other paid survey organisations (including Sermo, Qmee, Prolific, IPSOS iSay) promote YouGov surveys and offerings in spite of many members decrying both the invasion of privacy that can come with participation in YouGov, and the infrequency of payouts when compared to other services.
YouGov participant inclusion process
YouGov and their academic partners (including Reuters Institute and the University of Oxford) tell us all their surveys are conducted by ‘active sampling’, whereby they select participants through some process of matching the survey client, survey goals or participant requirements to relevant panel members.
Who are the respondents?
What becomes abundantly clear is that YouGov participation is both privacy invasive and requires anything from minor (five to ten minutes and ten points for an add-on survey with one to three questions) to significant time and effort (15 to 30 minutes for a normal survey that will earn you 20-50 points), depending on how fast they want to earn their £50 payments. Even YouGov evangelists admit that it will take regular and active people 12-18 months, and most people much longer, to earn that £50 payout.
I contend from reviewing many hundreds of pages of literature, commentaries, discussion groups and academic papers that there are predominately two types of participants in the respondent pool.
The first type, the majority, include the unemployed and people on public welfare benefits, pensioners and those with low income. These people have the time and motivation to participate because an additional £50 every three to 12 months is significant.
The second type of respondent are professional survey-takers, who create multiple online accounts and/or personas to maximise their potential to receive survey requests and payouts. This is their full-time and possibly more than 12-hours-per-day job.
This would suggest that there is a significant and undisclosed gap in the pool, with professional voices such as doctors, lawyers, company executives and senior public servants very likely entirely unrepresented.
What we need to understand, but what YouGov carefully guards as secret, is the hard demographics of respondents in each of their published surveys. You will often see that YouGov rarely gives us any demographics data at all, or that what it provides is limited to one type such as percentages of respondents in graduated age groups or some other singular demographic where inclusion is intended to do little more than imply that their method was rigorous and scientific and results are reliable.
I will cover more regarding my analysis of YouGov respondent demographics in a future article; suffice to say I think that if YouGov was required to provide an honest accounting of demographics data for every survey, we would begin to see that low-income people, pensioners and the unemployed make up the vast majority of their respondent pool, which is not a truly representative sample of any nation.
Overall, my general position on YouGov and similar sites aligns with these posters in the r/beermoneyuk Reddit group:
