Ever wonder why political opinion polls can claim a margin of error of 3% when they survey only 1067 people out of an electorate of, say, 2 million? 1067 is a tiny percentage of 2 million – way less than 1% in fact. So how is this possible? For most people, such a low response rate (in percentage terms) would provoke a sceptical reaction.
Response rates get a lot of attention when it comes to surveys. Whether it’s a business conducting market research or an organisation conducting an online survey to gauge member satisfaction, the focus is often on reaching some target response rate, for example, 40%.
Of course there is merit in this focus on response rates – the higher the response rate, the more representative the data. But response rates are only part of the picture. The margin of error is also very important.
Imagine two online surveys achieve an identical response rate – 20%. The first survey was of 200 people – so it got 40 responses. The second survey was of 1000 people – so it got 200 responses. But guess what? The first survey’s margin of error is 13.89% whereas the second’s is 6.2%.
This has some interesting implications :
- As the size of a population increases, and if the percentage response rate stays the same, the margin of error decreases.
- As the size of a population increases, the required sample size to achieve a given margin of error decreases!
So – be mindful of response rates – but don’t focus exclusively on them!
