(Textbook Page 105)
Sample surveys are the most common form of marketing research and a mechanism for getting trustworthy, projectable answers to marketing questions. Rather than interviewing the entire population (a census), we collect information from some members of the target audience (a sample). However, we need to follow specific procedures if the results are to be valid and reliable.
Strict sampling principles can be applied when every member of the population is equally accessible. Online surveys do not abide by these principles, but it is important to understand them because the validity of any survey is related to the way the sample is chosen.
Conducting a census may be time consuming, expensive (even more than the value of the information obtained), perhaps not even feasible, and would not necessarily be more accurate than a properly conducted sample survey.
1. Define the Relevant Population: The target audience that the findings are to represent must be defined as precisely as possible. It is not always straightforward.
2. Define the Sampling Unit: the smallest individual element you will analyze, be it an individual, a household, an occasion or an organization. It depends on the purpose of the study.
3. Select a Sample Frame: This is how the population is accessed to draw the sample. Ideally it should be as similar to the population as possible. It must be comprehensive (understand who and how many would be left out), precise (exclude everyone who should be excluded) and practical (using a list if one is available).
4. Choose a sampling method:
a. Only when we use probability or random sampling (each unit has an equal chance of being included) can we measure the accuracy of the results. This Unit describes different options, including single versus multi-stage and cluster sampling.
b. Non-probability samples are generally accepted as long as the sample is representative with no biases. This textbook unit describes approaches to judgement, accidental, quota, snowball and adaptive samples.
c. Online samples drawn from good access panels are generally given the same weight as a probability sample. “Panelists” are a group of pre-screened respondents who are willing to complete a defined number of surveys over a set time period. They are profiled when recruited, so demographic and some behavioural questions don’t need to be asked on each survey and very specific respondent definitions can be selected. This textbook unit describes different types of panels and how they are used, and runs through less robust online samples (e.g. river and domain sampling).
5. Setting the Sample Size: The sample size is (along with how the sample is chosen) a determining factor in how close the results will be to what we would have learned from interviewing the whole population (true values). Deciding what is large enough depends on how precise you need the findings to be.
This unit describes two factors directly related to how close our estimates would be to ‘true values’:
a. Non-sampling errors are biases that would persist even if we increased the sample size. They include refusals, language barriers, respondent fatigue, incorrect data collection or sampling procedures, poorly designed questionnaires, etc. They cannot be easy to spot nor easily measured and can affect the validity of the results.
b. Sampling errors result from interviewing only a small proportion of the population. Assuming a probability sample, it can be measured using statistical methods and expressed as a ‘confidence interval’ or ‘margin of error’ (e.g. +/- x points 95 percent of the time). Sample size affects the reliability.
Researchers use two sets of criteria to decide how large a sample should be:
An ‘ideal’ sample size for a particular study is one that is accurate enough for your purposes and large enough to allow sub-group comparisons if they are important to your business question.
Note that we do not need to consider the size of the population.
