By Graunt Kruger
Doing research has never been easier. More and more research, especially surveys, is being done online. What does that mean for their accuracy and reliability? Can we trust the results?
When we think of surveys, we default to the idea of random sampling. It means that each member of the population has a similar chance of being selected into the study. Random sampling is also known as basic probability sampling. The best way to think about this is imagining that all names are written on strips of paper and put into a hat. The names are drawn one by one. Each name has an equal chance of being drawn. Of course, in a full-scale study the names in a hat are substituted for a computer program that randomly selects any of the names in a database.
The value of random sampling is that it allows the researcher to control for bias and enables the researcher to make inferences about the population. This means that the findings of a study based on random sampling, with a sample large enough to be representative of the population, can be confidently said to be the views of the population. This is often why well-designed research is so powerful.
Random sampling relies on a list of some sort that forms the basis of the process and it assumes that all selected respondents will answer the survey. But, as is often the case with online research, there might not be a master list. Then it isn’t possible to draw a random sample. This kind of research is known as non-probability research and researchers do not know the chance or probability that each person has of being selected.
There are different kinds of non-probability sampling:
Convenience sampling: due to constraints, the researcher chooses to conduct research at times or locations that are convenient/
Snowball sampling: one person refers another to the researcher, these subjects know each other and they could share certain views or beliefs.
Purposive sampling: the researcher intentionally chooses who is in the study and who is excluded.
Voluntary sampling: respondents self-select themselves to participate in the study; they are volunteering because of some underlying reason.
The challenges with these methods is that unintentional bias can be created and they may provide misleading information. However, non-probability research does have some advantages. It has lower costs, is much faster to execute and can be used to study groups that are too small to even do a probability sample. This method is useful for exploratory studies that intend to uncover whether a problem actually exists that warrants further study.
Does it mean that studies using non-probability methods are less scientific, less reliable, less accurate and less trustworthy? The short answer is that it depends on the objective of the research. If, for example, the research objective is to draw inferences about the entire population, then reader beware. The findings of non-probability studies cannot be used to infer the attitudes of the entire population.
If, however, the objective is to understand the views of a specific group – for example fathers with children in day care – then snowballing can be very effective to reach that group. It is important to understand though that the results of the research cannot be used to infer that the views of the respondents in this kind of study represent the views of the population. The findings can however be used to understand and describe the views held by the respondents.
Certainly the ease and reach of using the internet – especially the mobile internet in the South African context – has become a major enabler for doing social science research. These technologies did not invent non-probability methods but they have made them more pervasive. The onus is on researchers to ensure that they understand that the findings, implications and recommendations that are drawn from nonprobability studies are not the same as those from studies with random samples.
Sources:
Bhattacherjee, A. (2012). Social science research: Principles, methods, and practices.</em><em>Kalton, G. (1983).
Quantitative Applications in the Social Sciences:Introduction to survey sampling.
Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412984683.
