Sampling Bias

… a core concept used in Quantitative Methods and Atlas104

Click for OnlineStatBook page

Concept description

Wikipedia (reference below) defines sampling bias as a bias in which a sample is collected in such a way that some members of the intended population are less likely to be included than others.

OnlineStatBook (link on right) notes that:

“It is important to keep in mind that sampling bias refers to the method of sampling, not the sample itself. There is no guarantee that random sampling will result in a sample representative of the population just as not every sample obtained using a biased sampling method will be greatly non-representative of the population.”

OnlineStatBook describes three common types of sampling bias:

Self-selection bias

“A self-selection bias can result when the non-random component occurs after the potential subject has enlisted in the experiment.”

“Imagine that a university newspaper ran an ad asking for students to volunteer for a study in which intimate details of their sex lives would be discussed. Clearly the sample of students who would volunteer for such a study would not be representative of the students at the university. Similarly, an online survey about computer use is likely to attract people more interested in technology than is typical. In both of these examples, people who “self-select” themselves for the experiment are likely to differ in important ways from the population the experimenter wishes to draw conclusions about. Many of the admittedly “non-scientific” polls taken on television or web sites suffer greatly from self-selection bias.”

Undercoverage bias

“A common type of sampling bias is to sample too few observations from a segment of the population. A commonly-cited example of undercoverage is the poll taken by the Literary Digest in 1936 that indicated that Landon would win an election against Roosevelt by a large margin when, in fact, it was Roosevelt who won by a large margin. A common explanation is that poorer people were undercovered because they were less likely to have telephones and that this group was more likely to support Roosevelt.”

Survivorship bias

“Survivorship bias occurs when the observations recorded at the end of the investigation are a non-random set of those present at the beginning of the investigation. The gains in stock funds is an area in which survivorship bias often plays a role. The basic problem is that poorly-performing funds are often either eliminated or merged into other funds. Suppose one considers a sample of stock funds that exist in the present and then calculates the mean 10-year appreciation of those funds. Can these results be validly generalized to other stock funds of the same type? The problem is that the poorly-performing stock funds that are not still in existence (did not survive for 10 years) are not included and therefore there is a bias toward selecting better-performing funds.”

Atlas topic, subject, and course

Research Design (core topic) in Quantitative Methods and Atlas104 Quantitative Methods.


Wikipedia, Sampling bias, at, accessed 11 June 2017.

David Lane, OnlineStatBook, at, accessed 11 June 2017.

Page created by: Ian Clark, last modified 11 June 2017.

Image: David Lane, OnlineStatBook, at, accessed 11 June 2017.