Lesson 3 of 6 · Sample size and selection bias
Separate the sample from the population
8 minute lesson · 3 checks along the way
Your goal for this lessonEvaluate what a survey can tell you about objects it tends to miss.
In this lesson 3 sections · 3 checks
Using archive links & definitions
Words with a dotted underline open a short definition right here in the lesson.
Read & understand
A catalog is produced by an observing process
A catalog contains objects that were observed, detected and accepted under particular rules. Those rules can favor some kinds of object. The observed distribution therefore combines the underlying population with the survey’s geometry, sensitivity, timing and selection decisions.
Transiting exoplanet surveys illustrate this problem. A planet must pass across its star from our viewpoint, and its signal must be detectable in the available observations. Research by Kipping and Sandford analyzes geometric and detection biases. The detected sample cannot simply be treated as a random list of every planet that exists.
Check 1 of 3
10 pointsUse the reading above. You can try again without losing points.
Read & understand
Work through unequal detection
Imagine a fictional population containing 100 objects of type A and 100 of type B. A survey detects 80% of A and 20% of B. The expected detections are 80 and 20, so A makes up 80% of the detected sample despite being half the underlying population.
This example uses expected counts and known detection fractions for clarity. A real correction needs evidence for those fractions and their uncertainty. Dividing counts by an assumed efficiency can produce a precise-looking answer without making the assumption correct.
A survey changes the observed proportions
Original teaching diagram. Fictional population. 100 A and 100 B. A is 50% of population. Expected detections. 80 A and 20 B. A is 80% of detections. Known detection fractions are assumed for this exercise; real corrections need evidence.
Open full-size diagram in a new tab: A survey changes the observed proportionsCheck 2 of 3
10 pointsUse the reading above. You can try again without losing points.
Read & understand
More of the same may preserve bias
Increasing a survey’s size can reduce random sampling variation while leaving the selection process unchanged. A much larger biased sample is not automatically representative. Ask which objects were eligible, which were observable, which passed quality rules and which ended up in the final analysis.
For a fictional follow-up, a second instrument can detect the previously difficult type more efficiently. Combining results still requires attention to overlapping objects and differing selection rules. The goal is not simply to add catalog rows, but to infer the intended population with a documented account of how the evidence was gathered.
Check 3 of 3
10 pointsUse the reading above. You can try again without losing points.
Put it together
Your lesson progress
Each check connects to a part of the story. Complete all 3 to earn your 25-point lesson bonus.
Explore further · Optional
Go to the source
Explore the evidence behind this lesson. These optional readings and videos open in a new tab.
- Read: Kipping and Sandford: observational biases for transiting planetsPrimary research on geometric and detection selection; the lesson’s counting exercise is fictional, not a result from this paper.
- Read: NIST: confidence limits for a meanCompare the explanation and its assumptions with the worked example in this lesson.