D. Statistical Analysis · One-Variable Data AnalysisSamples, Statistics and Sampling Variability
D1.2 D1.4About 65 minutes
Learning goals- I can tell a parameter (describes a population) from a statistic (describes a sample).
- I can explain why different random samples give different values of a statistic.
- I can describe, using a simulation, how the variability of a sample mean or sample proportion decreases as the sample size increases.
- I can calculate the standard error of a sample proportion and of a sample mean.
Words to know
parameter
A number that describes a whole population, such as the population mean μ or the population proportion p.
statistic
A number calculated from a sample, such as the sample mean x̄ or the sample proportion p̂ ("p-hat"). It is used to estimate a parameter.
sample proportion p̂
The fraction of a sample with a certain characteristic: p̂ = (number in the sample with the characteristic) ÷ n.
sampling variability
The natural variation of a statistic from one random sample to another.
sampling distribution
The distribution of the values of a statistic over all possible samples of the same size.
standard error (SE)
The standard deviation of a sampling distribution. For a proportion, SE = √(p(1 − p) ÷ n); for a mean, SE = σ ÷ √n.
Why do polls disagree?
In the same week, two polling companies ask Ontario voters the same question. One reports 41% support for a party; the other reports 37%. Did someone make a mistake? Not necessarily. Each company asked a different random sample of about 1 000 people, and two random samples almost never give exactly the same result.
This lesson is about that natural wobble, called sampling variability. A sample problem in the curriculum (in the Organization of Data for Analysis expectations, C1.2) asks you to use a database such as Census at School to see how the median, mean or a proportion varies across random samples of the same size, and how bigger samples come closer to the population value. Understanding this wobble is the key to the next lesson, margin of error. (How to choose a sample, and the kinds of bias that make samples unrepresentative, are studied in the unit Organization of Data for Analysis.)
Parameters and statistics
- A parameter describes the population. It is usually unknown, because measuring everyone is expensive or impossible. Symbols: μ, σ, p.
- A statistic describes a sample. You calculate it from your data. Symbols: x̄, s, p̂.
We use statistics to estimate parameters: x̄ estimates μ, s estimates σ, and p̂ estimates p. A helpful memory aid: Parameter goes with Population, Statistic goes with Sample.
Worked exampleExample 1: Parameter or statistic?
Identify each number as a parameter or a statistic. (a) Statistics Canada's census counts every household; the median household size it reports is a summary of all of them. (b) A random sample of 400 Grade 12 students in Ontario has a mean sleep time of 7.1 hours on school nights. (c) In a poll of 1 000 voters, 38% support a proposal.
- (a) It describes the whole population counted: a parameter (assuming the census reached every household; in practice some are missed).
- (b) It describes 400 sampled students: a statistic, x̄ = 7.1 h, estimating μ for all Ontario Grade 12 students.
- (c) A statistic, p̂ = 0.38, estimating the proportion p of all voters.
(a) parameter, (b) statistic, (c) statistic.
Watching sampling variability
Imagine a large school where exactly 40% of students (p = 0.40) walk to school. A computer took 50 random samples of 10 students and recorded p̂ for each, then 50 random samples of 100 students. Here is what one run of the simulation gave:
- Samples of 10: p̂ ranged from 0.10 to 0.70. Only 27 of the 50 samples (54%) were within 0.10 of the true 0.40. The standard deviation of the 50 values of p̂ was about 0.166.
- Samples of 100: p̂ ranged from 0.30 to 0.52. 48 of the 50 samples (96%) were within 0.10 of 0.40. The standard deviation was about 0.047.
Both sets of p̂ values are centred near 0.40: random samples do not systematically miss in one direction. But the bigger samples are far more consistent.
The standard error
The standard deviation of a statistic over many samples is called its standard error. For random samples (from a large population):
- Sample proportion: SE(p̂) = √(p(1 − p) ÷ n)
- Sample mean: SE(x̄) = σ ÷ √n
Check with the simulation: √(0.4 × 0.6 ÷ 10) ≈ 0.155 and √(0.4 × 0.6 ÷ 100) ≈ 0.049. The simulated values (0.166 and 0.047) are close, as they should be.
Notice the √n. To cut the standard error in half, you need four times as many people. Precision gets expensive quickly. Also, for large samples, the sampling distributions of p̂ and x̄ are approximately normal, which is exactly what the next lesson uses to build margins of error.
Worked exampleExample 2: Standard error of a proportion
Suppose exactly 25% of the households in a large city have a cat (p = 0.25). For random samples of n = 300 households, find SE(p̂). Then find the sample size needed to make the SE half as large.
- SE = √(0.25 × 0.75 ÷ 300) = √(0.1875 ÷ 300) = √0.000625 = 0.025.
- To halve the SE, multiply n by 4: n = 1 200. Check: √(0.1875 ÷ 1 200) = √0.00015625 = 0.0125 ✓.
SE = 0.025 (2.5 percentage points) for n = 300; a sample of 1 200 halves it to 0.0125.
Worked exampleExample 3: Standard error of a mean
The heights of maple seedlings in a large tree nursery have σ = 6 cm. A worker measures a random sample of 36 seedlings. (a) Find SE(x̄). (b) Using the normal model and the 68–95–99.7 rule, within what distance of the true mean μ do about 95% of such sample means fall?
- (a) SE = σ ÷ √n = 6 ÷ √36 = 6 ÷ 6 = 1 cm.
- (b) Sample means are approximately normal with standard deviation 1 cm, so about 95% fall within 2 × 1 = 2 cm of μ.
SE = 1 cm; about 95% of sample means are within about 2 cm of the true mean. (Individual seedlings vary by much more: 95% are within about 12 cm of μ.)
Size matters more than fraction
For a large population, the precision of a random sample depends on the sample size n, not on what fraction of the population is sampled. A random sample of 1 000 people estimates a proportion for Canada (more than 40 million people) about as precisely as for a city of 1 million. That surprises many people, but the formula √(p(1 − p) ÷ n) has no population size in it.
Just for funSampling is like tasting soup: one well-stirred spoonful tells you about the whole pot, and you don't need a bigger spoon for a bigger pot. (Stirring is the random part. Without it, you might taste only the noodles on top.)
Hands-on activityCoin samples: see the wobble shrink
You need: one coin, paper and a pencil, and about 30 minutes. (A spreadsheet with =IF(RAND()<0.5,1,0) can replace the coin for step 3.)
- Small samples: flip the coin 4 times and record the proportion of heads p̂ (0, 0.25, 0.5, 0.75 or 1). Repeat to get 20 samples.
- Bigger samples: flip 16 times and record p̂. Repeat to get 20 samples (320 flips; share the work with family members).
- Make two dot plots of p̂ on the same scale from 0 to 1. Find the mean and standard deviation of each set.
- Compare with the theory: SE = √(0.5 × 0.5 ÷ 4) = 0.25 and √(0.5 × 0.5 ÷ 16) = 0.125.
What to look for: both dot plots centre near 0.5. The size-16 samples should be about half as spread out, because 16 is 4 times 4. Your standard deviations will not match the theory exactly with only 20 samples; they should be in the right neighbourhood (roughly 0.15 to 0.35 for size 4, and 0.08 to 0.17 for size 16).
Your turn
1Is the symbol μ used for a parameter or a statistic? What about x̄?
μ is a parameter (population mean); x̄ is a statistic (sample mean).
2A survey of 250 randomly chosen Ontario drivers finds that 190 have winter tires on all four wheels. Find p̂.
3Find SE(p̂) when p = 0.5 and n = 100.
√(0.25 ÷ 100) = √0.0025 = 0.05.
4Find SE(x̄) when σ = 15 and n = 25.
5A sample of 100 gives SE = 0.04. About what SE would a sample of 400 give?
Four times the sample size halves the SE: about 0.02.
6Two random samples from the same population give x̄ = 61.2 and x̄ = 59.8. Does this mean one sample was done incorrectly?
No. Different random samples naturally give different statistics. This is sampling variability.
More practice: warm-up
1What symbol is used for a sample proportion?
2Which usually varies more from sample to sample: x̄ from samples of 20 or x̄ from samples of 200?
3Find p̂ if 45 of 150 people say yes.
4What is the standard error of a statistic?
The standard deviation of its sampling distribution.
5How must n change to halve the standard error?
More practice: on your own
1The true proportion of left-handed students in a large board is p = 0.10. Find SE(p̂) for samples of 50 and of 450, and describe the difference.
n = 50: √(0.09 ÷ 50) ≈ 0.042. n = 450: √(0.09 ÷ 450) ≈ 0.014. Nine times the sample size gives one third of the SE (√9 = 3).
2A researcher wants SE(x̄) ≤ 0.5 kg when σ = 4 kg. What is the smallest sample size?
4 ÷ √n ≤ 0.5 → √n ≥ 8 → n ≥ 64.
3Using the Census at School idea: a student takes 10 random samples of 30 students each and finds the sample medians of hours of sleep: 7.5, 8.0, 7.0, 7.5, 8.0, 7.5, 7.0, 8.5, 7.5, 7.5. What is the mean and range of these sample medians, and what would you expect to happen to the range with samples of 300?
Mean = 75 ÷ 10 = 7.5 h; range = 8.5 − 7.0 = 1.5 h. With samples of 300, the sample medians would be much closer together (smaller range), clustering near the population median.
4Why do pollsters rarely use samples of 10 000 people, even though they would be more precise?
The SE shrinks only with √n: going from 1 000 to 10 000 people (10 times the cost) reduces the SE by only a factor of about 3.2. Beyond a point, the extra precision is not worth the cost, and other errors (such as biased wording or people who refuse to answer) matter more.
5A sample of 1 000 people is chosen from volunteers who clicked an online link. Does the standard error formula describe its error well? Explain.
No. The formula assumes a random sample. Volunteers who choose to answer can differ systematically from the population (self-selection bias). A large biased sample is still biased; the SE formula only measures random variation.
Challenge
1Show that for a fixed n, SE(p̂) = √(p(1 − p) ÷ n) is largest when p = 0.5. What is the largest possible SE for n = 400?
p(1 − p) = p − p² = 0.25 − (p − 0.5)². This is largest when (p − 0.5)² = 0, that is, p = 0.5, where p(1 − p) = 0.25. For n = 400: √(0.25 ÷ 400) = 0.025.
2A population has μ = 50 and σ = 10, and is normal. What percent of individual values are above 55? What percent of sample means from samples of n = 25 are above 55?
Individuals: z = (55 − 50) ÷ 10 = 0.5, P(Z > 0.5) = 1 − 0.6915 = 30.85%. Sample means: SE = 10 ÷ 5 = 2, z = 5 ÷ 2 = 2.5, P(Z > 2.5) = 1 − 0.9938 = 0.62%. Averages vary much less than individuals.
3Explain why the sample proportions in the simulation for n = 10 could only take the values 0, 0.1, 0.2, ..., 1, and why this makes the normal model a poor fit for small samples.
With 10 students, the number who walk is a whole number from 0 to 10, so p̂ = count ÷ 10 jumps in steps of 0.1. The sampling distribution is discrete and lumpy (and can be skewed when p is far from 0.5), so a smooth normal curve fits it poorly. As n grows, the steps get finer and the shape becomes close to normal.
Quiz
2. What happens to the standard error of p̂ when the sample size is multiplied by 9?
3. SE(x̄) for σ = 20 and n = 16 is
4. Two random samples from the same population give different sample means. The best explanation is
5. For random samples from a large population, the precision of p̂ depends mainly on
6. Which statement about a large self-selected (volunteer) online sample is true?
Big idea (D1.2, D1.4): a statistic (x̄, p̂) varies from sample to sample. For random samples it centres on the parameter, and its spread, the standard error, shrinks like 1 ÷ √n. Four times the sample size gives half the standard error.
How to help: Do the coin activity together; it is the most convincing way to see that bigger samples are more consistent. When you see two polls that disagree, ask your teen whether the difference could be sampling variability.
Common difficulties:
- Mixing up parameters and statistics (P with Population, S with Sample).
- Thinking that a bigger fraction of the population matters more than a bigger n.
- Believing a huge sample cannot be biased.
- Forgetting the square root: doubling n does not halve the SE.