D. Statistical Analysis · Two-Variable Data Analysis and Validity

Residuals and Residual Plots: How Good Is the Fit?

D2.4 D2.5About 70 minutes

Learning goals

I can calculate the residual for each data point and explain what its sign means.
I can draw a residual plot and use its pattern to decide whether a linear model is appropriate.
I can calculate the standard error of the residuals and use it as a typical prediction error.
I can explain why a high r does not guarantee that a linear model is the right model.

Words to know

residual
e = y − ŷ, the vertical distance from a point to the model. Positive above the line, negative below.

residual plot
A graph of the residuals (vertical axis) against x (horizontal axis), with a reference line at 0.

random scatter
Residuals with no visible pattern, which suggests the model has captured the trend.

standard error of the residuals (s_e)
A typical size of a residual: s_e = √[Σe² ÷ (n − 2)].

heteroscedasticity (fan shape)
A pattern in which the residuals spread out more as x increases, so predictions are less precise for some x-values.

A great r, and still the wrong model

A used-car website lists the value of one car model at ages 1 to 8 years. The correlation between age and value is about −0.988: extremely strong. Surely a straight line is the perfect model?

Not quite. In this lesson you will learn to look at what the line misses: the residuals. They often reveal problems that r hides.

Residuals, one point at a time

For each data point, the residual is

e = y − ŷ = actual − predicted

e > 0: the point is above the line (the model under-predicts).
e < 0: the point is below the line (the model over-predicts).
For the least-squares line, the residuals always add to 0 (you proved this in the last lesson's Challenge).

Worked example

Example 1: Residuals for the study-time model

Using ŷ = 3.6x + 50.533 (keep the extra digit for accuracy), find the residual for each student.

For x = 2: ŷ = 3.6(2) + 50.533 = 57.733, so e = 58 − 57.733 ≈ 0.27.
Repeat for each point (see the table below).
Check: 0.27 + 0.67 + 1.47 − 4.13 + 0.67 + 1.07 = 0.02, which is 0 apart from rounding ✓.

The residuals are about 0.27, 0.67, 1.47, −4.13, 0.67 and 1.07. The student who studied 6 h scored about 4.1 points less than the model predicts; everyone else is within 1.5 points.

Hours, x	Mark, y	Predicted ŷ	Residual e = y − ŷ
2	58	57.73	0.27
3	62	61.33	0.67
5	70	68.53	1.47
6	68	72.13	−4.13
8	80	79.33	0.67
9	84	82.93	1.07

Residuals for the study-time model ŷ = 3.6x + 50.533.

Residual plots

A residual plot graphs each residual against its x-value, with a dashed line at e = 0. It is like tilting the scatter plot until the line of best fit is horizontal, then zooming in on the misses.

What to look for:

Random scatter around 0, with no pattern: the linear model is appropriate.
A curve (a U-shape or an arch): the relationship is nonlinear; a straight line systematically misses. Try a curved model.
A fan (residuals spreading out as x grows): predictions are less precise for large x.
One residual far from the rest: a possible outlier (next lesson).

The study-time residuals bounce above and below 0 with no curve or fan. Only one point (x = 6) is far from 0. A linear model is reasonable here.

Three common residual patterns. Only the first one says "a straight line is a good model".

Worked example

Example 2: The car that is not linear

Values ($1000s) of a car model at ages 1 to 8 years: 30, 26, 22.5, 19.5, 17, 15, 13, 11.5. The least-squares line is ŷ = −2.613x + 31.07 (r ≈ −0.988). Find the residuals and decide whether the linear model is appropriate.

Age 1: ŷ = −2.613 + 31.07 ≈ 28.46, e ≈ 30 − 28.46 = 1.54.
Similarly, the residuals for ages 1 to 8 are about 1.54, 0.15, −0.73, −1.12, −1.01, −0.39, 0.22, 1.33.
Pattern: positive at both ends, negative in the middle: a U-shape.
The car loses more value in the early years than later, so the data curve; a line cannot follow it.

Even with r ≈ −0.988, the residual plot shows a clear curve, so a linear model is not the best choice. It would under-predict values for very new and older cars. (An exponential decay model, like the ones in the Grade 11 unit on exponential functions, fits depreciation better.)

Left: the line looks almost perfect. Right: the residuals show the truth: they curve from positive to negative and back to positive.

How big is a typical miss? The standard error

To summarize the size of the residuals in one number, use the standard error of the residuals:

s_e = √[Σe² ÷ (n − 2)]

It works like a standard deviation of the residuals. (We divide by n − 2 because two numbers, a and b, were estimated from the data.) A prediction from the model is typically off by about s_e; most points lie within about 2s_e of the line.

Worked example

Example 3: Standard error for the study model

Find s_e for the study-time model.

Square the residuals: 0.071, 0.444, 2.151, 17.084, 0.444, 1.138 (from e ≈ 0.267, 0.667, 1.467, −4.133, 0.667, 1.067).
Σe² ≈ 21.33.
s_e = √(21.33 ÷ (6 − 2)) = √5.333 ≈ 2.31.

s_e ≈ 2.31 percentage points. A mark predicted by the model is typically off by about 2 or 3 points. The 6-hour student's residual (−4.13) is less than 2s_e ≈ 4.6 in size, so it is unusual but not extreme.

Residuals with technology

Desmos: after y_1 ~ a x_1 + b, click plot next to residuals; Desmos adds a column e₁ you can graph against x₁.
Spreadsheet: in column C compute each ŷ as slope × x + intercept (using the values from SLOPE and INTERCEPT; spreadsheets use the asterisk key for ×), in column D compute =B2-C2, then make a scatter chart of columns A and D.

Always look at the residual plot before reporting a linear model.

Just for fun

A residual plot is the only graph that is happiest when it looks completely boring. Random dots? Perfect. A smile or a frown? Back to the drawing board.

Hands-on activity

Pendulum: does a line fit?

You need: a piece of string about 1.2 m long, a small heavy object to tie on (a set of keys or a metal washer), a doorway or a hook to hang it from (get permission before using a hook or chair), a ruler or tape measure, and a stopwatch (phone).

Tie the object to the string. Hang it so the length from the support to the middle of the object is 20 cm.
Pull it about 10 cm to the side and let go (do not push). Time 10 full swings (there and back) and divide by 10 to get the period.
Repeat for lengths 40, 60, 80 and 100 cm.
Find the least-squares line for period (y) against length (x), then calculate and plot the residuals.
Describe the residual plot. Is a straight line the best model?

What to look for: periods should be about 0.9 s at 20 cm and 2.0 s at 100 cm. r will be very high (above 0.98), but the residuals usually form an arch (negative, positive, negative), because the period actually grows with the square root of the length. Timing 10 swings and dividing reduces your reaction-time error.

Your turn

A model is ŷ = 2.5x + 10. For the point (6, 22), find the residual. Is the point above or below the line?

A model is ŷ = −1.2x + 40. Find the residual for (10, 31).

The residuals for a least-squares model are 2, −1, 3, −4 and e. Find e.

A residual plot shows the residuals going from negative to positive to negative (an arch). What does it tell you?

A residual plot shows residuals close to 0 for small x but spread widely for large x. What does it mean for predictions?

Residuals: 1, −2, 2, −1, 0, −1, 1. Find s_e.

Why can r be very close to ±1 while the residual plot shows a clear curve?

More practice: warm-up

Write the formula for a residual.

A point lies exactly on the line. What is its residual?

What should a good residual plot look like?

If a residual is positive, did the model over-predict or under-predict?

What do the least-squares residuals always add up to?

More practice: on your own

For ŷ = 1.8x + 2.2 and the data (1, 4), (2, 6), (3, 7), (4, 10), (5, 11), find all the residuals and check that they add to 0.

Using the residuals from the previous question, find s_e.

A model for a cellphone's battery life predicts 14 h, with s_e = 0.5 h. Another predicts 14 h with s_e = 3 h. Which prediction is more trustworthy? Explain.

The residuals of a model of house price against floor area fan out for larger houses. Give a reason this might happen.

A student fits a line to the height of a ball thrown in the air against time and gets r = 0.1. The residual plot is an arch. What model should they try?

Challenge

Explain why a residual plot can show a pattern more clearly than the original scatter plot.

For the car data, the linear model predicts a value for age 12 years. Find it, and explain using the residual pattern why it is probably too low.

Show that for the least-squares line, Σ(x − x̄)·e = 0 for the study data residuals 0.267, 0.667, 1.467, −4.133, 0.667, 1.067 (x̄ = 5.5).

Check your understanding

Quiz

A model predicts 72 and the actual value is 65. The residual is:

A residual plot shows random scatter around 0. This suggests:

The residual plot for a model forms a clear U-shape. The best action is to:

Residuals are 3, −1, −2, 2, −2. What is s_e?

A point has a large positive residual. Where is it?

A model has r = 0.99, but its residual plot is an arch. Which statement is true?

For parents and teachers

Big idea (D2.4, D2.5): the residuals (actual minus predicted) show what a model gets wrong. A residual plot with random scatter supports a linear model; a curve or fan means the model needs rethinking, even when r is very high.

How to help: Have your teen explain the car example to you: "the line looked perfect, but the residuals made a U." The pendulum activity is a good one to do together; one person releases, the other times.

Common difficulties:

Calculating predicted minus actual (the sign is then backwards).
Rounding the slope and intercept so much that residuals do not add to 0.
Judging a model by r alone and never drawing the residual plot.
Dividing by n instead of n − 2 for s_e.