Practise › Questions › Least squares regression and residuals
Least squares regression and residuals questions
A line of best fit chosen by a rule instead of by eye. The rule is to make the vertical gaps as small as possible in total, and the gaps left over tell you whether a line was the right idea.
7 original questions · 26 marks · the least squares regression and residuals notes · Further Statistics 2
Every question here is written for this library rather than taken from a past paper. Write your answer out before opening the worked one: the answers award marks point by point, and the marks are easier to see when you have something of your own to compare against.
Define a residual, and state the quantity the least squares line makes as small as possible.
Worked answer
A residual is the observed y minus the y the line predicts at the same x: a vertical gap, not a perpendicular distance. The least squares line minimises the sum of the squares of those residuals over all the data points. B1 for the residual as a vertical gap, B1 for the sum of squared residuals.For the points (2, 5), (4, 9), (6, 10), (8, 14), (10, 17), find the regression line of y on x.
Worked answer
Σx = 30, Σy = 55, Σx² = 220 and Σxy = 388, so Sxx = 220 − 900/5 = 40 and Sxy = 388 − 330 = 58. Then b = 58/40 = 1.45 and a = 11 − 1.45 × 6 = 2.3, giving y = 2.3 + 1.45x. M1 for Sxx and Sxy, A1 for the gradient, A1 for the intercept.The regression line of y on x for the points (2, 5), (4, 9), (6, 10), (8, 14), (10, 17) is y = 2.3 + 1.45x. Find the five residuals and the residual sum of squares, checking the total two ways.
Worked answer
Fitted values are 5.2, 8.1, 11.0, 13.9 and 16.8, so the residuals are −0.2, 0.9, −1.0, 0.1 and 0.2. They sum to zero, as they must, which is a free check on the arithmetic. Squaring and adding: 0.04 + 0.81 + 1.00 + 0.01 + 0.04 = 1.9.
By formula, Syy = 691 − 605 = 86, so the residual sum of squares is Syy − Sxy²/Sxx = 86 − 58²/40 = 86 − 84.1 = 1.9. M1 for the fitted values, A1 for the five residuals, M1 for squaring and adding, A1 for the residual sum of squares.A test score y is recorded against x hours of practice for five students, giving (2, 5), (4, 9), (6, 10), (8, 14) and (10, 17). The regression line is y = 2.3 + 1.45x. Interpret both coefficients, predict the score for 7 hours, and say why a prediction for 20 hours would be unwise.
Worked answer
The gradient says that each extra hour of practice is associated with about 1.45 more marks. The intercept, 2.3 marks with no practice at all, comes from x = 0, which lies outside the observed range of 2 to 10 hours, so it should be quoted with care.
At x = 7 the prediction is 2.3 + 10.15 = 12.45 marks.
At x = 20 the model is being extrapolated to double the largest observation. There is no evidence that the relationship stays linear that far out, and in this context it cannot, since scores are bounded. B1 for interpreting the gradient, B1 for interpreting the intercept, B1 for the prediction, B1 for the extrapolation comment.Describe what a plot of residuals against x should look like if a straight-line model is appropriate, and what two patterns would argue against it.
Worked answer
It should show random scatter about zero with no trend and roughly constant spread. Systematic curvature, so that residuals are negative at the ends and positive in the middle or the reverse, means the relationship is not linear. A spread that widens with x means the assumption of constant variability fails. A single very large residual instead flags a possible outlier. B1 for random scatter about zero, B1 for curvature ruling out a linear model, B1 for a widening spread.Five points are recorded as (2, 5), (4, 9), (6, 10), (8, 14) and (10, 17), and the regression line of y on x is y = 2.3 + 1.45x. The last point is then found to have been misrecorded, and should read (10, 30). Recalculate the regression line and comment on the effect.
Worked answer
Σy becomes 68 and Σxy becomes 518, so Sxy = 518 − 30 × 68/5 = 518 − 408 = 110, while Sxx is unchanged at 40 because the x values have not moved.
Then b = 110/40 = 2.75 and a = 13.6 − 2.75 × 6 = −2.9, giving y = −2.9 + 2.75x.
One faulty reading has nearly doubled the gradient and pushed the intercept below zero. The least-squares method squares the residuals, so a single large miss counts for far more than several small ones. M1 for recomputing the sums, A1 for the new Sxy, A1 for the new line, B1 for the comment on the effect. It is the reason for plotting the residuals before trusting a fit.Five pairs of readings are coded using p = (x − 6)/2 and q = y − 10, giving the coded points (−2, −5), (−1, −1), (0, 0), (1, 4) and (2, 7). Find the regression line of q on p, and hence find the regression line of y on x.
Worked answer
Σp = 0, Σq = 5, Σp² = 10 and Σpq = 29. Since Σp = 0 the correction terms vanish: Spp = 10 and Spq = 29.
So the coded gradient is 29/10 = 2.9, and with p̄ = 0 and q̄ = 1 the coded intercept is 1. The coded line is q = 1 + 2.9p.
Now decode. Substitute q = y − 10 and p = (x − 6)/2 into that line: y − 10 = 1 + 2.9(x − 6)/2 = 1 + 1.45(x − 6) = 1.45x − 7.7.
So y = 2.3 + 1.45x.
M1 for the coded sums, A1 for Spp and Spq, M1 for the coded gradient and intercept, A1 for the coded line, M1 for decoding, A1 for the line of y on x.
Read what the coding did. Dividing x by 2 multiplied the gradient by 2, so decoding halves it back to 1.45. Subtracting 10 from y and 6 from x moved the intercept, and only substituting back recovers it. Coding is chosen to make Σp = 0 and the numbers small, and it never changes the fitted line, only the arithmetic needed to reach it.
The same practice on paper: the printable workbook for this topic, questions and a worked answer book.
Practise least squares regression and residuals one question at a time
The player marks nothing for you. It shows one question, waits, then shows the worked answer so you can mark yourself, and brings a question back sooner when it went badly.