Maths › Further Statistics 1 › Goodness-of-fit tests
Goodness-of-fit tests
Put a model's predicted frequencies beside the observed ones, square the gaps, and one statistic decides whether the difference is more than chance would produce.
Builds on Hypothesis tests for Poisson and geometric models and Representing and interpreting data.
IN THIS TOPIC
- Compute expected frequencies from a proposed model and form the χ² statistic.
- Count degrees of freedom, subtracting one for each estimated parameter.
- Pool classes with expected frequency below 5 and recount the classes.
- Fit and test a Poisson or binomial model from start to finish.
COMMON MISCONCEPTION
A large chi-squared statistic proves the model is wrong; a small one proves it is right.
Measuring the gaps
Multiply each class's model probability by the total to get its expected frequency, then accumulate:
The booklet prints the statistic under Non-parametric tests, for goodness-of-fit and contingency tables alike, and the percentage points of χ² are tabulated a few pages later. Squaring makes over- and under-shoots count alike. Dividing by E stops a gap of 3 in a class expecting 5 from being drowned by the same gap in a class expecting 500. A large value means the model fits badly. The test never proves anything either way, since it only compares the statistic with a critical value and reports evidence.
WORKED EXAMPLE
Is the die fair?
A die is thrown 60 times, giving 8, 10, 14, 7, 13, 8. Test at the 5% level whether it is fair.
H₀: the die is fair, so each expected frequency is 10. H₁: the die is not fair.
χ² = (4 + 0 + 16 + 9 + 9 + 4)/10 = 4.2.
Degrees of freedom: 6 classes − 1 = 5, and the 5% critical value is 11.07.
4.2 < 11.07, so do not reject H₀. There is insufficient evidence at the 5% level that the die is unfair.
Counting the degrees of freedom
Start with the number of classes. Subtract 1, because the frequencies must total the sample size. Then subtract 1 more for every parameter estimated from the data. Fitting a Poisson model with λ taken from the sample mean costs a degree of freedom. Fitting one with λ specified in advance does not. A binomial with both n and p given costs nothing extra; a binomial with p estimated costs one.
Small expected frequencies distort the statistic, so classes with Ei below 5 are pooled with their neighbours before anything is computed. The class count used for the degrees of freedom is always the pooled one, and forgetting that is the commonest way to lose the final marks.
GUIDED PRACTICE
Degrees of freedom under two regimes
A Poisson model is fitted to data in 7 classes after pooling. Give the degrees of freedom when λ is specified in advance, and when λ is estimated from the sample mean.
Show the working
Specified in advance: 7 − 1 = 6 degrees of freedom.
Estimated from the data: 7 − 1 − 1 = 5.
Estimating a parameter lets the model bend towards the data, so one degree of freedom is surrendered to pay for the flexibility.
A Poisson fit from start to finish
The full version of this question gives you a frequency table, makes you estimate λ, build the expected frequencies, pool the tail, and only then test. Work down the table in one pass and keep the expected values to two decimal places until the very end.
Read the last row of the observed table before totalling anything. A table that stops at a stated value, four flaws here, records exactly how many events there were, so the sample mean is available and the fitted λ is that mean. A table whose last row reads '4 or more' does not. Counting each of those observations as a bare 4 gives a lower bound for the total and a lower bound for λ, never the mean, and every expected frequency built on it inherits the bias. Such a question has to hand you the total or λ itself. The model's final class is 'x or more' either way, because probability leaks past the largest value seen and the expected frequencies must total n.
WORKED EXAMPLE
Flaws in a batch of components
The number of flaws in each of 100 components is recorded as 30 with none, 36 with one, 21 with two, 10 with three and 3 with four. Test at the 5% level whether a Poisson model fits.
The sample mean is 120/100 = 1.2, so fit Po(1.2). H₀: a Poisson distribution is a suitable model. H₁: it is not.
Expected frequencies from Po(1.2), with the last class taken as '4 or more': 30.12, 36.14, 21.69, 8.67, 3.38.
The last expects 3.38, below 5, so pool it with the class before. That leaves four classes: 0, 1, 2 and '3 or more', with observed 30, 36, 21, 13 against expected 30.12, 36.14, 21.69, 12.05.
χ² = 0.0005 + 0.0006 + 0.0217 + 0.0747 = 0.097.
λ was estimated, so the degrees of freedom are 4 − 1 − 1 = 2, and the 5% critical value is 5.991.
0.097 < 5.991, so do not reject H₀. There is insufficient evidence at the 5% level against a Poisson model for the flaws.
ASSESSMENT FOCUS
- Compute expected frequencies to at least two decimal places before squaring anything.
- Pool classes with expected frequency under 5 first, then count classes for the degrees of freedom.
- Subtract one degree of freedom for every parameter estimated from the data, and name which parameter.
- For a geometric or Poisson fit, the final class is 'x or more' and its probability is 1 minus everything before it.
- Conclude with evidence language. 'Insufficient evidence to reject' is the phrase; 'the model is correct' is not.
CHECK YOURSELF
A goodness-of-fit test uses 5 classes after pooling, with the distribution's single parameter estimated from the data. State the degrees of freedom.
Show a hint
Classes minus one, minus one per estimated parameter.
Show the answer
5 − 1 − 1 = 3 degrees of freedom.
χ² sums (O − E)²/E, so each squared gap is weighted down by how large its class was expected to be.
Degrees of freedom are classes minus 1, minus one more for each parameter estimated from the data, counted after pooling.
WORKBOOK
Printable practice for this topic: original exam-style questions with room to work, and a fully worked answer book. Free to use; please do not redistribute or sell.
Or read them with their worked answers on the goodness-of-fit tests questions page.
CHECK YOUR PROGRESS
Rate how confident you feel with each objective for this lesson. Ratings are saved in this browser, on this device, unless you sign in.
- Compute expected frequencies from a proposed model and form the χ² statistic.
- Count degrees of freedom, subtracting one for each estimated parameter.
- Pool classes with expected frequency below 5 and recount the classes.
- Fit and test a Poisson or binomial model from start to finish.
Open the full revision checklist to see every objective in the course in one place.