Biology › Scientific method and quantitative biology › Statistics for biologists: spread, error bars and the three tests
Statistics for biologists: spread, error bars and the three tests
Biological material varies, so a difference between two means is never self-evidently a difference. A statistical test is the tool that asks how easily the variation you already have could have produced the difference you are excited about.
Before this Calculating a mean · Drawing and reading a scatter graph
Before you start
If the error bars on two means do not overlap, the difference between them is significant. This is the most confidently repeated wrong thing in A-level biology. Non-overlapping bars are a hint and nothing more, and how good a hint depends entirely on what the bars were drawn from: bars of one standard deviation, of one standard error and of a 95% confidence interval all have different lengths on the same data and support different claims. Two means whose standard-error bars are separated by a hair can fail a t-test, and two whose standard-deviation bars overlap can pass one comfortably. Only the test decides.
What you should be able to do
- Choose between a mean, a median and a mode for a given set of biological data, and say why.
- Calculate a standard deviation and explain what it tells you that a range does not.
- Interpret error bars, including what non-overlapping bars do and do not prove.
- State a null hypothesis in the form a mark scheme accepts.
- Choose between chi-squared, a t-test and Spearman's rank for a given question, and find the degrees of freedom.
- Compare a calculated value with a critical value and write the conclusion in the wording examiners want.
Mean, median, and the reason a range is not enough
The first job of a statistic is to stand in for a whole set of readings, and which one you choose depends on what those readings look like.
- Mean
- The total divided by the number of readings. Uses every value, and is what almost every biological calculation wants.
- Median
- The middle value when the readings are put in order. Unmoved by a single extreme value, which is why it is used for skewed data such as incomes, reaction times or the number of parasites per host.
- Mode
- The most common value. The only one of the three that works for categories such as blood group or flower colour.
One number is never enough, because two sets of readings with the same mean can be entirely different sets. What goes alongside it is a measure of spread, and the obvious candidate — the range — turns out to be a poor one.
The range uses two numbers and discards the rest, so it cannot tell readings that cluster tightly with two outliers from readings spread evenly. It also gets worse as you collect more data, since more readings means a better chance of catching an extreme one, and a statistic that depends on how long you spent measuring is not describing the organism. The standard deviation fixes both problems: it is roughly the typical distance of a reading from the mean, and every reading contributes to it.
s = √[ Σ(x − x̄)² ÷ (n − 1) ]sum the squared deviations from the mean, divide by one less than the number of readings, take the square root
The n − 1 is not a typo. You are using the sample's own mean to measure the sample's spread, which makes the deviations slightly too small, and dividing by n − 1 corrects for it. Every A-level formula sheet uses n − 1, and so does a calculator set to sample rather than population.
A standard deviation from nine readings
Sample A consists of the nine readings 14, 19, 19, 20, 20, 20, 21, 21 and 26. Sample B consists of 14, 15, 17, 19, 20, 21, 23, 25 and 26. Both sets are idealised, chosen so that the arithmetic stays visible. Calculate the mean and standard deviation of each and comment.
Both totals are 180, so both means are 180 ÷ 9 = 20; both medians are the fifth value in order, which is 20; both ranges are 26 − 14 = 12. On every statistic so far the samples are identical.
For sample A the deviations from the mean are −6, −1, −1, 0, 0, 0, +1, +1 and +6, whose squares total 76. Then 76 ÷ 8 = 9.5, and √9.5 = 3.08.
For sample B the deviations are −6, −5, −3, −1, 0, +1, +3, +5 and +6, whose squares total 142. Then 142 ÷ 8 = 17.75, and √17.75 = 4.21.
Sample B is about a third more variable than sample A, and nothing except the standard deviation said so. Squaring does the work: it makes every deviation positive so they cannot cancel, and it weights a reading six from the mean thirty-six times as heavily as one a single unit away.
Error bars, and the claim they do not support
An error bar shows how much the readings behind a mean varied. It turns a column that looks like a fact into a column with a width, which is the honest way to draw a biological measurement.
The first question about any error bar is what it was drawn from, because three things are commonly used and they mean different things:
| Bar | What it shows | What happens as the sample grows |
|---|---|---|
| ± 1 standard deviation | How much the individual readings vary | Stays about the same: it is a property of the organism |
| ± 1 standard error | How precisely the mean has been estimated | Shrinks, roughly as one over the square root of n |
| 95% confidence interval | A range that would contain the true mean in 95 of 100 such samples | Shrinks, and is roughly twice the standard error for a decent sample |
That table is why the overlap rule fails. Standard-deviation bars describe variation between organisms and barely narrow however many you measure, so two genuinely different means can easily have overlapping SD bars. Standard-error bars describe how well you know the mean and shrink with sample size, so with a large enough sample almost any two means separate. The same data drawn two ways gives opposite impressions, and neither drawing is a test.
What the bars are honestly good for is triage. Large overlap means a significant difference is unlikely and the test will probably say so; no overlap means there may well be one and the test is worth doing. Everything between belongs to the test alone, and the sentence to write is 'the error bars do not overlap, which suggests a difference; a t-test is needed to establish whether it is significant.'
Error bars also earn a mark by saying what varied. If the bars on a graph of enzyme activity are long at 50 °C and short at 20 °C, that is a finding: the reaction has become less repeatable near the denaturation threshold, and saying so is better biology than describing the means alone.
The null hypothesis, and what p = 0.05 actually says
Every statistical test starts from a null hypothesis: a specific statement that there is no effect. It is written before the data are looked at, and the test's whole job is to ask whether the data are awkward for it.
- Null hypothesis
- A statement that there is no significant difference, no significant association or no significant correlation between the variables, and that any difference observed is due to chance.
- Significance level
- The probability threshold, conventionally 0.05, below which the null hypothesis is rejected.
- Degrees of freedom
- A number derived from the size of the data set that decides which critical value you read from the table.
Writing it well is a mark on its own, and the failure mode is vagueness. Name both variables and the populations:
| Test | A null hypothesis in the form examiners accept |
|---|---|
| Chi-squared | There is no significant difference between the observed numbers of offspring in each phenotype and the numbers expected from a 9:3:3:1 ratio. |
| t-test | There is no significant difference between the mean shell length of limpets on the sheltered shore and the mean shell length of limpets on the exposed shore. |
| Spearman's rank | There is no significant correlation between soil moisture content and the number of plant species per quadrat. |
Now the part worth getting right rather than merely learning. A p-value is the probability of getting a result at least as extreme as the one you got, if the null hypothesis were true. Textbooks shorten that to 'the probability that the difference is due to chance' and mark schemes accept it, so you may write it — but keep the long version in your head, because the short one invites two conclusions that are false.
It does not say there is a 95% probability the effect is real. The p-value is calculated assuming the null hypothesis is true, so it cannot turn round and tell you how likely that assumption was. And it says nothing about the size of the effect: half a millimetre between two mean shell lengths will come out significant if you measure enough limpets, because significance depends on sample size as much as on the difference. Significant means unlikely to be this big by chance alone, and the sentence after a significant result should always say how big the difference was as well as that it was there.
The 0.05 is a convention and it sets what you are willing to risk: one time in twenty you will reject a true null hypothesis and announce an effect that is not there. That is the price of the test, and it is why a study that runs twenty comparisons and reports the one that came out significant has reported nothing.
Choosing the test
Three tests cover almost everything an A-level investigation produces, and choosing between them is a question about the shape of your data rather than about the biology.
| You have | You want to know | Test |
|---|---|---|
| Counts of individuals in categories, and a ratio you predicted | Whether the counts differ from the prediction more than chance would explain | Chi-squared |
| Two sets of measurements, each scattered about a mean | Whether the two means differ | Student's t-test |
| Two measurements made on each of the same individuals or quadrats | Whether they vary together | Spearman's rank correlation |
Two conditions are worth checking first. Chi-squared needs raw counts, never percentages, and every expected value should be at least 5; combining categories or collecting more data is the fix when they are not. The t-test assumes measurements that are continuous and roughly normally distributed, which lengths, masses and volumes usually are and counts of individuals usually are not.
All three then work the same way. Calculate a number from your data, work out the degrees of freedom, look up the critical value at p = 0.05, and compare — and for all three, a calculated value at or above the critical value means reject the null hypothesis. Spearman's is the one to watch, because its value runs from −1 to +1 and it is the size you compare: a correlation of −0.87 is exactly as significant as one of +0.87 and points the other way.
The three tests, worked
The numbers below are constructed rather than collected, chosen so the arithmetic stays readable. The sequence of steps is identical every time.
chi-squared = Σ [ (O − E)² ÷ E ]O is the observed count, E the expected count, and the sum runs over every category
Chi-squared against a predicted ratio
A dihybrid cross is predicted to give offspring in the ratio 9:3:3:1. Of 320 offspring, the four phenotypes are counted as 168, 66, 70 and 16. Test whether the results differ significantly from the prediction.
Null hypothesis: there is no significant difference between the observed numbers of each phenotype and the numbers expected from a 9:3:3:1 ratio.
Expected values. The ratio has 9 + 3 + 3 + 1 = 16 parts and 320 ÷ 16 = 20 offspring per part, so the expected counts are 180, 60, 60 and 20 — every one well above 5.
The sum. (168 − 180)² ÷ 180 = 0.80; (66 − 60)² ÷ 60 = 0.60; (70 − 60)² ÷ 60 = 1.67; and (16 − 20)² ÷ 20 = 0.80. Adding them gives chi-squared = 3.87.
Degrees of freedom = number of categories − 1 = 4 − 1 = 3. The critical value at p = 0.05 with 3 degrees of freedom is 7.81.
Conclusion. 3.87 is less than 7.81, so the null hypothesis is accepted. There is no significant difference between the observed results and a 9:3:3:1 ratio, and the differences that exist can be attributed to chance — a result consistent with two genes assorting independently.
t = (x̄₁ − x̄₂) ÷ √( s₁²/n₁ + s₂²/n₂ )take the difference between the means as a positive number; degrees of freedom = n₁ + n₂ − 2
A t-test on two means
Limpet shell lengths are measured on a sheltered shore and on an exposed shore, ten from each. The sheltered sample has a mean of 22.0 mm with a standard deviation of 2.0 mm; the exposed sample has a mean of 19.0 mm with a standard deviation of 2.5 mm. Test whether the means differ significantly.
Null hypothesis: there is no significant difference between the mean shell length of limpets on the sheltered shore and the mean shell length of limpets on the exposed shore.
The denominator first. s₁²/n₁ = 4.0 ÷ 10 = 0.400 and s₂²/n₂ = 6.25 ÷ 10 = 0.625, totalling 1.025, and √1.025 = 1.012.
The statistic. The means differ by 22.0 − 19.0 = 3.0 mm, so t = 3.0 ÷ 1.012 = 2.96.
Degrees of freedom = 10 + 10 − 2 = 18, and the critical value at p = 0.05 is 2.10.
Conclusion. 2.96 is greater than 2.10, so the null hypothesis is rejected. There is a significant difference between the means: limpets on the sheltered shore have shells 3.0 mm longer on average, and a difference this large would arise in fewer than 5 samples in 100 if the two shores really held the same population.
rs = 1 − [ 6 Σd² ÷ n(n² − 1) ]d is the difference between the two ranks of one individual, and n is the number of individuals — not the number of measurements
Spearman's rank on ten quadrats
Soil moisture and the number of plant species are recorded in each of ten quadrats. Ranking each variable from 1 to 10 and taking the difference in rank for each quadrat gives squared differences of 1, 4, 4, 1, 1, 4, 4, 1, 1 and 1. Test for a correlation.
The measurements are not reproduced here on purpose. Spearman's test never sees them — it replaces every value with its position in the order — so the ranks are the whole of the input, and printing invented moisture readings would only invite you to interpret scatter nobody measured.
Null hypothesis: there is no significant correlation between soil moisture content and the number of plant species per quadrat.
The statistic. Σd² = 22, and with n = 10, n(n² − 1) = 10 × 99 = 990. So rs = 1 − (6 × 22) ÷ 990 = 1 − 0.133 = 0.87.
The critical value for n = 10 at p = 0.05 is 0.648. Spearman uses n, the number of pairs, rather than a degrees-of-freedom figure.
Conclusion. 0.87 is greater than 0.648, so the null hypothesis is rejected: there is a significant positive correlation between soil moisture and the number of plant species. It is positive because rs is positive, and being significant does not make it causal — both could be responding to something else about where the quadrats fell.
Ranking has one detail that gets tested: equal readings take the mean of the positions they occupy. Values of 3, 7, 7 and 9 rank as 1, 2.5, 2.5 and 4, because the two sevens would have held positions 2 and 3, and the next value still takes position 4.
Writing the conclusion
Every conclusion from these tests has the same four parts, and mark schemes award them separately. Leave one out and you lose it, however good the arithmetic was.
| The part | What it looks like |
|---|---|
| The comparison | 'The calculated value of 2.96 is greater than the critical value of 2.10…' |
| The conditions | '…at p = 0.05 with 18 degrees of freedom.' |
| The decision | 'The null hypothesis is therefore rejected.' |
| The biology | 'There is a significant difference between the mean shell lengths on the two shores.' |
Two points of wording. Boards accept 'accept the null hypothesis' when the calculated value falls short, and you should write it because it earns the mark; what it means is that the data give no reason to abandon the null hypothesis, which is not evidence that it is true, since a small sample fails to reject almost anything. If there is room for one extra clause in an evaluation, 'a larger sample might have detected a difference this one could not' is the clause worth having.
And keep 'significant' for what the test said. In ordinary English it means important; here it means the calculated value beat the critical value at the stated probability. Calling a difference significant when no test was done, and calling a significant difference large, are two ways of losing the same mark.
TRY IT — Reading a result properly
A student compares the mean number of stomata per square millimetre on the upper and lower surfaces of privet leaves, twelve leaves of each. The calculated value of t is 1.84 and the critical value at p = 0.05 with 22 degrees of freedom is 2.07. Write the conclusion, and say what the student should do next.
Check your answer
The calculated value of 1.84 is less than the critical value of 2.07 at p = 0.05 with 22 degrees of freedom, so the null hypothesis is accepted. There is no significant difference between the mean number of stomata per square millimetre on the two surfaces of the leaves sampled.
The student must not write that the two surfaces have the same stomatal density. Failing to detect a difference is not the same as showing there is none, and twelve leaves per surface gives the test limited power to detect a small one.
What to do next is collect more leaves: a larger sample raises the degrees of freedom, lowers the critical value and shrinks the standard error of each mean. The sampling is worth checking too — twelve leaves from one plant describe that plant rather than privet, and no statistics repair a sample drawn badly.
In the exam
- Write the null hypothesis with both variables named and the populations identified. 'There is no difference' on its own scores nothing.
- Quote the critical value with its conditions — the probability level and either the degrees of freedom or n. A comparison with an unlabelled number is not a comparison.
- Degrees of freedom differ by test. Chi-squared uses categories − 1, the t-test uses n₁ + n₂ − 2, and Spearman uses n itself.
- Chi-squared needs raw counts, never percentages, and its expected values must be at least 5 for the result to be trusted.
- 'The error bars do not overlap so the difference is significant' is marked wrong. Say it suggests a difference and that a test is needed.
- A significant result is not a large one. Where a question asks you to evaluate a conclusion, the size of the difference and the size of the sample are both fair game.
- Do not write that a test proves anything. It supports rejecting or retaining a null hypothesis at a stated probability, and that is the strongest verb available.
Check yourself
Two groups of woodlice are given a choice chamber, one half humid and one half dry. Of 80 woodlice, 58 are found on the humid side and 22 on the dry side after ten minutes. State a null hypothesis, choose and carry out a suitable test, and write the conclusion. The critical value of chi-squared at p = 0.05 with 1 degree of freedom is 3.84.
Answer
Test: chi-squared, because the data are counts of individuals in two categories against a ratio that can be predicted — with no effect of humidity the woodlice would be spread 1:1.
Null hypothesis: there is no significant difference between the observed numbers of woodlice on the humid and dry sides and the numbers expected if humidity had no effect on their distribution.
Calculation: the expected values are 80 ÷ 2 = 40 on each side, both well above 5. Then (58 − 40)² ÷ 40 = 8.10 and (22 − 40)² ÷ 40 = 8.10, so chi-squared = 16.2. Degrees of freedom = 2 categories − 1 = 1, and the critical value is 3.84.
Conclusion: 16.2 is far greater than 3.84 at p = 0.05 with 1 degree of freedom, so the null hypothesis is rejected. Significantly more woodlice were found on the humid side than an even distribution would give.
Worth adding in an evaluation: the test shows the distribution was uneven, not why. Separating a kinesis, in which woodlice simply move less once they reach damp air and so accumulate, from a taxis, in which they move towards it directionally, needs individual paths tracked rather than a headcount at the end.
Questions
Question 15 marks
Two heterozygous pea plants are crossed and the offspring are expected in a ratio of 3 tall to 1 dwarf. Of 400 offspring, 280 are tall and 120 are dwarf. Calculate chi-squared and use it to reach a conclusion. The critical value at p = 0.05 with 1 degree of freedom is 3.84.
Mark scheme
- M1 expected values are 400 × 3/4 = 300 tall and 400 × 1/4 = 100 dwarf, both comfortably above 5
- M1 (280 − 300)² ÷ 300 = 1.33 and (120 − 100)² ÷ 100 = 4.00
- A1 chi-squared = 1.33 + 4.00 = 5.33
- B1 degrees of freedom = 2 categories − 1 = 1, and the calculated value of 5.33 is greater than the critical value of 3.84 at p = 0.05
- B1 the null hypothesis is rejected: there is a significant difference between the observed numbers and a 3:1 ratio, so the difference is unlikely to have arisen by chance alone
Question 24 marks
Five leaves taken from one shrub have widths of 12, 14, 15, 17 and 22 mm. Calculate the mean width and the standard deviation of these widths, giving the standard deviation to three significant figures.
Mark scheme
- M1 mean = (12 + 14 + 15 + 17 + 22) ÷ 5 = 80 ÷ 5 = 16 mm
- M1 the deviations from the mean are −4, −2, −1, +1 and +6, and the squares of these total 16 + 4 + 1 + 1 + 36 = 58
- M1 divide by n − 1 rather than by n: 58 ÷ 4 = 14.5, then take the square root
- A1 standard deviation = 3.81 mm
Question 34 marks
A student writes that the error bars on two mean values do not overlap, so the difference between the means is significant. Explain why this reasoning is unsound.
Mark scheme
- B1 an error bar may show one standard deviation, one standard error or a 95 per cent confidence interval, and the three have different lengths drawn on the same data, so the same means can appear to overlap or not depending on the choice
- B1 standard-deviation bars describe variation between individuals and barely narrow however many organisms are measured, so two genuinely different means can easily have overlapping bars
- B1 standard-error bars describe how precisely each mean has been estimated and shrink roughly as one over the square root of the sample size, so with a large enough sample almost any two means separate
- B1 significance is decided only by comparing a calculated test value with a critical value at a stated probability, so bars can suggest that a test is worth doing and no more
Question 44 marks
Compare Student's t-test with Spearman's rank correlation, referring to the data each one needs and the question each one answers.
Mark scheme
- B1 a t-test needs two separate sets of measurements each scattered about a mean, whereas Spearman's needs two measurements made on each of the same individuals or quadrats
- B1 a t-test asks whether the two means differ, whereas Spearman's asks whether the two variables vary together
- B1 the t-test uses the measurements themselves and assumes they are continuous and roughly normally distributed, whereas Spearman's replaces every measurement by its rank and never sees the values
- B1 degrees of freedom for the t-test are n₁ + n₂ − 2, whereas Spearman's critical value is read against n, the number of pairs; for both, a calculated value at or above the critical value means the null hypothesis is rejected
Question 53 marks
A researcher measures 2000 fish from each of two lakes and reports that the mean masses differ by 0.4 g, a difference a t-test finds significant at p = 0.05. Suggest why this significant result may still be of little biological interest.
Mark scheme
- B1 significance depends on the size of the sample as well as on the size of the difference, so a sample of 2000 from each lake can detect a difference this small
- B1 significant means unlikely to have arisen by chance under the null hypothesis; it does not mean the difference is large, and 0.4 g is a tiny fraction of the mass of a fish
- B1 the sentence reporting a significant result should give the size of the difference as well, and further work would be needed to show whether a difference this small has any effect on the fish
Question 62 marks
An investigation compares the mean number of stomata per square millimetre on leaves grown in sun and on leaves grown in shade. State a suitable null hypothesis, and state the significance level conventionally used in biology.
Mark scheme
- B1 there is no significant difference between the mean number of stomata per square millimetre on sun leaves and the mean number on shade leaves
- B1 the conventional significance level is p = 0.05, below which the null hypothesis is rejected
Worth remembering
- The mean uses every reading, the median ignores extremes, the mode is the only one that works on categories.
- The range is fixed by two readings and grows with sample size; the standard deviation uses them all.
- Error bars can suggest a difference and can never establish one, and what they suggest depends on whether they show SD, standard error or a confidence interval.
- The null hypothesis says there is no significant difference, association or correlation, and is written before the data are seen.
- Counts against a predicted ratio go to chi-squared, two means to a t-test, two measurements per individual to Spearman's rank.
- Calculated value at or above the critical value means reject the null hypothesis; below it means the data give no reason to.
- Significant means unlikely to have arisen by chance under the null hypothesis. It does not mean large, and it does not mean proved.