Inferential testing — AQA A-Level Psychology
Test yourself on Inferential testing with AQA A-Level practice questions.
7 days Premium · Then the free plan continues · No card, no charge
Inferential testing explained
Descriptive statistics tell you what happened in your sample; inferential statistics tell you whether it is safe to say the same about the population the sample came from.
Read the full explanation
The logic never changes. Assume the null hypothesis is true, that there is no difference or no correlation and any pattern is chance. Run a test that turns the whole data set into one number, the calculated or observed value, and compare it with a critical value read from a table. If chance alone would produce a result like this fewer than five times in a hundred, the null hypothesis is rejected and the finding is called significant. None of that is proof: significance is a statement about probability, so a real effect can be missed and a trivial one can reach significance because the sample was enormous.
Introduction to statistical testing; the sign test. When to use the sign test; calculation of the sign test.
The sign test asks the crudest useful question: did more people move one way than the other? Use it when the hypothesis predicts a difference, the design is repeated measures or matched pairs, and the data are nominal, each participant recorded simply as better or worse, a plus or a minus. To calculate it, find each participant's change, keep only its sign, throw out anyone whose score did not move, count the pluses and the minuses, and take S as the total of the LESS frequent sign. N is how many participants are left once the ties have gone. With fifteen participants, two unchanged, three pluses and ten minuses, S is 3 and N is 13; the critical value for a one-tailed test at the 5% level is 3, and since S is equal to or less than that, the result is significant.
Probability and significance: use of statistical tables and critical values in interpretation of significance; Type I and Type II errors.
Probability is the risk of being wrong that a researcher agrees in advance to run. Psychology settles on p less than or equal to 0.05: the possibility that a result like this came about by chance is five in a hundred or fewer. Reading a table of critical values needs three pieces of information, whether the hypothesis was directional (one-tailed) or non-directional (two-tailed), the value of N or the degrees of freedom, and the level of significance, and then the rule for that test, since some require the calculated value to be smaller than the critical value and others larger. A Type I error is a false positive, rejecting a null hypothesis that was true, and gets likelier if the level is relaxed. A Type II error is a false negative, missing a real effect, and gets likelier as the level is made stricter.
Factors affecting the choice of statistical test, including level of measurement and experimental design. When to use the following tests: Spearman’s rho, Pearson’s r, Wilcoxon, Mann-Whitney, related t-test, unrelated t-test and Chi-Squared test.
Choosing a test is three questions asked in order, and what earns the marks is the justification rather than the name. First, is the study looking for a difference between conditions or an association between two variables? Second, if it is a difference, was the design related (repeated measures or matched pairs) or unrelated (independent groups)? Third, what is the level of measurement: nominal categories, ordinal ranks, or interval data on a scale with equal units? Each combination gives one test. Nominal: sign test if related, chi-squared if unrelated. Ordinal: Wilcoxon if related, Mann-Whitney if unrelated. Interval: the related and unrelated t-tests. For correlations, ordinal data take Spearman's rho and interval data take Pearson's r, which like the t-tests also assumes a normal distribution.
Your focus
- Explain how inferential statistics move a claim from the sample to the population by way of probability.
- Apply the decision rule by comparing a calculated value with a critical value and stating what happens to the null.
- Discuss why a significant result is never proof, since a large enough sample makes a trivial effect significant.
Show all 13 objectives
- State the three conditions for a sign test, a predicted difference, a related design and nominal data.
- Calculate S as the count of the less frequent sign once ties are discarded, and state N as the participants left.
- Apply the critical value table by checking that S is equal to or less than it and stating the decision on the null.
- Explain what p less than or equal to 0.05 means and name the three things needed to read a critical value table.
- Apply the right direction of comparison, less than for the sign test and greater than for chi-squared.
- Discuss the trade-off between Type I and Type II errors and when a stricter level such as 0.01 is worth it.
- Outline the three questions that choose a test, difference or correlation, related or unrelated, and level of data.
- Identify the correct test for a described study, from the sign test and chi-squared through to Pearson's r.
- Justify a chosen test by quoting the design and the level of measurement of the study in front of you.
- Explain the extra requirements a parametric test carries, interval data, a normal distribution and similar variance.
Inferential testing exam tips
Marking Points
- Descriptive statistics (mean, range, graphs) summarise the sample; inferential statistics use probability to decide what can be claimed about the wider population.
- The null hypothesis is what is actually tested: it states there is no difference or no correlation and that any pattern in the data is due to chance.
- A statistical test converts the data into a calculated (observed) value, which is then compared with a critical value taken from a table of significance.
- The accepted level of significance in psychology is p less than or equal to 0.05, that is a 5% or smaller possibility that the result arose by chance.
- If the result reaches significance the null hypothesis is rejected and the alternative hypothesis retained; if it does not, the null hypothesis is retained.
- Evaluation: a significant result is probabilistic, never certain, and statistical significance is not the same as psychological importance or a large effect.
- Conditions for use, all three needed: the hypothesis predicts a difference, the design is related (repeated measures or matched pairs), and the data are nominal, that is the direction of change.
- Step one: calculate the difference for each participant and record only whether it is a plus or a minus.
- Step two: discard every participant whose score did not change, and let N be the number of participants that remain.
- Step three: total the pluses and total the minuses; the calculated value S is the frequency of the less common sign.
- Step four: find the critical value for that N, for a one-tailed or two-tailed test, at the 5% level, and compare it with S.
- Step five: S must be equal to or LESS than the critical value for the result to be significant; then state the decision about the null hypothesis in the study's own terms.
- The 5% level (p less than or equal to 0.05) is the conventional level of significance in psychology: no more than a five in a hundred possibility that the result is down to chance.
- Three things are needed to find a critical value in a table: N or the degrees of freedom, whether the test is one-tailed or two-tailed, and the chosen level of significance.
- For the sign test, Wilcoxon and Mann-Whitney the calculated value must be equal to or LESS than the critical value; for the t-tests, Spearman's rho, Pearson's r and chi-squared it must be equal to or GREATER.
- Type I error: the null hypothesis is rejected when it is in fact true, so an effect is claimed that does not exist (the optimistic error).
- Type II error: the null hypothesis is retained when it is in fact false, so a real effect is missed (the pessimistic error).
- The two errors trade off against each other: a lenient level such as 0.10 makes a Type I error likelier, a stringent one such as 0.01 makes a Type II error likelier, and 0.05 is the compromise.
- A stricter level is chosen where the consequences of a false positive are serious, for example in drug trials, or where a finding is being replicated.
- Three factors decide the test: whether the hypothesis predicts a difference or a correlation, the experimental design (related or unrelated), and the level of measurement (nominal, ordinal or interval).
- Difference and nominal data: sign test for a related design, chi-squared for an unrelated design.
- Difference and ordinal data: Wilcoxon signed-ranks for a related design, Mann-Whitney U for an unrelated design.
- Difference and interval data: related t-test for a related design, unrelated t-test for an independent groups design.
- Correlation: Spearman's rho for ordinal data and Pearson's r for interval data; chi-squared also serves as a test of association between two nominal variables.
- The parametric tests (both t-tests and Pearson's r) carry extra requirements: interval level data, a roughly normal distribution, and similar variance across the conditions.
- A justification must name the factor as well as the test, quoting the design and the level of measurement of the study in front of you.
Examiner Tips
- 💡Asked what a significant result means, answer as a probability sentence: the possibility that these results are due to chance is equal to or less than 5%, so the null hypothesis is rejected.
- 💡In a scenario question write the null hypothesis in that study's own variables and conditions, not in the abstract, because the AO2 marks are for applying the idea to the material given.
- 💡Name the population you are generalising to. Inferential testing is about going beyond the sample, and saying so earns the knowledge mark that a bare definition misses.
- 💡Show the working in stages: the table of signs, then S, then N, then the critical value with tail and level, then the comparison and conclusion.
- 💡Always say whether the hypothesis was directional, because the table has separate one-tailed and two-tailed columns and the wrong column gives the wrong decision.
- 💡If the question gives raw scores, subtract in a consistent direction for every participant, and write the signs down before counting anything.
- 💡Write the conclusion in a fixed order: the calculated value, the critical value for that N and tail, which is bigger, the decision about the null hypothesis, and the level. All of it is creditable.
- 💡When a question moves the level from 0.05 to 0.01 it is asking about error types: a stricter level cuts the risk of a Type I error and raises the risk of a Type II error. Say both halves.
- 💡In an applied question name which error the researcher actually risks, using the scenario's own claim, rather than defining both errors in the abstract.
- 💡Reproduce the grid in rough before answering: three rows for the levels of measurement, columns for related, unrelated and correlation. A copied grid answers the question in seconds.
- 💡In an applied question quote the study's own detail as the reason, for example 'each participant was tested in both conditions, so the design is repeated measures', which is what the AO2 mark is for.
- 💡If the level of measurement is arguable, say which you have decided on and why, then name the test that follows. An examiner credits a defended choice.
Common Mistakes
- Writing that the test 'proves' the hypothesis. An inferential test gives a probability, so the null hypothesis is rejected rather than disproved and the effect is never established with certainty.
- Offering descriptive evidence as if it were inferential: quoting that one mean is bigger than another, or that a bar is taller, and calling the difference significant without any test.
- Saying only that 'the results were significant' with no level, no tail and no comparison of calculated with critical value, which leaves an examiner nothing to credit.
- Stating the null hypothesis as the opposite prediction ('participants will do worse') instead of as no difference or no correlation.
- Counting the more frequent sign as S, which reverses the whole test and usually turns a significant result into a non-significant one.
- Leaving the tied participants in N, so the wrong row of the table is used; ties are discarded from N as well as from the count of signs.
- Importing the rule from the t-test or chi-squared and asking whether S is bigger than the critical value, when the sign test needs S to be equal to or smaller.
- Using the sign test on an independent groups study, or on a correlation, because the arithmetic looks easy; it only works on related data.
- Reporting S as a percentage or a difference score rather than a simple count of participants.
- Swapping the errors over. Type I is claiming an effect that is not there; Type II is failing to find one that is.
- Reading p less than or equal to 0.05 as '95% certain the hypothesis is true'. It is the probability of the data arising by chance, not the probability that the hypothesis is correct.
- Applying one comparison rule to every test, when half of the named tests want the calculated value to be smaller than the critical value.
- Ignoring the one-tailed and two-tailed columns, or using N where the test needs degrees of freedom, so the right test is read against the wrong critical value.
- Saying a non-significant result means the null hypothesis is true, rather than that there is insufficient evidence to reject it.
- Naming the test and stopping. The question almost always asks for a justification, and the marks sit in the design and the level of measurement, not in the name.
- Treating rating scales, questionnaire scores and rank positions as interval data and reaching for a t-test, when they are ordinal and need Wilcoxon or Mann-Whitney.
- Mixing up Wilcoxon and Mann-Whitney. Both are ordinal tests of difference, so the design is the only thing that separates them: Wilcoxon related, Mann-Whitney unrelated.
- Using chi-squared on means, totals or percentages. It needs frequencies of separate people falling into independent categories.
- Reading matched pairs as independent groups, which flips every related choice to its unrelated partner.