Factors affecting the choice of statistical test, including level of…

The idea

Choosing a test is three questions asked in order, and what earns the marks is the justification rather than the name. First, is the study looking for a difference between conditions or an association between two variables? Second, if it is a difference, was the design related (repeated measures or matched pairs) or unrelated (independent groups)? Third, what is the level of measurement: nominal categories, ordinal ranks, or interval data on a scale with equal units? Each combination gives one test. Nominal: sign test if related, chi-squared if unrelated. Ordinal: Wilcoxon if related, Mann-Whitney if unrelated. Interval: the related and unrelated t-tests. For correlations, ordinal data take Spearman's rho and interval data take Pearson's r, which like the t-tests also assumes a normal distribution.

What earns the marks

• Three factors decide the test: whether the hypothesis predicts a difference or a correlation, the experimental design (related or unrelated), and the level of measurement (nominal, ordinal or interval).
• Difference and nominal data: sign test for a related design, chi-squared for an unrelated design.
• Difference and ordinal data: Wilcoxon signed-ranks for a related design, Mann-Whitney U for an unrelated design.
• Difference and interval data: related t-test for a related design, unrelated t-test for an independent groups design.
• Correlation: Spearman's rho for ordinal data and Pearson's r for interval data; chi-squared also serves as a test of association between two nominal variables.
• The parametric tests (both t-tests and Pearson's r) carry extra requirements: interval level data, a roughly normal distribution, and similar variance across the conditions.
• A justification must name the factor as well as the test, quoting the design and the level of measurement of the study in front of you.

Common mistakes and how to put them right

• Naming the test and stopping.

  Put it right: The question almost always asks for a justification, and the marks sit in the design and the level of measurement, not in the name.

• Treating rating scales, questionnaire scores and rank positions as interval data and reaching for a t-test, when they are ordinal and need Wilcoxon or Mann-Whitney.

• Mixing up Wilcoxon and Mann-Whitney.

  Put it right: Both are ordinal tests of difference, so the design is the only thing that separates them: Wilcoxon related, Mann-Whitney unrelated.

• Using chi-squared on means, totals or percentages.

  Put it right: It needs frequencies of separate people falling into independent categories.

• Reading matched pairs as independent groups, which flips every related choice to its unrelated partner.

Exam tips

• Reproduce the grid in rough before answering: three rows for the levels of measurement, columns for related, unrelated and correlation. A copied grid answers the question in seconds.
• In an applied question quote the study's own detail as the reason, for example 'each participant was tested in both conditions, so the design is repeated measures', which is what the AO2 mark is for.
• If the level of measurement is arguable, say which you have decided on and why, then name the test that follows. An examiner credits a defended choice.