Introduction to statistical testing; the sign test. When to use the sign…

The idea

The sign test asks the crudest useful question: did more people move one way than the other? Use it when the hypothesis predicts a difference, the design is repeated measures or matched pairs, and the data are nominal, each participant recorded simply as better or worse, a plus or a minus. To calculate it, find each participant's change, keep only its sign, throw out anyone whose score did not move, count the pluses and the minuses, and take S as the total of the LESS frequent sign. N is how many participants are left once the ties have gone. With fifteen participants, two unchanged, three pluses and ten minuses, S is 3 and N is 13; the critical value for a one-tailed test at the 5% level is 3, and since S is equal to or less than that, the result is significant.

What earns the marks

• Conditions for use, all three needed: the hypothesis predicts a difference, the design is related (repeated measures or matched pairs), and the data are nominal, that is the direction of change.
• Step one: calculate the difference for each participant and record only whether it is a plus or a minus.
• Step two: discard every participant whose score did not change, and let N be the number of participants that remain.
• Step three: total the pluses and total the minuses; the calculated value S is the frequency of the less common sign.
• Step four: find the critical value for that N, for a one-tailed or two-tailed test, at the 5% level, and compare it with S.
• Step five: S must be equal to or LESS than the critical value for the result to be significant; then state the decision about the null hypothesis in the study's own terms.

Common mistakes and how to put them right

• Counting the more frequent sign as S, which reverses the whole test and usually turns a significant result into a non-significant one.

• Leaving the tied participants in N, so the wrong row of the table is used.

  Put it right: Ties are discarded from N as well as from the count of signs.

• Importing the rule from the t-test or chi-squared and asking whether S is bigger than the critical value, when the sign test needs S to be equal to or smaller.

• Using the sign test on an independent groups study, or on a correlation, because the arithmetic looks easy.

  Put it right: It only works on related data.

• Reporting S as a percentage or a difference score rather than a simple count of participants.

Exam tips

• Show the working in stages: the table of signs, then S, then N, then the critical value with tail and level, then the comparison and conclusion.
• Always say whether the hypothesis was directional, because the table has separate one-tailed and two-tailed columns and the wrong column gives the wrong decision.
• If the question gives raw scores, subtract in a consistent direction for every participant, and write the signs down before counting anything.