Understand simple probability (to include: explaining the difference…

The idea

Simple probability is the likelihood of an event occurring, expressed as a number between 0 and 1, a fraction, or a percentage. In psychology, probability (p) determines if results are statistically significant. A significance level is the threshold probability set to decide whether to reject the null hypothesis, representing the maximum acceptable risk of a Type 1 error (rejecting a true null). Psychology normally works at p = 0.05, tolerating a 5% maximum risk of a Type 1 error. The stricter 0.01 level tolerates only a 1% risk and is chosen where a false claim would be costly, such as in drug trials. Tightening the criterion to 0.01 cuts the risk of a Type 1 error but raises the risk of a Type 2 error (retaining a false null).

What earns the marks

• Defining simple probability: expressing the likelihood of an event as a numerical value between 0 and 1, a fraction, or a percentage.
• Defining significance level: the threshold probability set by the researcher for rejecting the null hypothesis, representing the maximum acceptable risk of a Type 1 error.
• Distinguishing the two figures: 0.05 permits a 5% risk of a Type 1 error and 0.01 permits only a 1% risk, making 0.01 more stringent and demanding a more extreme calculated value.
• Linking the criterion to error: working at 0.05 rather than 0.01 makes a Type 1 error more likely; working at 0.01 makes a Type 2 error (missing a real effect) more likely.
• Justifying a stricter criterion by naming the consequence in that study, such as a new drug being licensed on a false positive.

Common mistakes and how to put them right

• Defining the significance level as the probability that the results arose by chance, which conflates the alpha threshold with the p-value.

• Claiming that a significant result proves the hypothesis, or that there is a 95% chance the hypothesis is true, when the figure describes the threshold for rejecting the null hypothesis.

• Swapping the two errors, so that rejecting a true null hypothesis is called a Type 2 error instead of a Type 1 error.

• Expressing simple probability incorrectly, such as giving a probability value greater than 1 or a negative number.

Exam tips

• A typical question prints a calculated value with an extract from a table and the phrase 'at the 0.05 level', so read the number of participants off the stem before choosing the row and column.
• Where asked why a researcher used the stricter criterion, the answer needs the consequence of a wrong decision in that particular investigation, not a general remark about accuracy.
• Explaining the difference between the two criteria requires both figures to appear together with what each one means as a percentage or proportion.