Reliability across all methods of investigation. Ways of measuring…

The idea

Reliability refers to the consistency of a measure across all methods of investigation, including experiments, observations, and questionnaires. If a tool is reliable, it produces the same results when repeated under identical conditions. Test-retest reliability measures external consistency by administering the same test to the same participants on two separate occasions and correlating the scores (a correlation coefficient of +0.80 or higher indicates good reliability). Inter-observer reliability assesses consistency between two or more independent observers coding the same behaviour; their records are correlated to ensure objectivity. Reliability can be improved by standardising procedures, operationalising variables clearly, training observers, or replacing ambiguous questionnaire items with closed questions.

What earns the marks

• Define reliability as the consistency of findings or measuring instruments across different research methods.
• Explain test-retest reliability as administering the same test to the same people twice, with an appropriate time gap to prevent recall but avoid genuine change.
• Describe inter-observer reliability as multiple researchers independently recording behaviour using the same behavioural categories, then correlating their data.
• Identify that a Spearman's rho or Pearson's r correlation coefficient of +0.80 or greater demonstrates strong reliability.
• Explain methods to improve reliability, such as standardising instructions, operationalising behavioural categories, or rewriting ambiguous questions.

Common mistakes and how to put them right

• Confusing reliability with validity.

  Put it right: Reliability is about consistency (getting the same result), whereas validity is about accuracy (measuring what is intended).

• Stating that inter-observer reliability involves observers discussing their findings to reach an agreement.

  Put it right: Observers must record data independently before comparing and correlating their results.

• Suggesting a short time gap for test-retest reliability.

  Put it right: The gap must be long enough so participants forget their original answers, but short enough that their actual traits have not changed.

Exam tips

• When asked how to assess reliability in a scenario, explicitly state whether test-retest or inter-observer is most appropriate for the method used.
• Always mention the statistical threshold (+0.80 correlation coefficient) when explaining how to establish reliability.
• If asked to improve reliability in an observation, specifically mention operationalising behavioural categories so they are mutually exclusive and observable.