Component 2: Reliability (knowledge, understanding and application of) — Eduqas A-Level Psychology

Reliability is a fundamental concept in psychological research, referring to the consistency of a measure or procedure. In the Eduqas A-Level specification, students must understand different types of reliability, including test-retest, inter-rater, and split-half reliability, and how they are assessed. This topic is crucial because without reliability, research findings cannot be trusted or replicated, undermining the scientific credibility of psychology.

The concept of reliability is closely linked to validity and is assessed in both research methods and in the evaluation of specific studies. Students are expected to apply their knowledge to a range of scenarios, from questionnaires and observations to experiments and interviews. Understanding reliability also involves knowing how to improve it, such as through standardisation, training, and refining measurement tools, which is essential for designing robust psychological research.

• →Reliability refers to the consistency of a measure or research procedure, meaning it produces stable and dependable results across different occasions, observers, or items.
• →Test-retest reliability assesses whether a measure yields similar results when administered to the same participants on two separate occasions, typically using a correlation coefficient.
• →Inter-rater reliability measures the degree of agreement between two or more independent observers or raters, often calculated as a percentage agreement or correlation.
• →Split-half reliability assesses the internal consistency of a test by splitting items into two halves and correlating participants' scores on each half.
• →Improving reliability involves standardising procedures, training observers, using clear operational definitions, and removing ambiguous items from questionnaires or tests.

• Identifies the split-half method as a way of assessing internal reliability by comparing scores on two halves of the same test.
• States that high internal reliability means the items of a test or scale measure the same underlying construct consistently.
• Distinguishes internal reliability from inter-rater reliability, which concerns agreement between separate observers or scorers.
• Distinguishes internal reliability from test-retest reliability, which concerns consistency of a whole test across occasions.
• Defines external reliability as the consistency of a measure across occasions, situations or researchers.
• Identifies the test-retest method as a way of checking consistency over time by correlating two administrations.
• Explains that a high positive correlation between the two sets of scores indicates good external reliability.
• Describes how standardised procedures and clear operational definitions improve consistency across researchers.
• Applies the concept by identifying threats such as practice effects, fatigue or ambiguous instructions.
• Identifies standardisation of procedure, instructions and materials as a way of reducing random variation and so improving consistency.
• Describes observer training and a clearly defined coding frame as ways of improving consistency between raters.
• Explains that piloting identifies ambiguous items or unclear tasks that would otherwise produce inconsistent responses.
• States that rewriting, removing or adding items can improve the internal consistency of a self-report measure.
• Distinguishes reliability from validity by noting that a consistent measure may still fail to measure the intended construct.
• Defines inter-rater reliability as the extent to which independent observers agree and describes comparing their records.
• Describes test-retest reliability as administering the same measure on two occasions and correlating the scores.
• Explains split-half reliability as correlating scores on two halves of a single test to assess internal consistency.
• Matches each method to the source of inconsistency it addresses, namely raters, occasions or items.
• Notes that a strong positive correlation indicates high reliability for the method concerned.

• 💡Read the stem for the key phrase 'within the same test' and match it to split-half rather than to agreement between raters.
• 💡Eliminate any option that mentions two or more observers or scorers, because that describes inter-rater reliability.
• 💡Eliminate any option that mentions repeating the test on a second occasion, because that describes test-retest reliability.
• 💡Name the specific method, such as test-retest, and state what is correlated with what.
• 💡Explain the direction of the expected correlation and why a high positive value supports external reliability.
• 💡Give an applied example, such as repeating a memory test a week later, to demonstrate understanding.
• 💡Read the stem carefully to see whether it asks about improving reliability or about assessing it, because the two require different answers.
• 💡Link each strategy to the source of inconsistency it addresses, such as rater differences, item ambiguity or situational variation.
• 💡Use precise terminology such as standardisation, inter-rater agreement and internal consistency rather than vague phrases like 'making it better'.
• 💡Reject options that confuse reliability with validity or that promise a guaranteed outcome.
• 💡Identify the source of inconsistency named in the stem, such as different observers, different occasions or different items, and select the matching method.
• 💡Use the term correlation when describing how scores are compared, and specify that a strong positive correlation indicates consistency.
• 💡Distinguish the three methods clearly in your reasoning rather than listing them without explanation.
• 💡Reject options that describe assessing validity or that mix up which method addresses which source of inconsistency.
• 💡Always define reliability clearly and distinguish it from validity. Use the correct terminology, such as 'consistency', 'test-retest', and 'inter-rater reliability'.
• 💡When asked to improve reliability, link your suggestions to the specific method and context. For example, for an observation, suggest using a behavioural checklist and training observers; for a questionnaire, suggest removing ambiguous items.
• 💡Use numerical values where appropriate, such as stating that a correlation of +0.80 or above indicates high reliability, or that 80% agreement is the threshold for inter-rater reliability.

• Treats inter-rater reliability as a check on internal consistency; recognise that agreement between separate raters is external reliability, while internal consistency concerns the items within one test.
• Confuses test-retest reliability with internal reliability; note that repeating the whole test later assesses stability over time, whereas split-half compares halves of a single administration.
• Assumes a single administration of a test cannot reveal reliability; remember that split-half uses one administration and compares its two halves.
• Confuses external reliability with internal reliability; keep external reliability about consistency across occasions or researchers and internal reliability about consistency within one instrument.
• Treats a single administration of a test as sufficient evidence of external reliability; recognise that test-retest requires two occasions to compare.
• Overlooks practice effects or fatigue as threats; note that these can lower the correlation between two administrations.
• Treats reliability and validity as interchangeable; keep them separate, because consistency does not guarantee that the intended construct is measured.
• Assumes that increasing sample size automatically fixes an unreliable measure; correct this by targeting the procedure, items or raters that generate the inconsistency.
• Describes only one method of improvement when the question asks for ways in the plural; supply several distinct strategies.
• Claims that a pilot study guarantees reliability; present piloting as a way of identifying problems before the main study.
• Confuses inter-rater reliability with inter-rater validity; keep the focus on agreement between observers rather than on whether the behaviour is the right one to measure.
• Applies test-retest to a trait expected to change over the interval; choose a suitable interval so genuine change is not mistaken for unreliability.
• Treats split-half reliability as a comparison between two different tests; describe it as two halves of the same test.
• Assumes a low correlation proves the measure is invalid; state that it indicates inconsistency and does not by itself settle validity.
• Students often confuse reliability with validity. Reliability is about consistency, while validity is about accuracy. A measure can be reliable but not valid, but it cannot be valid without being reliable.
• Students may think that a high correlation always means high reliability. However, the correlation must be positive and strong (typically +0.80 or above) to indicate good reliability; a weak correlation indicates poor reliability.
• Students sometimes believe that reliability is only relevant to questionnaires. In fact, reliability applies to all research methods, including observations, experiments, and interviews, and can be assessed in different ways.

1. 1Start by defining reliability and distinguishing it from validity. Create a table comparing the two concepts with examples.
2. 2Learn the three main types of reliability: test-retest, inter-rater, and split-half. For each, note how it is assessed and what an acceptable level is.
3. 3Practice applying reliability to different research methods (e.g., questionnaires, observations, experiments) by identifying potential threats to reliability and suggesting improvements.
4. 4Complete past paper questions on reliability, focusing on 4-6 mark questions that require explanation and application. Mark your answers using the mark scheme and note where you lost marks.
5. 5Create flashcards for key terms and test yourself regularly. Use active recall to ensure you can define and apply each concept without notes.

• 📋Definition and explanation questions: e.g., 'Explain what is meant by reliability in psychological research.' (2-4 marks). Advice: Provide a clear definition and use an example to illustrate.
• 📋Application questions: e.g., 'Identify and explain one way to improve the reliability of the observation described in the study.' (4-6 marks). Advice: Link your answer to the specific method and context, and use appropriate terminology.
• 📋Evaluation questions: e.g., 'Discuss the reliability of the research method used in the study.' (6-8 marks). Advice: Consider strengths and weaknesses, and refer to types of reliability and how they could be assessed.
• 📋Data interpretation questions: e.g., 'Calculate the inter-rater reliability from the data provided and explain whether it is acceptable.' (4 marks). Advice: Show your working and interpret the result in relation to the 80% threshold.