OCR · A-Level · Mathematics
Course version: H240 · Version from Sept 2018
2.02j Be able to clean data, including dealing with missing data, errors…
Full specification wording
Data Presentation and Interpretation: 2.02j Be able to clean data, including dealing with missing data, errors and outliers.
Data cleaning is the process of checking a data set before analysis and correcting or removing values that would distort summary statistics or graphs.
- Point 9 of 9
Study notes
From the teaching notes for this specification point.
The idea
Data cleaning is the process of checking a data set before analysis and correcting or removing values that would distort summary statistics or graphs. Missing data arises when a value is absent; you decide whether to omit the whole record, or to substitute a sensible value such as the mean of the remaining observations, and you state that choice. Errors are values that are impossible or inconsistent, for example a recorded height of 17.5 m for a person or a negative mass; you correct them if the true value is known, otherwise you remove them. Outliers are values far from the rest of the distribution; you identify them, for example by comparing with the interquartile range, then justify keeping, correcting or excluding them. Always report what you changed and why, because cleaning alters the mean, range and correlation.
What earns the marks
- Identifies a missing value and states a defensible treatment such as omitting the record or substituting a representative value.
- Recognises an impossible or inconsistent entry as an error and corrects or removes it with justification.
- Distinguishes an outlier from an error by checking whether the value is plausible but extreme.
- Justifies the decision to retain, correct or exclude a value by reference to its effect on the analysis.
- Reports the cleaning steps taken so that the reader can judge how the data set was altered.
Common mistakes and how to put them right
Treats every unusual value as an error.
Put it right: Check first whether the value is physically possible but simply extreme.
Removes outliers silently without stating the effect on the mean or range.
Put it right: Record each change and its consequence.
Substitutes a missing value with the mean of the whole column including the gap.
Put it right: Compute the substitute from the observed values only.
Assumes cleaning never changes conclusions.
Put it right: Recalculate summary statistics after any correction or removal.
Exam tips
- Read the question for the word 'justify' and give a reason for each value you keep, change or delete.
- Check units and context before calling a value an error, since a plausible extreme is an outlier rather than a mistake.
- State the effect of your cleaning on a named statistic so the examiner sees the reasoning, not just the action.
Try the lesson
The start of the Data Presentation and Interpretation: 2.02j Be able to clean data, including dealing with missing data, errors and outliers. lesson, made from teaching notes for this specification point.
Have a go first
A survey records the heights of 30 students in centimetres, but one entry reads 17.2. What is the most appropriate first step in cleaning this data set?
There is more to this lesson
The full lesson goes on to check your understanding and help you earn the marks. Create a free account to carry on, with your progress saved.
A week of Premium from your first lesson · Then the free plan continues · No card, no charge
Test your understanding of Data Presentation and Interpretation: 2.02j Be able to clean data, including dealing with missing data, errors and outliers.
Active recall beats re-reading. Take an instant quiz now to test what you’ve learned and lock it into long-term memory.
A week of Premium from your first lesson · Then the free plan continues · No card, no charge