Skip to topic
    ← Back to course topics

    Range and precision — AQA A-Level Computer Science

    In AQA A-Level Computer Science, range refers to the difference between the smallest and largest representable numbers in a binary format, while precision denotes the level of detail or smallest step size between consecutive values. Allocating more bits to the exponent increases range, whereas allocating more bits to the mantissa increases precision within fixed-width floating-point representations.

    Range and precision explained

    Fixed point representation places the binary point at a set position, so every number uses the same number of bits for its integer and fractional parts.

    Read the full explanation

    This makes arithmetic simple and fast, but it limits the range of values that can be held and fixes the precision across that range. Floating point representation stores a number as a mantissa and an exponent, with the binary point effectively moving to suit the magnitude. This allows a much wider range of values, from very small to very large, and keeps precision roughly proportional to size. The cost is greater complexity in storage and arithmetic, and the need for hardware or software support to handle the exponent. When comparing the two, consider the range each can represent, the precision each offers across that range, and the speed and complexity of operations.

    Your focus

    1. Compare the advantages and disadvantages of fixed point and floating point forms in terms of

    Which command word is this?

    Set out the cause and effect clearly. For example, explain how altering the exponent impacts range by detailing the wider power-of-two domain.

    Range and precision exam tips

    Topic Overview

    Range and precision are fundamental attributes of numerical data representation in Computer Science, defining the spectrum and resolution of values a system can store. In AQA A-Level Computer Science, students must understand how fixed-point and floating-point systems divide available bits between representing magnitude and fractional detail.

    Mastering this topic is essential for comprehending hardware limitations, rounding errors, and numerical instability. It connects directly to computer architecture, processor registers, and algorithm design, where engineers must select optimal data types to balance computational accuracy against memory overhead.

    Key Concepts
    • →Range is the difference between the minimum and maximum values representable within a specific binary format.
    • →Precision is the degree of granularity or resolution representable, defined by the smallest distance between two consecutive values.
    • →In floating-point representations, range is governed by the number of bits allocated to the exponent, while precision is dictated by the bits allocated to the mantissa.
    • →Under a fixed bit-length constraint, an inherent trade-off exists: widening the exponent broadens range but truncates precision, and vice versa.
    • →Normalization ensures maximum possible precision by shifting the mantissa so it starts with '01' (positive) or '10' (negative), eliminating redundant leading bits.
    Marking Points
    • Explains that fixed point has a limited range because the binary point is fixed.
    • Explains that floating point offers a much wider range through its mantissa and exponent.
    • Explains that fixed point gives uniform precision across its range.
    • Explains that floating point precision varies with magnitude, being finer for small values.
    • Identifies fixed point as simpler and faster to compute with, and floating point as more complex.
    Examiner Tips
    • 💡Structure your comparison around range and precision as separate points, then add speed and complexity.
    • 💡Use a concrete example, such as a financial value versus a scientific measurement, to justify which form suits which context.
    • 💡Keep statements specific: say what limits the range of fixed point and what determines the precision of floating point.
    • 💡When asked to discuss trade-offs, state explicitly that the total bit budget is fixed; failing to mention this often caps your marks.
    • 💡Show working by stating values of both mantissa and exponent in denary before computing the final power of two calculation.
    • 💡Memorize the normalized patterns: positive mantissas always start with '01', and negative mantissas always start with '10'.
    Common Mistakes
    • States that floating point is always more precise, without noting that its precision depends on magnitude.
    • Describes fixed point range as unlimited or treats both forms as having the same range.
    • Gives advantages without linking them to range or precision as the statement requires.
    • Thinking that adding more bits to the mantissa increases range: In reality, mantissa bits only refine fractional accuracy; only exponent bits extend range.
    • Assuming fixed-point arithmetic maintains constant relative precision: Fixed-point formats have constant absolute precision (step size), but their relative precision worsens significantly for very small values.
    • Believing that zero can be represented in normalized form: Zero is a special non-normalizable case because its mantissa cannot start with '01' or '10'.
    Revision Plan
    1. 1Day 1-2: Review two's complement and practice converting fixed-point binary fractions to and from denary.
    2. 2Day 3-4: Master floating-point normalization rules for positive ('01') and negative ('10') values and determine boundary values.
    3. 3Day 5-6: Complete exam-style scenario questions evaluating the trade-off of shifting bits between mantissa and exponent.
    4. 4Day 7: Work through timed past paper questions covering underflow, overflow, and precision loss.
    Exam Question Types
    • 📋Boundary Calculations: Finding the minimum or maximum representable positive/negative values in a given format.
    • 📋Format Comparison: Assessing the impact on range and precision when changing the bit division in a fixed-size word.
    • 📋Error Identification: Explaining whether a computational error is due to overflow (exceeding range) or rounding/truncation (lacking precision).
    Command Word Expectations (AQA)
    Explain

    Set out the cause and effect clearly. For example, explain how altering the exponent impacts range by detailing the wider power-of-two domain.

    Calculate

    Show clear stages of working, convert binary components (mantissa and exponent) into denary, and provide the final value.

    Compare

    Identify both similarities and differences between two formats, directly contrasting their respective range and precision capabilities.

    How Students Lose Marks (Examiner Pitfalls)
    Pitfall: Confusing the effects of altering mantissa and exponent bit allocation on range versus precision.
    ❌ Weak Answer (Loses Marks):Increasing the mantissa bits makes the numbers bigger and allows the system to store a larger range of numbers.
    Example improved answer:Increasing the number of bits allocated to the mantissa increases the precision of the representation, allowing values to be recorded to a finer degree of accuracy with smaller gaps between adjacent representable values. Conversely, it reduces the bits available for the exponent, which decreases the overall range of representable numbers.
    Examiner Tip: Always explicitly link 'mantissa' to 'precision/accuracy' and 'exponent' to 'range/magnitude' in comparative questions.
    Pitfall: Failing to recognize the direct trade-off when overall word length remains constant.
    ❌ Weak Answer (Loses Marks):You can get higher precision and higher range by changing the format.
    Example improved answer:For a fixed total bit-length (e.g. 16 bits), there is an inverse trade-off between range and precision: allocating more bits to the mantissa improves precision but reduces the exponent size and thus limits range; allocating more bits to the exponent expands the range of values that can be represented but decreases the precision due to fewer fractional bits in the mantissa.
    Examiner Tip: State clearly that total bit-width is fixed; otherwise, an examiner cannot award marks for identifying the direct trade-off.
    Step-by-Step Worked Solutions

    Question: A computer system uses an 8-bit two's complement floating-point format with a 5-bit mantissa and a 3-bit exponent. Calculate the maximum positive value and the smallest positive non-zero value that can be represented in normalized form.

    1. 1.Step 1: Identify the bit allocation and sign requirements. The format uses 5 mantissa bits and 3 exponent bits. For normalized positive numbers, the mantissa must start with '01'.
    2. 2.Step 2: Find the maximum positive value. The mantissa should be maximized: 01111 in two's complement equals 0.5 + 0.25 + 0.125 + 0.0625 = 0.9375 (or 15/16). The exponent should be maximized as a positive 3-bit two's complement integer: 011 equals +3. Therefore, value = 0.9375 * 2^3 = 7.5.
    3. 3.Step 3: Find the smallest positive normalized value. The smallest positive normalized mantissa is 01000, which equals 0.5. The most negative 3-bit two's complement exponent is 100, which equals -4. Therefore, value = 0.5 * 2^-4 = 0.5 * 0.0625 = 0.03125 (or 1/32).
    Final Answer: Maximum positive value = 7.5; Smallest positive normalized non-zero value = 0.03125 (1/32)

    Question: Explain the effect on both the range and precision of representable numbers if a system modifies its 16-bit floating-point format from a 10-bit mantissa and 6-bit exponent to an 8-bit mantissa and 8-bit exponent.

    1. 1.Step 1: Analyze changes to the mantissa. The mantissa decreases from 10 bits to 8 bits. Fewer bits representing the fractional part means reduced resolution, leading to larger intervals between consecutive numbers and therefore decreased precision.
    2. 2.Step 2: Analyze changes to the exponent. The exponent increases from 6 bits to 8 bits. With two additional bits in two's complement, the exponent can represent powers from -128 up to +127 (previously -32 to +31), drastically increasing the magnitude span.
    3. 3.Step 3: Synthesize the impact on range and precision. The overall range of representable numbers is significantly increased (both very large and very small magnitudes can be stored), but the precision is reduced, increasing potential rounding errors.
    Final Answer: Precision decreases because the mantissa loses 2 bits; Range increases significantly because the exponent gains 2 bits.
    Active Recall Memory Test
    Which component of a floating-point number dictates its precision, and why?
    Key Fact: The mantissa, because it determines the number of significant digits (resolution) representable.
    Which component of a floating-point number dictates its range, and why?
    Key Fact: The exponent, because it determines the maximum and minimum magnitude scaling factor (powers of two).
    What starting bit pattern signifies a normalized negative floating-point mantissa in two's complement?
    Key Fact: The two leading bits must be '10'.
    What error condition occurs when a calculation produces a positive result smaller than the smallest representable non-zero number?
    Key Fact: Underflow.
    Frequently Asked Questions
    What is the primary difference between range and precision?
    Range refers to the span between the smallest and largest values that a binary representation can accommodate. Precision refers to how accurately a number can be represented, reflecting the smallest detectable difference between two adjacent values. Range is about capacity and limits, whereas precision is about detail and resolution.
    Why is normalization necessary in floating-point representations?
    Normalization ensures that each non-zero real number has a unique binary representation and maximizes precision. By eliminating leading non-significant bits (ensuring the mantissa begins with '01' for positive numbers or '10' for negative numbers), the maximum number of bits is reserved for significant fractional data.
    What happens if a calculation exceeds the maximum representable range?
    If a calculation generates a number larger than the maximum positive or negative limit dictated by the exponent, an arithmetic overflow error occurs. The computer typically sets an overflow flag in the status register, throws an exception, or wraps around incorrectly, producing invalid output.
    Can a floating-point system represent every real number within its range?
    No, floating-point representations are discrete, meaning there are gaps between consecutive representable numbers. Real numbers that fall between these gaps cannot be stored exactly and must be rounded or truncated to the nearest representable value, resulting in precision errors or rounding errors.
    Why would a programmer choose fixed-point over floating-point arithmetic?
    Fixed-point arithmetic requires simpler hardware and is computationally faster because operations can be performed using standard integer arithmetic logic units (ALUs). It is frequently used in low-power embedded microcontrollers, digital signal processors, or applications like financial calculations where exact decimal precision is preferred over dynamic range.