Systems architecture: Cache

The idea

Cache is a small, fast block of memory that sits between the CPU and main memory (RAM). It holds copies of frequently used instructions and data so the processor can fetch them quickly instead of waiting for slower RAM. Because cache is close to the CPU cores and built from fast memory, access times are far shorter than main memory, which reduces the average time the CPU spends waiting for data. Cache is organised in levels: L1 is smallest and fastest and is usually per core, L2 is larger and slower, and L3 is larger again and often shared. When the CPU needs an item, it checks cache first; a hit means it is found, a miss means it must be fetched from RAM and a copy stored in cache. Larger cache and higher hit rates generally improve performance.

What earns the marks

• Cache is a small amount of very fast memory located close to (or on) the CPU.
• Cache stores copies of frequently used instructions and data to reduce access time compared with fetching from main memory.
• A cache hit means the required item is found in cache; a cache miss means it must be retrieved from main memory and a copy placed in cache.
• Cache is arranged in levels (L1, L2, L3) that differ in size and speed, with L1 smallest and fastest.
• Using cache reduces the time the CPU waits for data, improving overall processing performance.

Common mistakes and how to put them right

• Confusing cache with main memory (RAM).

  Put it right: Cache is much smaller and faster and holds copies of frequently used items, while RAM is larger and slower and holds programs and data in use.

• Thinking cache permanently stores all data.

  Put it right: Cache holds copies of frequently used instructions and data, and contents can be replaced when space is needed.

• Believing a cache miss means the data does not exist.

  Put it right: A miss only means the item was not in cache, so it is fetched from main memory.

Exam tips

• Link the answer to speed: state that cache is faster than main memory and reduces CPU waiting time.
• Use the terms hit and miss correctly, explaining what each means for where data is fetched from.
• If asked about levels, remember L1 is smallest and fastest and higher levels are larger and slower.