This is the fastest memory of all. The L2 cache on the CPU chip can be accessed four times faster than if it were on a separate chip. When the processor needs to execute an instruction, it looks first in its own data registers. If the needed data isn't there, it goes to the L1 cache and then to the L2 cache. It might not even be there, in which case the system has to retrieve it from the disk.
Every miss introduces a delay, or latency, as the processor tries a slower level.
For high-end processors, it can take from one to three clock cycles to fetch information from L1, while the CPU waits and does nothing. It takes six to 12 cycles to get data from an L2 on the processor chip, and dozens or even hundreds of cycles for off-CPU L2. Caches are more important in servers than in desktop PCs because servers have so much traffic between processor and memory generated by client transactions. Although the bus connecting processor and memory ran only at 25 MHz, this cache let many programs run entirely within the 486 chip at 50 MHz. As this performance mismatch grows, hardware makers will add a third and possibly fourth level of cache memory, says John Shen, a professor of electrical and computer engineering at Carnegie Mellon University in Pittsburgh.