Each instruction has its own latency and restart rate that must be considered when determining performance. On agressive processor implementations, the restart rate is 1. Most processors do not achieve this for all instructions, especially the complicated floating-point instructions like divide and square root. An instruction is ready when all of its operands are available. Instructions are decoded and put into the pool, and once executed, they are handed to a reordering device that restores the order so that your program appears to be executed in the order it was written.
Most modern processors have several units that can each execute ready instructions.
This feature is also referred to as superscalar architecture. Most caches are 2 or 3-way associative. The speed gain is substantial when desired memory values are cached. If a value is in L1 cache, it typically can be used by the CPU just 1 clock tick after it has been requested. With prefetching, the value can be in the CPU when it is needed with no delay.