Just because a Porsche and a Honda are both running at 4,000 RPM does not mean that they are travelling at the same speed. Clock rate or cycle time is limited mostly by chip technology. This is determined by the instruction set, addressing modes, and register set. If a processor has very few registers, extra time is required to shuffle data between memory and registers to make space for the next operation. Also, the ability for a compiler to optimize to take advantage of special instructions and architecture plays an important role in determining performance, as it doesn not matter how good the underlying architecture is if the compiler ignores its capabilities or generates inefficient code.
Modern processors use various techniques to improve the cycles per instruction.
One approach is superscalar architecture. In superscalar processors, there are multiple scalar units in a single processor, and each can be working on a different task simultaneously. A common modern approach is pipelining and superpipelining. Pipelining works like an assembly line, with various parts of the processing broken up into small sequential tasks, with instructions and data moved through the pipeline in an orderly manner. Once a pipeline is filled, the number of cycles per instruction drops dramatically as many operations are done simultaneously.