If R is the number of requests that the memory can service simultaneously, then For processors that can produce multiple memory requests, it is important not only to reduce latency but also to increase bandwidth by designing a memory system that is capable of servicing multiple requests simultaneously. Memory hierarchies provide decreased average latency and reduced band-width requirements, whereas parallel or interleaved memories provide higher bandwidth. The basic unit of construction of a semiconductor memory system is a module or bank. A single bank can service only one request at a time. The time that a bank is busy servicing a request is called the bank busy time.
Caches have much shorter bank busy times than do main memory banks.
Cache contains redundant copies of portions of the address space which is wholly contained in the main memory. The address of each cache line in data memory is stored in the tag memory along with the state. Cache memory must be content-addressable and all tags must be compared concurrently in order to achieve the low latency which is the point of cache memory. Cache can be simplified by direct mapping each memory location to a single location in the cache. This simplification is achieved at the cost of a lower hit ratio.