A cache memory for storing 2^J-byte cache lines where J is an integer greater than three, the cache memory comprising: an array of 2^N sets each of which holds tags that each are X bits, wherein N and X are both integers greater than five; wherein the array has 2^W ways; an input that receives a Q-bit memory address, MA[(Q-1):0],, having: a tag portion MA[(Q-1):(Q-X)]; and an index portion MA[(Q-X-1):J]; wherein Q is an integer at least (N+J+X-1); characterized in that the cache memory further comprises: another input that receives a mode, said mode being one of a first, second or third mode; set selection logic that selects one set of the array using the index portion and a least significant bit of the tag portion when operating in a first mode or a second mode; comparison logic that compares all but the least significant bit of the tag portion with all but the least significant bit of each tag in the selected one set and indicates a hit if there is a match when operating in the first mode or the second mode; and allocation logic that, when the comparison logic indicates there is not a match when operating in the first mode or the second mode: allocates into any one of the 2^W ways of the selected one set when operating in the first mode; and allocates into one of a subset of the 2^W ways of the selected one set when operating in the second mode, wherein the subset of the 2^W ways is limited based on one or more bits of the tag portion; whereby the set selection logic selects two sets of the array using the index portion when operating in a third mode; the comparison logic compares the tag portion with each tag in the selected two sets and indicates a hit if there is a match when operating in the third mode; and otherwise the allocation logic allocates into one set of the two selected sets when operating in the third mode, wherein when allocating into one of the subset of the 2^W ways of the selected one set, the allocation logic allocates into one of the subset of the 2^W ways limited based on a result of two or more bits of the tag portion rotated by a number of bits specified by an allocation mode that is configurable. an array of 2^N sets each of which holds tags that each are X bits, wherein N and X are both integers greater than five; wherein the array has 2^W ways; an input that receives a Q-bit memory address, MA[(Q-1):0],, having: a tag portion MA[(Q-1):(Q-X)]; and an index portion MA[(Q-X-1):J]; wherein Q is an integer at least (N+J+X-1); a tag portion MA[(Q-1):(Q-X)]; and an index portion MA[(Q-X-1):J]; wherein Q is an integer at least (N+J+X-1); set selection logic that selects one set of the array using the index portion and a least significant bit of the tag portion when operating in a first mode or a second mode; comparison logic that compares all but the least significant bit of the tag portion with all but the least significant bit of each tag in the selected one set and indicates a hit if there is a match when operating in the first mode or the second mode; and allocation logic that, when the comparison logic indicates there is not a match when operating in the first mode or the second mode: allocates into any one of the 2^W ways of the selected one set when operating in the first mode; and allocates into one of a subset of the 2^W ways of the selected one set when operating in the second mode, wherein the subset of the 2^W ways is limited based on one or more bits of the tag portion; whereby the set selection logic selects two sets of the array using the index portion when operating in a third mode; allocates into any one of the 2^W ways of the selected one set when operating in the first mode; and allocates into one of a subset of the 2^W ways of the selected one set when operating in the second mode, wherein the subset of the 2^W ways is limited based on one or more bits of the tag portion; whereby the set selection logic selects two sets of the array using the index portion when operating in a third mode; the comparison logic compares the tag portion with each tag in the selected two sets and indicates a hit if there is a match when operating in the third mode; and otherwise the allocation logic allocates into one set of the two selected sets when operating in the third mode, wherein when allocating into one of the subset of the 2^W ways of the selected one set, the allocation logic allocates into one of the subset of the 2^W ways limited based on a result of two or more bits of the tag portion rotated by a number of bits specified by an allocation mode that is configurable.