That's not the base problem, I don't think.

People only want to mete out punishment to people in the out-group, not to people in their community.

The late criminal law professor William Stuntz compiled an incredible amount of historical conviction data for his book, "The Collapse of American Criminal Justice". What he found is that conviction rates have been steadily increasing for the past 200 years.

One of the threads he identifies is that when the jury has a personal interest in leniency or rehabilitation of the defendant, they're lenient. Leniency was much more common 100 years ago, and took a nose dive in the mid- and late-20th century.

For example, the conviction rate and severity of punishment of black men in the Deep South was comparatively low both during slavery and during Jim Crow. Why? Because black men were generating income for whites, either as slaves or sharecroppers. Convictions and jail time were disruptive. (The exception that proves the rule is when the state began to sell prison labor, though the effect was attenuated--juries' economic relationship to the prison labor system was more remote.) Now that black men have been so thoroughly excluded from the labor market, it should be no surprise that conviction rates and severity of punishment have outpaced that for other groups, even after accounting for racial animus. (Or rather, racial animus is unchecked when you don't even have any meaningful economic or social interest whatsoever.)

Another example is immigrant communities. Because of the configuration of populations in urban areas as well as historical court systems, jury pools were often drawn from the same ethnic community as where the defendant committed his crime. It might seem counter-intuitive, but the same community that suffered most directly from a crime also tended to be much more lenient. Even conviction rates on crimes like homicide were ridiculously low compared to now. Why? Because 1) juries were more understanding of extenuating circumstances (i.e. a bar brawl where everybody know the deceased had it coming) and 2) because they knew that sending the defendant away, especially if they were the bread winner of a family (son, husband), would be detrimental to the well-being of that family and even the community (i.e. they come back hardened by prison). The OJ Simpson trial is a rather grotesque example of this phenomenon which most white people can't even begin to understand, and it doesn't help that the victims were white. But the general pattern (not the motivations) was common historically even when the victim came from the community.

With white flight to the suburbs and with the centralization of courts, modern jury pools are drawn from many different local communities, and disproportionately from communities economically and culturally more removed. These communities are afraid of crime spreading, and are more likely to want to send a message by being harsh. And because the defendant is unlikely to come from their community, they're much more removed from the externalities caused by harsh sentencing, either of the individual case or from systematic harsh sentencing. Though the externalities are significant in the aggregate and clearly effect all of us.

And I think the opposite phenomenon is also possible--that is, an overcorrection to being too lenient. In the case of California's decriminalization of drugs, many people have been arguing that some amount of jail time is absolutely critical (for various complex reasons) in helping with rehabilitation. But people who don't see the drug culture up close don't understand the nuances. So we're likely to see big shifts from being too harsh to being too lenient. Without the social and economic relationships, feedback is too attenuated. People can't understand the situation, nor do they have any specific interest--mostly they're following abstract arguments and trends.