It is not that simple. That's because hate and hate crimes are rooted in prejudice and prejudice has long been a part of this world. You don't legislate centuries of ideology out of people. For every McVeigh who is caught, for every person jailed for a hate crime, there are others vowing revenge. That is the biggest scare.
Hate never goes away, it just comes out in different people in more violent means.
If we want to eliminate hate crimes, we must first start with prejudice and that starts with all of us. We have to figure out why we hate. Take it further. Most of us cannot answer those questions. That's good because most of us don't feel that way.