This is a tough question to answer because each weight class has its own challenges. Also, if we go by which weight division is the most competitive with the most talent, that can fluctuate over time. Right now, the lightweight and welterweight divisions are probably the most talented, so they could be the hardest for a boxer making their way to the top. However, traditionally, the heavyweight division is considered by many to be the hardest boxing weight class there is.