That's part of the beauty of Champloo: even the strongest of characters have foes stronger than them. While Mugen certainly defeated many strong opponents and grew over time as a samurai, he was unable to land a single blow against the hand of God, the final foe of the series who ultimately loses to Jin.