Duelists might both consent to a duel, but they'll still be committing a crime in every state. Even states that permit prizefighting require a state-granted license, and without it, consent will not save the participants from prosecution. Levels of violence are also important. We as a nation have decided that there are limits to the levels of violence we can tolerate, and the professional sports we allow conform to those limits. Finally, sports have rules and governing bodies to control the violence.
Most sports have similar arrangements.
And beyond the rules, there is the nearly boundless discretion of officials. Referees and umpires can eject players, and leagues can suspend them and impose forfeits and fines. These are all good reasons for letting ballplayers play ball. And most athletes conduct themselves like law-abiding citizens on and off the field. So the starting point, when we think a player might have stepped over the line, must always be the rules of society, not the rules of the game.