If he was confident in his fist-fighting abilities and placed an exorbitant bet (the maximum being 70 crowns), that's a good chunk of coin to forfeit so early on. Unless you bet the minimum of one crown, it's not suggested that Geralt throw the fight.