Police said the fighting broke out between Croats disappointed by their team 's loss to Brazil and Muslims who had been cheering for Croatia 's opponents .