Ahl Kitaab/mushrik
Due regard would make sense if they were simply making a mistake. But they knew full well after hujjah that what they were doing was shirk
So it seems they were simply punished because of what they held as a belief rather than their arrogance