While the top layer of the orange zest (peel) has some bitterness from tannins, it's the white part beneath the skin  - the pith - that is very bitter. By removing much of the pith, we can avoid most of the bitterness. Another option is to boil the peel several times which I'm too lazy to do :)