This can happen with some lemons, to avoid this try selecting thin-skinned lemon or Meyers lemons. The bitter flavour comes from the lemon pith or seeds, but it can happen with any form of citrus.