CMB lensing by Taylor expanding the field relative to the nearest pixel