Override just the one side padding

The previous implementation overrode all line paddings unnecessarily and broke the line number positions.