button: fix the border calculation when drawing text

The border is remove above the code so there is no need to remove it
twice.