If you delay signing up for a Medicare Part D plan you may end up paying a penalty when you begin your plan. This penalty will stay and remain as long as you have Part D drug coverage.