Remember, in most cases, if you don’t enroll in Part A (if you have to buy it) and Part B when you’re first eligible, your enrollment may be delayed and you may have to pay a late enrollment penalty. You can sign up for Medicare during your Initial Enrollment Period or during a Special Enrollment Period if you meet certain conditions. Additionally, you may have a gap in coverage if you don't have other insurance.