There should be at least 5mm of room between your stem and handlebars when you're adjusting them. This means if you have a headset with a 1" threadless steerer tube, there should be at least 5mm between the top of your stem and where it meets your handlebars.