This is completely up to the rider in question! You will want to figure out if you are comfortable, and have the ideal handling that you want from your bike. If you feel like you need the handlebars closer to you, then yes, you probably should get a shorter stem size.