For button mushrooms, you don't need to because they are tender. Bigger mushrooms may have woodier or tougher stems, so you may want to remove them.