I do. Many recipes will advise you to remove the stems, which are edible but tend to be tough and fibrous. You can certainly do that if you wish. Personally, I find that the stems of small, young mushrooms are fine to eat.