The ends are completely edible, and you don't need to remove them if you don't want to. The main reason most people do is because the tip of the stem end where the bean used to be attached to the rest of the plant can be a little fibrous.