If you're worried about damage to hair or hurting your little one when removing rubber elastics, we recommend a pony pick.  They are inexpensive and small enough to keep on hand.  Most rubber elastics come with so many in a package you don't need to worry about cutting them after each use.