Most people just attach the pump to their belt or waistband using a clip or case, or keep it in their pocket. If this is not convenient, e.g. if you're wearing a dress, there are other options such as attaching it to your arm or leg under your clothes. At night, you can either just put the pump next to you in bed, put it under the pillow, or attach it to your clothing. It will be quite safe, as it can't go anywhere. The pump is very rugged and will resist being dropped on the floor, or the occasional soaking, though it's wise to protect it from such things where possible.
Some pumps are even waterproof.
For vigorous sports, you may have to devise a safer method of wearing it, or disconnect. The number of people using pumps varies a great deal between countries. In Germany about 1 in 7 people on insulin uses a pump. In the US, it's about 1 in 8. In Britain it's about 1 in 1000.