During a massage, you should be comfortable, and this includes whether you are undressed or not. You can undress based on your comfort level. Many people undress completely while others keep on undergarments.