You should undress to the level you are comfortable. For a full body massage, most get completely undressed. However, if you will be more comfortable during the session if you leave your underwear on, that's fine. The therapist will work around the clothes you left on as best as he/she can. If removing all your clothes makes you too nervous and unable to relax, then you are not getting the optimal benefit from the session.
Your massage therapist should give you privacy to undress and get comfortable on the table.