Yes. This is usually the graduation moment that all cyclists go through, but I'm afraid to say cycling shorts are to be worn without underwear. The chamois pad is designed to sit directly on your skin, especially if chamois cream is involved, and adding a layer of cotton (or whatever other material your underwear is made of) creates the opportunity for saddle sores as the fabric may crease and fold, leaving pressure ridges.
What's more, cotton especially is very absorbent, and so not hugely hygienic.