Polypropylene long underwear is very good if you have it. Wool or woolblend pants, shirts, and sweaters are also good. Wool socks are a necessity. You should also consider that you are going it be crawling and climbing in a restricted area so a comfortable fit is also important. Make sure you bring a FULL change of clothes to change into after.
Boots are good but sneakers are also fine.
Have a second pair to wear home. These can easily be put in your helmet out of your way. CLUB GEAR Caving Helmet. These wonders of nature have taken tens of thousands of years to develop. One careless move by a caver can ruin a formation that has taken thousands of years to form.