New York caves are often wet, thus, keeping warm is your next concern. Therefor, if the cave you plan to visit is going to be wet, then you should be very selective in the clothes you wear. Cotton is out, you should wear some type of synthetic or wool. Polypropylene long underwear is very good if you have it. Wool or woolblend pants, shirts, and sweaters are also good.
Wool socks are a necessity.
You should also consider that you are going it be crawling and climbing in a restricted area so a comfortable fit is also important. Make sure you bring a FULL change of clothes to change into after. Boots are good but sneakers are also fine. Have a second pair to wear home. These can easily be put in your helmet out of your way.