First of all, you should know that the caves in Indiana are about 52 to 54 degrees F year round. Your clothes will get dirty. Don't wear your favorite sweater and your roommates khakis, because they will not look pretty after your trip. You should wear old jeans, a T-shirt and a sweatshirt. You may also want to consider a pair of coveralls.
Overalls are a bad idea, because they are going to catch on rocks and such.
Coveralls are best though. Gloves Gloves are essential, since you will want to have your hands warm and clean. I use washable leather work-type gloves. This keeps my hands pretty dry and I like the leather because it breathes. Knitted gloves will get wet and cold.