Neoprene works by creating and maintaining a thin layer of body-temperature water next to your skin. This water does not move away from you, which would carry heat. And very little cold water moves in to replace it. There is some leakage around wrists, zippers, etc. The neoprene rubber is also a good insulator because as a closed-cell foam it does not get water inside it, like a fabric does.
Neoprene also acts as a wind and vapor barrier to reduce convective heat losses.
It provides moderate abrasion resistance, some padding, and a bit of flotation. With divers this is not much of a problem, since they are completely immersed. Also, divers usually don't have to exert themselves as much as a caver doing a hard climb. So cavers can overheat in wetsuits, and ways to reduce this problem are discussed below. Drysuits have been tried for caving, but they are even less breathable than neoprene and they are prone to punctures in caves.