There's a place to turn off and view them and to read the historical marker . The lobby still has the marble teller area installed in the thirties, and the horns from a longhorn grace the entrance to the vault. If you stop there, try the onion rings and check out the tin ceiling.
Small parties can reserve the kitchen and front for get-togethers.
There is also a more extensive park with a pool on the West side of town. It's a beautiful old stucco and terra-cotta building. Inside, there's a mural entitled Cowboys Driving Cattle by Kenneth Evett. The 1912 Carnegie library still serves Caldwell and has a good collection of Kansas references.