Yes, both upper and lower thermostats can be checked for continuity using a digital multimeter. By placing the probes on the terminals of the thermostat, one can determine if an electrical current can pass through.