Run the supply with the lamps connected for about half an hour or until the case of the supply gets warm. It should measure almost exactly 5 volts. A reading lower than 4.9 volts usually means the capacitors in the supply are failing. On the other hand, if the voltage starts rising, going up past about 5.2 volts, it means the regulator chip in the supply is failing. That supply should measure 5 volts even with no load on it.
Excessive voltage can and does damage the computer.
I've seen bad ones that would always work when cold, then fail when they warmed up. The lamp on the 5V line will get very bright if the regulator shorts because the output voltage will rise to about 11 volts. A leaky regulator will usually cause the voltage to rise slowly, so you need to monitor it with the meter as the supply warms up. The only thing to look for with the 9VAC is it's presence or absence. Loss of the 9 volts AC will not damage the computer.