I can only do a hard reset. You are probably using Coaster on a PowerMac 6xxx or PowerMac 7xxx. There have been reports with the symptoms of Coaster hanging the Mac during recording. This stems from an unresolved hard- or software incompatibility on these models. The only solution is not to use Coaster on these machines.
Coaster uses the sound input manager for recording.
Manager compliant, there should be no problems as long as the driver supports installing a sound input interrupt routine with not too big a buffer and the ability to do asynchronous recording. I cannot set the input to my third-party sound input card in Coaster. It only shows the icon of the normal Mac input and no popup. Within Coaster, you can only set the sound input source of the current active sound input device. It should now show the name and icon of the third-party card and you should be able to record from it.