Do not try to close the ALSA handle if it was not even opened.