Cleaned up the layer shutdown logic a little to fix a crash.