Stop failing in destructors

I'm not sure what the right thing to do is here, but ignoring errors is
what the standard library does.