Revert "Work around weird behavior in GCC 4.8 where it thinks this destructor's declaration and definition hase mismatching noexcept specs even though neither one has a noexcept spec at all."

This reverts commit c088dd95c610eb34381190fbad7e20ca68fff323.

Reverted in favor of dwrensha's more complete fix.