Now that we depend on GCC >= 4.8, we can drop the noexcept from the dtors.