Ah, I missed your tailing `if exit_status`, but the separate-thread approach avoids stomping on `$!` altogether:

Without:
~~~ sh
╭─{ yaauie @ beorn in ~ }
╰─○ ruby -e '
$stderr.puts "#{RUBY_VERSION}p#{RUBY_PATCHLEVEL} (#{RUBY_ENGINE})"
at_exit { puts "other: #{$!.inspect}" };
at_exit {
  puts "ours, begin: #{$!.inspect}"
  begin; raise "heck"; rescue; end
  puts "ours, end: #{$!.inspect}"
}
exit 1
'
~~~
~~~
2.1.1p76 (ruby)
ours, begin: #<SystemExit: exit>
ours, end: nil
other: nil
[error: 1]

~~~

With:
~~~ sh
╭─{ yaauie @ beorn in ~ }
╰─○ ruby -e '
$stderr.puts "#{RUBY_VERSION}p#{RUBY_PATCHLEVEL} (#{RUBY_ENGINE})"
at_exit { puts "other: #{$!.inspect}" };
at_exit {
  puts "ours, begin: #{$!.inspect}"
  Thread.new { begin; raise "heck"; rescue; end }.join
  puts "ours, end: #{$!.inspect}"
};
exit 1
'
~~~
~~~
2.1.1p76 (ruby)
ours, begin: #<SystemExit: exit>
ours, end: #<SystemExit: exit>
other: #<SystemExit: exit>
[error: 1]
~~~