With each level above -O3 an additional set of optimizations will be performed. This reduces code size and increases speed as well. This generally doesn't matter anyway. This will reduce code size as well, and will increase execution speed. The disadvantage of this is executable size, of course.
Nothing comes from nothing.
There is also a alternative method for handling exceptions. Exception won't cross the boundary between different-style exception handling code, so most likely you will got a 'unhandled exception'. Your executables will become a bit larger, but this is a better choice if you have a small number of executables compiled with current version of pgcc.