Set sb-ext:*invoke-debugger-hook* in addition to cl:*debugger-hook*.