Extract formatter backtrace handling

First pass at #1505, opening for feedback.