Move debugger logic out of specFilter to fix #659

See also discussion in #702