Properly deal with adding/removing breakpoints