Remove unneeded debugger gem

It was not bundling correctly and causing CI problems.