Tested GPSFactor. Re-Corrected the the scale part of jacobian based on numerical derivatives. Works perfectly now.