Improve type inference performance for binary operations by removing unnecessary overloads