If the Google Phone app came as in-built on your phone, it is impossible to remove it. However, you can disable it. If you installed it from the Play Store, you could easily delete it. In both cases, you can download an alternate dialer of your choice.