The drill takes care of drilling and boring grinds much better than an impact driver.
Some companies even design drill bits specifically for the impact driver.
So you’ll achieve better results with a drill.