To assess their automation expertise, review their experience with tools like Selenium or QTP. Ask for examples of test scripts they've written and discuss how they've implemented automation in past projects.