Google's goal is to make their users have the best experience possible. Clearly displaying succinct answers at the top of SERPs is what's best for the user. Keep in mind that some users may take note of your site if Google pulls your answer to the no-click search query, and may be more likely to visit your site in the future.