Meteors large enough to do damage on a local scale probably hit the Earth a couple times a century or so. In the 1910s, something exploded in the air above Tunguska, Siberia, destroying trees for many miles around, and killing a couple of people. If that object had hit New York City, the entire place would have been destroyed completely. Luckily, though, the amount of the Earth that is covered by people is very small, so most localized impacts won't do much damage. The rate of impacts large enough to cause large-scale catastrophes is unknown.
Estimates range from once every million years to once every few hundred million years.
The only thing we can say for certain is that asteroids and comets DO hit the Earth, and those that do must cause serious damage. I think that, as humans, we would do all in our power to prevent an impact and to protect ourselves if we could not prevent it. The costs of looking for dangerous asteroids and comets are also low - a few million dollars a year, or a penny from every man, woman and child on the planet. So, as long as we keep a watchful eye on the sky, there is no need to worry ourselves about an asteroid impact. We take the necessary precautions, and when it comes, most people will be ready.