No, there have been no really large impacts during recorded history. Of course not. You wouldn't be here to read about it if there had been. But there was a 15 megaton event over Siberia in 1908 that would have flattened the whole of London if the circumstances had been only slightly different. And Chinese records tell us of a meteorite explosion killing tens of thousands in the 15th century.
Statistics can be misleading.
Take air crashes. I'm a member of the Qantas frequent flyer club, an airline that has never had an accident. Circumstances can change quickly. Until recently Concorde was regarded as a paragon of air safety. Two decades ago virtually everyone, weather forecasters included, thought that hurricanes never hit England.