Those who fixate on heroic fantasy to the exclusion of other books may indeed have done so because they found the genre at an impressionable age, or because their own lives weren't varied or evolving. A very few have had direct adventures of the heroic kind. One of the more intellectual Conan fans I ever met was a Jesuit professor of navigation who had been a Navy SEAL in Viet Nam. His task was to bring back American captives from behind enemy lines. This guy never suffered any of the post-traumatic stresses we so often hear about.
Life is short even if nothing happens to abbreviate it further.
Dangerous as life may be, there's yet a lot to be said for it, a fact that increases the tragedy of its brevity. It can be read escapistly, but it is not inherently escapist. In our daily lives we aren't apt to get in a duel of steel, but anyone who passes through this life without dangerous encounters of any kind has got to be a milquetoast holed up in a dreary cell. Heroic fantasy is many things to many people, some of those things questionable or sad. But in most cases it is not indicative of stunted reading tastes.