Samuel eventually responded to God's call and became one of the most faithful prophets in Israel's entire history. I don't believe he was a perfect person. He probably told an occasional fib to his friends and participated in other sins that are familiar to all of us. There are probably as many reasons for not hearing God's call and not seeing His activity around us as there are sins in the world. In addition to the discouragement of those Kentucky pioneers and the conclusion jumping of Samuel, some people deliberately filter out any sounds other than those of their own choosing.
Some people look in the wrong places.
An inebriated fellow was crawling around on his hands and knees one night under a street lamp. Sam said he had lost his keys. I'll help you look. Some people hear God's call but only half-listen, so they misinterpret and misunderstand. It is possible that Moses saw it.