They know about usability, at least more than the average bear. In my opinion, this is both stunning and scary. Some people may think the problem was their browser, their Internet connection, or their computer. Users may not feel comfortable reporting perceived problems. They may see it as an insult to the designer or webmaster.
People are apathetic.
It's too much trouble to report it, or not enough time. Beverly certainly has some excellent points. Many Web developers, including usability professionals, are lulled into thinking that a feedback link will catch problems. We can make excuses and we can develop theories as to why users don't give us feedback. You are mainly responsible for effective user testing.