I know a way people suffer because of HFT - the flash crash(es). Should the market fall, the HFT leave the scene. Joe Consumer suddenly finds that he has no one on the other end of the trading counter. The market is essentially an illusion. http://en.wikipedia.org/wiki/2010_Flash_Crash
The discussion page is actually quite interesting, and makes for an educational thread in itself.

Short summary -
HFT left the table after creating a mess. A large sale in e-minis started a trading pattern between where algorithms kept trading back and forth between each other, causing the prices to drop. After a point there was No liquidity and stub trades, which were never supposed to see the light of day, got executed. This resulted in some stocks going for a dollar, while others for many times market value.
Sometime after this markets trading was brought to a halt and the regulators stepped in. A large portion of the trades made were nullified. Effectively the markets were reset, order was restored manually, before it was brought back online. At this point the market returned to equilibrium and started recovering.

From the Wikipedia article, the SEC report on HFT is below:

Impact of High Frequency traders: Regulators found they exacerbated price declines. As noted above, regulators found that high frequency traders exacerbated price declines as they sold aggressively to eliminate their long positions and as they withdrew from the markets in the face of uncertainty.[8][9][10][11] Other theories postulate that the actions of high frequency traders (HFTs) was the underlying cause of the flash crash. One hypothesis, based on the analysis of bid-offer data by Nanex, Llc., is that HFTs send non-executable orders (orders that are outside the bid-offer spread) to exchanges in batches. Though the purpose of these orders is unknown, some experts speculate that their purpose is to increase noise, clog exchanges, and outwit competitors.[26] However, other experts believe that deliberate market manipulation is unlikely because there is no practical way in which the HFTs can profit from these orders, and it is more likely that these orders are designed to test latency times and to detect early price trends.[27] Whatever the reasons behind the possible existence of these orders, this theory postulates that they exacerbated the crash by overloading the exchanges on May 6.[26][27] On September 3, 2010 the regulators probing the crash concluded: "that quote-stuffing – placing and then almost immediately cancelling large numbers of rapid-fire orders to buy or sell stocks – was not a “major factor” in the turmoil." [28] Some have put forth the theory High-frequency trading actually have been a major factor in minimizing and reversing the flash crash.[29]

Do note that the person defending the HFT is a HFT trader himself - Jim Simons of quantitative hedge fund Renaissance Technologies. His contention is that HFT saved the day by brining liquidity back. I think this claim is simply wrong and his motivations dubious at best. If no HFT funds were there, and this had to be manually checked, the flash crash would not have happened.

They have since added circuit breakers to the exchange to prevent re-occurrences.