In a group conversation, in particular, many hearing-impaired people miss the first few words of the conversation before they know someone is talking and then can identify the speaker. Unlike people with perfectly normal hearing, a hearing loss may affect their ability to quickly localize the source of a sound, which interferes with their ability to immediately focus their attention on the person talking. If they can't do this, then clearly they are not going to be able to use speechreading clues. When, for example, the hearing-impaired person is engaged in another activity, he or she may not even realize that someone is talking until well into the next sentence. So, it is a good idea to get the person's attention before starting to talk.
People do understand more when they are prepared to listen.
This is probably the most destructive practice of all for most people with hearing loss. Consider a dinner or meeting with six people around the table. It is simply impossible, for example, for the hard of hearing person to talk to the person across the table, while the people to the right and life of him are having a separate conversation. The rule here, frequently broken but nevertheless which has to be attempted, is to structure the event so that only one person talks at a time. A gentle reminder, if breakdowns appear to be occurring too often, is not unreasonable.