Ignoratio elenchi (also known as irrelevant conclusion or irrelevant thesis) is the informal fallacy of presenting an argument that may in itself be valid, but does not address the issue in question. "Ignoratio elenchi" can be roughly translated by ignorance of refutation, that is, ignorance of what a refutation is; "elenchi" is from the Greek , meaning an argument of disproof or refutation. (Some sources give by ignorance of the issues or even by ignoring the issues as a translation of ignoratio elenchi.)

Aristotle believed that an ignoratio elenchi is a mistake made by a questioner while attempting to falsify a respondent's argument. He called it an ignorance of what makes for a refutation. For Aristotle, ignoratio elenchi amounts to ignorance of logic. In fact, Aristotle goes so far as to say that all logical fallacies can be reduced to what he calls ignoratio elenchi.

Modern use limits this term much more narrowly to the kind of mistake described in the first paragraph above.

Red herring
Similar to ignoratio elenchi, a red herring is an argument, given in reply, that does not address the original issue.  Critically, a red herring is a deliberate attempt to change the subject or divert the argument. This is known formally in the English vocabulary as digression which is a neutrally connotated "Red herring".  The actual problem is eclipsed by the process of re-focusing on something else.

Examples
Bill Maher on Scarborough Country

Maher: It's arbitrary, isn't it? If you had been born in Pakistan, you wouldn't be believing in Jesus Christ.  You would have been told another fairy tale and you would have been believing that.

Scarborough: Well, Bill, that's your opinion.

Whether Maher's argument is his opinion or not is irrelevant and does not address the argument made.

"Baseball player Mark McGwire just retired. He's such a nice guy, and he gives a lot of money to all sorts of charities.  Clearly, he will end up in the Hall of Fame."
The conclusion is ignoratio elenchi, since friendliness and charity are not the main qualifications for induction into the Hall of Fame.

"I should not pay a fine for reckless driving. There are actual dangerous criminals on the street, and the police should be chasing them instead of harassing a decent tax-paying citizen like me."
The existence of worse criminals is a secondary issue which has no bearing on whether the driver deserves a fine for recklessness.  If the speaker were deliberately attempting to divert the issue, this would be an example of a red herring. While the argument about how the police should spend their time may have merit, the question of whom the police should prioritize pursuing and the question of what should be done with those the police have caught are separate questions.

"The prime minister's tax policies may be popular, but I suspect he had an affair and is paying the woman to keep quiet. The media should investigate that! "
A red herring, the unrelated alleged affair, attempts to change the subject away from the popular policies.  However, if the original discussion were of the prime minister's public integrity (encompassing both popularity and conduct), this argument could be perfectly valid. Such as if a politician's election campaign revolved around family values. Then an argument about such an affair would be valid, as it is in direct conflict with a family values platform.

References

Other websites
 About.com: Fallacies of relevance
 Nizkor Project: Red Herring 
 Fallacy Files: Red Herring
 Infidels.org: Ignoratio elenchi
 The Art of Controversy: Diversion  (bilingual with the original German) by Arthur Schopenhauer
 Red herring in political speech
 Ignoratio Elechi on TV Tropes

Red Herring
Logical fallacies

nl:Drogreden#Ignoratio elenchi