Most cases settled because most people the RIAA
went after were guilty, and knew it, and realized
the the minimum damages they would owe if they
lost in court were quite a bit higher than the
RIAA's settlement offer.

That's exactly what they want you to believe.
That's why they pursue these lawsuits. It's
good old FUD. They can't sue everyone, but they
are hoping to sue a handful and scare millions
into obedience. It's pretty damn successful at
that, too. I know many people who won't download
anything for fear of a million Dollar lawsuit.

The people who have been sent the cease and desist letters and asked to settle have generally been sharing thousands of songs. The settlement offer is typically in the neighborhood of around $4k, which is not unreasonable--it's in the ballpark of what it would cost to pay for the download where the person acquired the song, and some for the downloads that others got from that person at the market rate for legal song downloads.

If the defendant choses to go to court, and loses, the minimum statutory damages are $750 per song, unless the court decides the defendant is an "innocent infringer" (basically someone who had no reason to believe that what they were doing was copyright infringement), in which case it can be lowered to $200 per song.

At $750/song, it only takes about 7 songs illegally downloaded and shared before taking the settlement is cheaper (just on the damages--I'm not even counting defendant's time and attorney fees) than going to court. At $200/song, the crossover is around 25 songs. This is far far under what most people who get caught were doing.

In a just system, the RIAA would be able to sue for me
for damages amounting to revenue lost - $20 per film they
catch me downloading.

The RIAA deals with music, not movies. As I showed above, the amount the RIAA offers to settle for is in fact reasonable under your definition--it is in the same ballpark as what the infringer would have had to pay to purchase the copies they download and share.

The damages one faces in court should be higher than that, for the simple reason that if the worst that happens in court is that you have to pay what you would have paid to not infringe in the first place then the risk of being sued is not a disincentive. You could just completely ignore copyright law, and when sued let the plaintiff win a default judgement and pay, putting you in the position you would have been in had you not infringed.

But somehow, somewhere, things went awry and people are getting
sued for absurd amounts of money. That alone should be a tell
tale sign that this is nothing but a big scam. Dressing it in fine
legal language doesn't change that.

The "absurd" amount of money comes from the law not anticipating that an individual would infringe thousands of copyrights for a purpose other than trying to make money from it, so statutory damages are designed to cover the range of damages that are appropriate for a commercial setting. Every RIAA defendant who lost big in court could have avoided that by accepting the first very reasonable settlement offer (or even more simply by not infringing in the first place). Hell, even after Jammie Thomas-Rasset lost in court and suffered a large judgement, the RIAA renewed their low settlement offer. After she rejected that, and got another trial, and suffered an even bigger judgement, they again offered a much lower settlement amount that she rejected.