While I agree with your first few paragraphs, and think that some of what you say could in principle be used in a skeleton argument mitigating against a large damages claim, it is unlikely that a downloader would be pursued as far as a formal claim for making a single copy, since the claimant is unlikely to recover all its costs even if it prevails in court. (Default judgments can be abused however; on the other hand it is probably more fruitful for sharp practice lawyers to pursue people who have downloaded something embarrassing with the threat of that being exposed on the claim form, even if there is no real intention to file the claim form to the court).

A downloader who has made lots and lots of downloads where the claimant has particularized the downloads and properly shown copyright in each them, might also escape with a low offer to settle, since the claimant still faces uncertainties if the defendant puts up a reasonable defence.

Instead, it is easier to aim for an uploader who has uploaded virtually uncountably many copies of a work. Statutory damages are often available in copyright law because if many many many copies are made of a work, it is hard to arrive at an exact quantum of damages, and few people want the courts to be burdened with assessing liability for each and every copy. Instead, statute often gives a "bulk" per-work figure which applies no matter how many copies are made, subject to conditions.

Faced with defending a claim in which one is alleged to be a serial uploader of many many copies of a work, there is no recourse via the type of argument you make. Sure, individual downloads (that one has allegedly uploaded) might be excused as non-infringing for logic similar to the argument you made (or more reliably, by an argument based in fair use or an equivalent, or some statutory exception). However, if there are still many downloads not so excused, the full statutory damages might still be sought and granted by the court.

An additional complication here is that the Dotcom matter is cross-jurisdictional and engages multiple aspects of the law. It is possible that the U.S. authorities sought N.Z. legal advice on what money amounts to use in pleadings, but it is equally possible that they have taken figures as they were made in U.S. proceedings, and can only justify them on the basis that they were made by some third party (and presumably should be answered in court by Dotcom, should he appear, which is at least one facet in the extradition action). A large figure also presumably justifies the ongoing costs the U.S. is incurring in trying to secure Dotcom's extradition, and whoever is making decisions in e.g. the State Department is happy with the $500 million figure ("Dotcom's REALLY bad, we should go after him still!"). On the other hand, extraditions for much lower-level criminals are frequently aggressively and expensively pursued by publicly-funded authorities.

> counterclaim ... accusation constitutes defamation

Statements made during the course of judicial proceedings almost always have strongly qualified -- or even absolute -- privilege in the face of defamation claims. It is hard to envisage any situation in which a court filing could be the basis of a successful defamation claim, unless its contents were overwhelmingly unrelated to the matter of the claim itself. A party can allege that the other party is engaged in abuse of process or perjury, however, and leave it up to the trial court to decide the matter.
Such an allegation must be accompanied by credible evidence, however, and the remedies tend to be proportional (up to striking out the offending party's claim or defence, with consequent liability for the other party's costs, but almost certainly not beyond that maximum).

If the sky-high claim form by a studio with copyright in, say, A Random Santa Movie were to also state that the defendant in a copyright infringement claim involving that movie also dresses up as Santa in order to abuse children and elves, then the studio could lose a defamation claim, for instance. However, if the allegation is that the defendant is a serial uploader of Santa-themed movies, that would not be actionable even if the claimant fails to prove that even one copy of A Random Santa Movie had ever been uploaded by the defendant.

In the law of England and Wales there are examples such as Seaman v Netherclift (1876) 2 C.P.D. 53.
(excerpts: http://swarb.co.uk/seaman-v-netherclift-1876/ ) and, for comparison, Munster v Lamb (1883) 11 Q.B.D. 568. (excerpts: http://swarb.co.uk/munster-v-lamb-ca-1883/ ). There is other case law covering such matters, however, and the Defamation Act may be engaged in some phases of proceedings. Generally, however, using a court of law to make a deliberately malicious statement with the intent to harm a person's reputation is far from good practice, but probably won't incur outright liability if it is sufficiently true and germane to the proceedings at hand.

Other jurisdictions will have different rules.