You're getting stuck on the abstraction of "knowing" something, treating it as if it were binary. We do an RCT with 95% confidence, and then we "know." Before that, we don't "know."

There isn't such a thing. We know nothing. We have different levels of confidence in facts.

Even very basic facts ("am I alive") I can only say with pretty high confidence (we might all be living in a computer simulation or somebody's dream). And even things with strong scientific evidence occasionally get disproven. Studies with just one RCT are wrong all the time -- most medical and social sciences studies are wrong (it turns out p=0.05 is nowhere near sufficient in established research fields).

You work through decisions with uncertainty by calculating an expected gain or expected loss:

https://en.wikipedia.org/wiki/Expected_return

The argument for taking hydroxychloroquine is not the same. There is weak evidence it might do something, but the drug also has a whole slew of negative effects (https://en.wikipedia.org/wiki/Hydroxychloroquine#Adverse_eff...). If you compute the expectations of taking hydroxychloroquine prophylactically, it's definitely a very, very, bad idea. The downsides obviously outweigh the upsides.

If you do have a serious case of COVID19, it depends on what assumptions you plug into the equations. Which is to say, I wouldn't recommend hydroxychloroquine based on the last data I've seen (although I haven't followed this closely in a while), but reasonable people looking at the same data (again, ignoring anything from the past month or so) might recommend it. It's not an insane decision.

On the other hand, the data strongly supported ramping up the production of hydroxychloroquine and stocking up on it in case it is eventually shown to work. Hydroxychloroquine costs maybe 30c per pill, and dropping a billion dollars ($1/person) to make sure we have 3 pills per person is an obviously good idea. That's less than 1/1000th the stimulus package. Likewise, we should have used face masks immediately (including a massive push to surge manufacturing) before we knew what effect they had. We have enough data from e.g. influenza to where they were an obviously good idea from day 1. We should have done likewise for manufacturing of ventilators (which are turning out to be less helpful than we though), and for a slew of other things which /might/ be helpful down the line.

For most meds and supplements, we have zero evidence. There's no reason to believe they do anything, and if they do, the odds of it being negative are the same as the odds of it being positive. Don't do it. I think the one exception is zinc, where based on very incomplete data, it seems like it makes sense if you're sick.

Military, financial, and business are domains where there are sophisticated models for how one makes rational decisions with incomplete information (before "knowing"). If you wait for complete information before moving, a competitor or adversary will crush you.

Coincidentally, a lot of people were bothered by shutdowns happening before we knew how serious COVID was or how it spread. Those were also an obviously good idea. But the confidence interval for CFR ranged from a slightly more serious flu all the way up to maybe 8% at the time we shut down here. The _expected_ CFR was 3.4-3.6%, and the shutdown was an obviously good idea based on that incomplete information. But news sources omitted how little we actually knew.