I generally find the idea of "alien comes to visit" is ridiculous, but I have to acknowledge that UFOs, specifically, the subset of it, "non-natural, artificial-looking, non-illusion (hey, they even appear on the radar!), real flying objects in the lower atmosphere that cannot be built by the current human civilization" is a true phenomenon after reading these reports.

But if they can be detected by a radar (at least some of them), it means their existence is not completely beyond the known realm of current physics. Doesn't it mean we can/should do lots of more objective observations of these phenomenon? For example, what are their infrared radiation patterns? What are their EM wave reflection patterns? What do they look like on the optical spectrum, what happens when you point a laser toward it, etc...

So why don't we see people doing that? Especially the military, because they always have enough funding to carry out a large experiment without immediate purposes, such as the HAARP, without assuming conspiracy theories, it's just an array of large and sophisticated HF transmitters, which just happened to be the subject of conspiracies. I think many military installations can be as suspicious as HAARP in the eyes of conspiracy theories, such as those VLF radio stations for communicating with nuclear submarines (conspiracy theorists can describe as them a mind-control device), or those phased-array for radio astronomy (conspiracy theorists can say, if moon landing is fake, why are they still launching these facilities? To create artificial natural disasters to install the new world order, or something... well, can I make a million dollars by becoming the next Alex Jones now?), but it just happened to be HAARP, rather than something like SuperDARN.

Jokes aside, one possible explanation comes to mind is that all researches are classified, and then it comes with full of conspiracy theories that rarely make sense. However, there are quite a few widely-documented UFO spotted by the public, which would allow the much larger civilian researcher groups to collect data, and these activities are very unlikely to be "silenced" as well, since they are already public. Another answer is that the researchers are biased, it may be true, just like the "classified military" explanation, but I don't think it can explain 100% of it. Military cannot classify everything, and not 100% of the established researchers are so biased that they even refuse to check it with some equipment (which is, as well, essentially for concluding that these phenomena are just illusions, not real).

I think the ultimate answer is their unpredictability. Their occurrence is essentially random. You cannot observe something with sophisticated equipment if you are not prepared to, even if you can, you cannot successfully establish a influential project to do that. An example would be ball lightning (https://en.wikipedia.org/wiki/Ball_lightning), most scientists before 1960s don't believe it's a real phenomenon, because no objective observations can be carried out, not even a single instance has been capture by the researchers until recently, in 2014.

I think the same goes for UFOs. And if you do see UFOs occur in a predictable manner, it's very likely that they are secret military aircraft or misidentified natural phenomena than "non-natural, artificial-looking, non-illusion, real flying objects in the lower atmosphere that cannot be built by the current human civilization"-kind of UFOs. On the other hand, imagine if there's a predictable occurrence of this kind of UFOs at specific locations around the world, there may even be recognized academic institutions to research them.