I've been doing this for more than 17 years. Some thoughts:

1. Anything that makes a real change will have side effects. Some of those side effects can be negative. For people not familiar with the concept of a healing crisis, this can be a confusing fact. They can end up temporarily feeling worse and conclude this is a bad thing rather than a temporary stage of a process headed in the right direction.

2. Prescribed medicine typically has a long list of provisos and side effects. If you have a serious condition, prescribed medicine trades short term gains for long term costs. Doctors claim credit for the short term gains, then blame the long term costs on your condition or age rather than on long term drug use. You need to be leery of the possibility that you are doing the same sort of thing and crediting any positive changes to this dietary change and dismissing any negative changes as coincidental or due to something else.

3. I've seen people post questions to a discussion forum along the lines of "What are non drug alternatives for treating X issue? I would like to participate in a drug trial and they won't accept patients taking this drug. I need to get off it to qualify." So people doing stuff like that are confounding the results of the trial and will report changes in symptoms and attribute them to the drug being tried rather than attributing them to getting off some other drug and using non drug alternatives.

4. In practice, it's quite hard to isolate one specific dietary factor because dietary changes typically involve at least two factors. If you add some new food to your diet and eat the same calories, it displaces some other food. You can't say for certain if the changes are due to adding the new food or removing what was replaced. If you are experimenting with your own diet, it helps to start with supplements, make one and only one change at a time and track things somehow. This makes it possible to get some idea of what effect X is having.

5. It's a moving target. If you are calcium deficient and add calcium to your diet, this may be exactly what you need to start feeling better. Initially, more calcium may be better. If your mental model boils down to "more calcium is better," you may wind up resolving the deficiency and then enter a situation where you are actually consuming too much calcium. This can lead to problems that you may not attribute to consuming high levels of calcium because you already decided "more calcium is better." You may develop new issues and utterly fail to relate them to consuming high levels of calcium.

6. People have a really hard time mentally modeling the road not taken. It isn't completely impossible, but people tend to be bad at it. If you have a specific diagnosis and you know the usual course of the condition, you can compare your health to the typical outcome and make some inferences, but it isn't absolutely conclusive. This is why twin studies are valued: We can't make two copies of you, but if there are already two people with identical genes, that's the closest we get to that.

7. Life is chemistry and there can be myriad other factors being overlooked. I try hard to track not only diet, but how much I walk, environmental factors, etc. I've done this for years and I still find myself blindsided at times and struggling to pinpoint why X is happening currently.

Most people aren't going to be able to read hundreds of studies. If you can and that gives you a mental model that makes sense to you, awesome. That doesn't actually strike me as a convincing rebuttal. It strikes me as more evidence that it's actually rather challenging to figure out dietary stuff. You won't solve it by reading an article or two.