Im struggling to see the difference between this and the current FDA process, and think it is 90% the same.

Drugs have "prescribing information", referred to in industry as "labeling", which follows a consistent format containing safety, trial results, side effects, and mechanism of action.[1] I recommend people read them for drugs they take.

This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.

Third party analysis is the main difference here. In the current state, firms run the analysis for FDA review using standard practices, and must explain and get approval for any unconventional methods

>There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).

Labeling contains many relevant numbers. Trial sizes, how many per arm, what was measured, and and final results. Maybe there could be some squishy qualitative summary, but that seems more risky. I would rather know that 1 out of 20 patients died than it got a "2" on the safety scale.

>Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).

Labeling contains drug specific information.

There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.

Drug labeling requires pre-application and and a standard 12 month review period by the FDA prior approval

There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.

The FDA provides advice on structuring trials and acceptable design, size, power, endpoints. Firms do this by scheduling calls with FDA staticians and experts. [2]

And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.

Maybe there is a difference here. I'm not sure what you mean by punishment? In the current system, The FDA can use the label as "punishment". The FDA may require addition "black box warnings" for drugs that are found to have serious side effects (e.g. high chance of death). They can also pull the label entirely, meaning the drug can not be sold.

Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.

This is basically how it works for medical device labeling in the EU. There are several "notified bodies" [3] which are private agencies to review the safety and efficacy. The firm then takes their mark of approval to the government agency.

https://www.accessdata.fda.gov/drugsatfda_docs/label/2014/20...
https://www.fda.gov/media/72253/download
https://climedo.de/en/blog/list-of-mdr-certified-notified-bo...