> An improvement common to all topics is an RPG-like skill graph, broken down into the dependencies of each micro-skill. This is both to give students greater awareness of their progress relative to material, and also let them manage their own learning.

I love that! I would have benefitted from that a lot in school.

> The need to vet some ideas, while learning the efficiency need to trust the output of reliable groups whose results have already been widely accepted.

Your project sounds great! I'm curious if you like Dr. Alex Freeman's approach to a new type of scientific publishing system, with her http://Science-Octopus.org (aims to make science more accountable and open access):

"Q: What problem are you aiming to tackle?

A: Fundamentally, I believe that the whole concept of the scientific paper drives people to tell stories rather than worry about the quality of the actual research. It's true that during the scientific process, you usually start with a problem and you finish with a real-world use of your solutions. But the whole chain, from one end to the other, takes a whole range of different skills and different people and different amounts of time. Forcing researchers to guard their work and get to the end of the chain before they can publish anything makes people work in silos, when we'd all be much better off if everybody shared their work as they went along and everybody collaborated to get to the end of the process.

There is a real problem in science—with widespread questionable research practices, lack of replication, publication bias, inequalities in access, slow progress, and wasted resources. And I think that the root of it all is the publication system, and the pressure to publish as the only way that scientists are being judged. The new publishing platform I envisage would stop forcing researchers to try to tell a story, allowing them to concentrate on doing their work to the best of their ability and on constructively collaborating.

Q: How would this new publication system work?

A: The overall idea behind the new platform, which I called Octopus, is to break the standard unit of publication up into eight smaller stages or pieces. These include formulated scientific problems, hypotheses, methods and protocols, data, analysis, interpretation, and real-world applications. The eighth piece is reviews where people share their comments about the other stages, but it's also treated as a publication itself because it is a constructive piece of collaboration. Researchers will be able to revisit their publications to acknowledge others' comments or suggestions and even include some of the useful reviewers as authors, with the old version remaining available so that readers can track the evolution of ideas.

The formulated scientific problem automatically becomes the beginning of a chain, and all the other pieces are vertically linked upward. If you have a great hypothesis, you can publish that and link it straight up to the problem that it is trying to address. And if you've collected some small amount of data, you can publish that, linking it directly up to the protocol that you followed. It will also be possible to create horizontal links between publications; maybe somebody has come up with an amazing algorithm whilst studying starling flocking behavior and somebody else working on oil pipelines has decided to use it.

Octopus will thus allow researchers to publish in much smaller author groups, and that can make the system much more accountable and meritocratic. If, say, you're an applied statistician, at the moment you're very rarely a first author on anything. But with Octopus, statisticians can do analyses of anybody else's data and get credit for it as the author of that publication. Octopus will also be made language-agnostic through automatic translation to lower language barriers and broaden access.

Another way to make sure that we are judging people on the actual quality of their work and not on the stories is through rating. Anybody who's logged in to the system will be able to rate entries as soon as they are published based on pre-set criteria reflecting what we as a scientific community value at each stage of the scientific process. So, for instance, there might be a rating for the size of the dataset, and another rating for how well it is annotated and how usable it is. Just like on eBay or Amazon, the most highly rated pieces of work will automatically become more easily visible.

Octopus will also allow the evaluation of individual authors through a professional page that shows their affiliations, their conflicts of interest, and all their publications and their ratings. Seeing these metrics, you will be able to get a feel for what type of researcher this is. Are they publishing lots of well-rated hypotheses? That's an idea-generating person. Are they writing a lot of constructive reviews? That's a very collaborative person."

Source: Meet Octopus, a new vision for scientific publishing: https://www.science.org/careers/2018/11/meet-octopus-new-vis...

Talk: Dr. Alexandra Freeman | Octopus - a radical new approach to scientific publishing | 29 October 2018: https://www.youtube.com/watch?v=Af6aITLEoD8

Or have I misunderstood your aims/interests and you are not interested in open access?