TITLE: University Published Paper Reproducibility Assessment Protocol - For students
AUTHORS: Scott Edmunds, Jesse Xiao, Hongling Zhou

[DESCRIPTION]
<div class = "text-blocks"><div class = "text-block">This protocol is for students carrying out a literature/data curation exercise to crowdsource and determining whether the data availability of papers from a university meets the national, funder, university and journal policies. The teachers setting up this assessment have a separate related protocol they need to follow, and they will ultimately collate and mark these results.</div></div>

[STEPS]
?. Find the assigned papersEach student should have been assigned a number of papers in a spreadsheet that they need to assess.
?. Find your assignments, and start the exercise by looking at the first assigned paper,  clicking on the identifier (handle, URL, etc.) and locating the paper to assess via its DOI.
?. Assess the assigned paper to see if there is dataQuickly scan through the first paper on the list alongside any supplemental files to look for mentions of any supporting data. If there is no data in the paper (e.g. it is an opinion article, an erratum, covers a purely theoretical subject, or some other type of paper purely discussing rather than carrying out data production) then mark it as not having data and go to the next paper on the list.
?. If there is data, find it in the paper
?. You will not need to properly read and digest the contents of the paper, but aim to spend 5-10 minutes going through the manuscript, trying to understand what type of experiments have been carried out, and what likely data has been produced in the process of publishing the research.
?. In particular, concentration should be given for checking the Methods, Results and if there is a specific "Data Availability" section in the paper. PLOS for example has a “Data Availability” section in their structured abstract/article front page and a "Supporting information" section at the end of their papers.
?. Make a note if all of the data is in the manuscript itself (figures, tables or included supplemental files) or accessible via external sources – e.g. via database accession numbers or linked via URLs or data DOIs to data repositories such as figshare, zenodo or Dryad.
?. If data is only available on request make a note of the reasons given and details of who to contact to access the data. Having access to the raw data is most important, so make a note in the comments section of the spreadsheet if that is available.
?. Check external sources for dataIf external data sources have been mentioned, e.g. database accessions or URL/DOI links to datasets, check if these are working by clicking on the URLs or finding the online databases mentioned.
?. If a database and accession number is listed, access the database mentioned in the paper and search it for the accession number (input the accession number and click search)..
?. If a URL or DOI is listed in the paper as being the source of the data, test to see if it works.
In both cases test if the datasets are available for download and there is not just a landing page
?. Summarise results on whether data is availableOnce you have checked the paper and any supplemental and external data sources for data and input all of this information into the spreadsheet, make an overall rating on whether you feel the data is available without restriction, if data is available on request, or if data is missing. Once this example is complete go to the next paper on the list and assess that in the same manner. Once this literature curation exercise is complete send the results back to the teacher to be marked and collated alongside all of the other crowdsourced results from the other students.