dataesr/scientific-paragraphs-categorization huggingface dataset:
In 2006, the World Health Organisation launched an international initiative to establish the International Clinical Trials Registry Platform. The platform provides one-point access to different registries across countries. In 2007, the FDA Amendments Act (FDAAA) expanded the registration requirements -for all drugs, biologics and devices subject to FDA regulation, the FDAAA requires that the registration be done as soon as the trial is initiated. The results should be posted within 12 months of the completion of the trial or within one month of approval of the product. In response to that requirement, in 2008 and 2009 respectively, ClinicalTrials. gov released the results database and also added an adverse events module. In the European Union (EU), the European Medicines Agency (EMA) took a more open policy compared with the FDA.
In 2011, EMA required that all the trials conducted in the EU should be registered and make their results available within 12 months. As mentioned earlier, the journals also play a very critical role in the transparency of clinical trial research. In 2013, The British Medical Journal put in place a requirement for a statement about anonymised participant level data availability at the time of manuscript submission. In 2016, the final rule made by the US Department of Health and Human Services further clarifies the types of trials that need to be registered on ClinicalTrials. gov and the National Institutes of Health (NIH) also issued a policy to promote the sharing of clinical trial data, requiring that any trials that received funding entirely or partly from the NIH should be registered before the start of the trial and have their results posted in a timely fashion.
To go a step further "In several studies, including one of our own, we found that the data collected in ClinicalTrials. gov is a minimal requirement and it does not capture everything you would like to know about the trial, especially with regard to the methodology and with regard to all the results data. It is definitely a first step, and how reliably they can be used for data analysis in systematic review and meta-analysis, I believe we still have a long way to go, but at least we know the existence of trials, so we know if we are missing results or the publication of certain results, we have a place to look. We are getting there. " "The funders have a very important role. In the US, for example, when you are submitting a new grant if your previous publications are not in the public domain, you cannot even submit the grant.
There is no need for an overarching story to tie together many findings and the outputs are recognised in databases like PubMed and Europe PMC. A second innovation is preregistration and registered reports. Preregistration is reporting the design of a planned study in an open registry, sharing the methods ahead of time. Registered reports 5 go a step further. Over 300 journals have agreed to review and publish papers based on the design of the study and its methodology, not the findings. This removes selections for only positive results and bases the publication of the paper, and therefore its prestige or perceived prestige, on the design, not the findings. However, it may not be suitable for all types of outputs, especially those where there is a rapid iteration between hypothesis and experiment.
To go a step further "Frequently, various information concerning the protocol, the data or the results of a study are published in open access on various platforms and in various forms. A crucial objective to enhance the value of these different works is to manage to link them, and preprint servers play a fundamental role in this direction. The OpenScience Framework, for example, provides authors with a very intuitive interface when submitting a preprint, allowing them to include links to all the works associated with their manuscript. This kind of reciprocal linking would greatly benefit from being generalized to all bibliographic databases. " "Micropublications are most useful for scientific communities in which the deposit of research data in disciplinary repositories is already a common practice.
There is a real impact factor culture in the way we train young researchers, and this culture is sometimes negotiated between young researchers and senior researchers: am I going to submit this article to this journal with a high impact factor, or have a safer strategy and submit to a journal with a lower impact factor, or the opposite? These strategies, which are found from one young researcher to another, from one country to another, are really structured around the impact factor and the international databases, I am thinking of the Web of Science, Scopus, which are there to root for and justify this strategy. We can speak of a culture in terms of scientific production, of "Publish or Perish". This focus on articles in high-impact journals, because there is a common terminology, has absolutely similar configurations and normative principles that really mobilize the same practices, actions and the strategies among all the young researchers that we have been able to interview over all these years, strategies that are moreover exacerbated with time.
To go a step further "Scholarly publication should not be restricted to journals and articles. It is a question of how to bring value to other skill sets and to other areas of expertise which do not necessarily result in the publication of a paper. " "In the change of paradigm, heading towards research assessment which is more quality-based, what you do is you make it possible for researchers to be active participants in this assessment. You ask researchers what fields of research and which research activities they consider to matter most over a period of time. You get indicators based on the opinions of researchers rather than ones provided by very costly databases owned by hegemonic commercial players. ".
The increase in these inequalities can also be seen in the transition to open access as it is currently taking place in European journals, particularly in connection with the move to the Article Processing Charges (APCs) model. In Latin America, this model represents a huge difficulty, due to the very high amounts involved. The Recommendation also defends multilingualism because English is unfortunately often the only code for interoperability between and across databases. I would also like to mention another asymmetry that is important for the transformation towards openness in research assessment, namely the very notion of excellence. I have tried to represent in a diagram the way in which the differentiated vision of the North and the South can be seen in the power relations of the academic system.
One of the key points in this respect is multilingualism. The hyper-centrality of English is now absolutely clear in the research world. However, although English is a dominant reality, it is not the only reality, and in South America we have a number of channels for the circulation of knowledge that bypass the use of English. For example, if we search Scopus or the Web of Science, we find Spanish and Portuguese articles as marginal shares. In contrast, the OLIVA 6 database, which combines entries from SciELO and Redalyc and has 345,391 entries in Spanish, more than the Web of Science (270,632 entries) and almost as many as Scopus (373,419 entries). The same observation can be made in BIBLAT  7  and LA Referencia 8 , the latter database containing many more references in Portuguese than Scopus and the Web of Science.
This is why we need to set up a regional CRIS, which already exists in Peru, Brazil and Argentina. Datasources are a fundamental issue to broaden up the landscape of scientific production under assessment, as is the need to really stimulate the multilingualism of local productions by giving them more visibility in international databases such as SciELO, Redalyc or BIBLAT. In South America, there are many small-scale journals published by small research teams or universities. At the opposite end of the spectrum, mainstream bibliographic databases include very few Latin American journals. Latindex, SciELO, Redalyc and BIBLAT show it is possible to publish quality journals, with a level of excellence defined collectively in the academic community with a necessary balance between local standards and international criteria, a balance that must be pursued through horizontal and not vertical imposition.
I argue that when we reorient our focus from publications to researchers who publish those publications, we will spot that multilingualism and language bias are crucial challenges for open science and research assessment. That is a key message of the Helsinki Initiative on Multilingualism in Scholarly Communication 1 , which aims to make language a nonissue in assessment. I am going to present an evidence-based picture of publication patterns of researchers in Europe. I draw on recent studies of the European Network for Research Evaluation in the Social Sciences and Humanities. Evidence from eight European countries  2  shows that the Web of Science database covers only a small part of all peer-reviewed publications in SSH. Scholarly book publications have been an important way of communicating research, but they are invisible in the Web of Science or Scopus databases.
For instance, only 15% of all publications of Polish scholars from SSH are indexed in Web of Science and only just over 50% of Danish scholars. Fortunately, for over a decade, various European countries have been using national current research information systems, thanks to which we can obtain a good picture of the whole publishing landscape in academia. There was a study of the percentage of peer-reviewed publications published in English, local languages and other languages in the social sciences and the humanities in eight European countries. In this study, we have taken into account all publication types and the results are not limited to publications indexed in the largest citation databases. In the Nordic countries most publications are in English, but in the Central and Eastern European countries the local language is dominant.
Some of them in this period published in three, four and more languages. As other studies show, multilingual publishing includes not only articles but also books. Evidence shows that the Web of Science and Scopus databases cover only one-fourth of all peer-reviewed journal articles published by researchers from these analysed countries, and only a small percent of the articles published in local languages. In this way, intended or unintended language priorities in assessment may lead to a systemic undervaluation of SSH research and researchers. The clearest example of the everyday language bias comes from the manuscript peer review, where it is frequently reported that reviewers judge research based on the quality of non-native English speakers' writing instead of the content of the manuscript.
It is not something which is neutral at all. It is a deep disruption for the better, but it requires us to create a large coalition and genuine ownership with the actors of that coalition, and that we then have a real practical way forward with well-planned steps and strong momentum within a reasonable timeframe. With Horizon Europe we will play our part by changing the way we assess proposals. We will move away from the use of journal impact factors in particular and look much more at the content of research. Animator: Maria Leptin, the essence of your organisation, the ERC  7  , is to select excellent researchers who will get fantastic grants over a period of many years, and they will be able to create their own teams and pursue their own scientific ideas. I guess that every scientist in the world would like to have an ERC grant and I guess that you must be challenged as an organisation as to how to assess these candidates and the work they have done? Maria Leptin: It is indeed a challenge.
It is much more than that. The history of software is quite short, unlike many other disciplines. In the 1960s we had the chance to put a man on the moon, and this was done, by the way, thanks to a woman, Margaret Hamilton. She led the team of engineers who developed 60 000 lines of code used in the mission. Those 60 000 lines of code were enough to send a man to the moon and bring him back. Today, some 50 years later, a Linux kernel with over 20 million lines of code is one of the many components fount in the phones that we have in our pockets that allows us to send a smiley to a friend, or to send a message to somebody. The reason why we had this lightning-fast growth is of course because software is changing the world we live in, but also because there is the open source software movement that started over 30 years ago.
Software engineering is a professional activity requiring significant effort, training, and skills. It is not enough to just write 100 000 lines of code to deserve recognition, but it is the usability of that software by others that makes this a result to be recognised. One could argue that we do not have such problems if we use existing tools; not to develop our own software, but to use existing ones and just rely on that basis. For instance, the presentation that I gave on this topic at the conference was designed with MS Office 2013 and I rely on that, aware that this might result in conflicts with other tools. Many developments of virtual reality applications in my group are developed in Unity, version 2019 at the moment, and they will hardly run with any other tool. Another example: one of the very top-ranked tools in the French Ministry's list of tools referred to earlier is MS Excel, again another proprietary tool that might prevent us from sharing our research.
There is no way to completely separate data from software. To make things even worse, there is not a single software or tool that we use for a certain research project. We are using more likely a complex IT environment as a matter of course. Most of the components are hidden beneath the surface. There are, for instance, operating systems. I assume you know if you are working on Windows, Linux or Mac, but do you know which drivers, libraries and codecs you have installed? Do you know which execution environment your software is using? Typically not, and to be honest I do not want to know that. Should this not be hidden? However, if you would really draw an image of all the software components you are using in your research environment, it may look like a wobbly tower of bricks. And I bet that somewhere there is this one tiny little component on which everything else is built -and that makes everything crash if it fails.
He does it in a very practical way, using textual data, like Drama Corpora, or visual data, like urban graffiti, or audio data from Spotify -cultural data in a broader sense. He creates a kind of data culture from this, and students love that and learn a lot. Since students are able to learn how to work this way, whatever their research field is, researchers should be able to learn that too. We are currently preparing a systematic evaluation of the competencies that the students gained over the past three years from these hackathons. At the moment, there is such an introductory course established across different fields of study in that discipline, and a broader one is in preparation. I hope it will be available online free for other fields, too, maybe in forms of micro-credentials to easily invoke it.
One solution that has been very popular within the publishing community is the link between GitHub and Zenodo. Authors of code can quite easily archive their code on to Zenodo and they get a DOI. DOI is very publisher-centric and they understand what this is. We have been using DOIs via Crossref for journal articles for a long, long time and it is easy thing for publishers to recognise, and the code is archived somewhere, so it feels like a winner to everyone in that situation. Then Zenodo can also link back to the GitHub site, so if there are changes people can see what is there. Using some work from a data scientist at Europe PMC, we mined 7 000 random XML documents we had, and we found 300 000 citations. Of that large number there were 10 citations to Zenodo that we found, using a Zenodo DOI.
As Roberto was talking about earlier, having a dedicated archive for software, Zenodo is not that. Of those 10 citations, three were datasets, one was a journal article, and the rest were software. However, at what the publishers called that content when they tagged the citations within their XML format, there was only one that was identified as software and the others were all a mishmash of different things, which is problematic if we are looking for proper credit, attribution and reuse of software. One of the issues that I have often thought about is that "Zenodo" does not equal "code", so it is not possible for publishers to say, 'Okay, fine, there is a Zenodo DOI there, tag it as code', because it is not that simple. Zenodo holds lots of different things. Publishers are very familiar with Crossref.
It is a publisher organisation. They have DOIs and they craft and use the tools there to validate reference, check them and pull in extra metadata if it is missing, or use it to add DOIs to their publications if authors have not provided them. DataCite is the DOI provider for Zenodo and a lot of publishers are not using the tools of DataCite to crosscheck their references, so if there are Zenodo-based DOIs they are quite often not getting the extra validation steps that would allow the publisher to add in the fact that it is code, data or something else. Another solution, other than the Zenodo solution, is the Software Heritage solution, as you have heard Roberto talk about. This was the way that eLife went. There was a strong feeling within the team that DOIs were publisher constructs, and they were not necessarily the way forward to reference and archive all the different varieties of software.
Software Heritage is archiving all this content from a vast number of different sources. Software Heritage links to specific parts of the code and it is reingesting, creating updates, and it indexes more sources. The other thing key thing for eLife was that we did not have to ask the authors to do the work to create the Zenodo link and archive their code themselves. We could do it for them, and that was a key thing for us because, as you know, my first point was, 'Oh, yet another thing they are asking me to do'. We felt that we did not want to insist that they did the extra work. We were doing it for them. In terms of the eLife experience, I am really proud of how many changes we made. Regarding the VoR (version of record) publications that we published over time, eLife is a modest-sized journal compared to all of the content now that I now deal with at Europe PMC, but by the end of 2021 the number of software citations matched the number of VoR publications.
One of the key things about the recommendation therefore is giving the publishers the ability to recognise a reference as software and indicate that in the XML so that that is then clear to anyone who is reusing that or who needs to mine for software references. The other thing about this recommendation was that it had to be as minimal as possible, because the more complexity you add to these things the harder it is for people to take them on. eLife put a lot of effort into very full-structured references, but it is hard. The next thing is that we wanted a pub-id. That is a well-known ID and an identifier for something that is properly archived. GitHub unfortunately does not fit into that remit, but Zenodo and Software Heritage do. In that subset of articles that we looked at from eLife and PeerJ we found that 202 had a pub-id and they were all to DOIs.
One of the issues I see for people is which path to take. For instance, at eLife we took the Software Heritage route. There is also Zenodo. I am sure there will be more coming up and different ways of dealing with things, and that does cause problems in itself sometimes. You have to reach that tipping point where people all go down one way, but at the moment we are not quite sure yet. There are many contributors to all of this. It is not just publishers, it is not just authors, it is not just the researchers. There are so many moving parts here, and a big issue is educating people, from publishers to researchers, and everybody needs the education. We also found that when we were talking to our vendors, we had to explain the production and editorial issues, and the feedback loops are extensive, as well as updating on tooling.
Johns Hopkins University Software is a pillar of open science. The universities produce basically three of these pillars: articles, data and software. Those are connected and in many ways those three pillars of open science and the connections between them relate to what Roberto described as the scholarly ecosystem. In large part, those are the projects that universities do through their research. They are typically funded through grants and other kinds of external funding, and are very much the things that happen within the walls of the university. At some point, if you wish to move the impact of research and education outside the walls of the university then you have to translate them. It is useful to publish papers and to search data. It is also extremely useful to share the software. Over the last two years we have all been experiencing COVID-19 and there has been a lot of discussion about how open science led to the vaccines that have made it possible for us to deal with it.
I would submit that the most important piece of that was actually not the sharing of articles or even the data being shared, but rather the software, the code that people were using to analyse the data associated with the vaccines that we eventually developed. The open source programmes office, or OSPO, is a university organisation that can support this translation function. It is the unit that we have created within Johns Hopkins 1 to help our researchers translate their research and their education activities outside the walls of the university. One of the ways that we have advanced the work of the OSPO is what you heard from my previous panellists, this statement or affirmation that software is a primary research object. If you ask people within a university, either researchers or administrators or even students, 'Do you care about open source software'? you will get different responses, but if you ask them if they care about research, every one of them will say, 'Yes, of course I do'.
Therefore, being able to make this argument that open source software supports research, is a tool, but is also a research object as we have heard, has been incredibly important in terms of advancing the work of this OSPO within Johns Hopkins University. We have also made the case, you heard earlier today, that software is a better way to understand what data are being produced rather than the articles that cite those data. The citation or the connection between an article and a dataset is one way, is often a direct mechanical kind of link, but the connection between data and software is much richer. If you look at the software, you can actually understand a great deal more about what data are being used and how those data are being analysed. I will talk about a specific example that describes this.