dataesr/scientific-paragraphs-categorization huggingface dataset:
Pretraining and Output Encoding We will use the two pretraining objectives in TURL  [11] , namely masked language modeling and masked entity recovery, to demonstrate how to pretrain the transformer model using unlabeled tables and learn to capture the semantic and relational knowledge embedded in the data. We will cover the masking procedure, which involves randomly masking out elements of the tables, and the preparation of target outputs, which the model will be trained to predict. We will proceed with some implementation details of the pretraining, including the hyperparameters, data size, and computing resource requirements. We will also provide utility code to visualize the attention weights and output table encodings, which can help participants understand how the model processes and represents the data.
Hands-on Prerequisites. During the hands-on part, participants will have access to a configured (Python) development environment that is accessible through the browser in Google Colaboratory 5. This environment will have required packages installed, and provide a general structure for the tutorial with template code, visual examples, etc. to minimize time to get started. Access to the pre-configured Google Colab notebook requires a Google account. This material can be found at: https://github. com/madelonhulsebos/ neural-table-representations-tutorial-2023. Ethics. The use of large-scale Transformers requires a lot of computations and GPUs/TPUs for training, which contributes to global 5 https://colab. research. google. com/ warming  [32, 33]. We stress this orthogonal issue and possible approaches to mitigate it in the tutorial.
The datasets used do not include private data. Difference with Previous Tutorials. Our tutorial partly overlaps with two of our tutorials that have been offered recently: 1) "Transformers for Tabular Data Representation: A Tutorial on Models and Applications"  [2]  in VLDB 2022 covers only the survey part in our proposal. Moreover, our SIGMOD tutorial aims at covering in more detail a smaller subset of applications. For these applications, we will give more examples and details on how to use them, to enable the attendees to execute them in the hands-on part. 2) "From Tables to Knowledge: Recent Advances in Table Understanding"  [30]  in KDD 2021 is much wider in scope as it covers semantic models and information extraction, such as NER, and did not include hands-on exercises. In this SIGMOD tutorial, we will focus more on the transformer architecture, covering pretraining and fine-tuning, with half of the time dedicated to a hands-on session where attendees can experiment with the models on their laptop.
, 2017). In 2018, FET was implemented in the Monte-Carlo code SERPENT2  (Wendt et al. , 2018) , and in 2021 in the code RMC  (Wang et al. , 2021). FET estimators are mainly used to gain precision on the spatial distribution of power at a pin level (to correctly evaluate fuel evolution for instance). Yet, little interest has been given to Functional Expansion Tally methods to evaluate the power distribution at a larger scale. Moreover, FET estimators are limited in the choice of the basis on which to project the power density -at a full-core level, Zernike or Legendre polynomial basis do not bear any relationship with the physics of the problem. This paper presents an approach to FET estimators to evaluate the power distribution in a full-core with assembly-level tallies, developed in the Monte-Carlo code SERPENT 2  (Leppänen, 2015).
The FM-FET offers a gain in computation time by reducing by several factors the number of histories to simulate. This method should thus be appropriate in multi-physics coupling between Monte-Carlo and CFD codes. More particularly, Global Transfer Function method are being developed for Monte Carlo -Thermal hydraulics Picard iterations  (Terlizzi and Kotlyar, 2021, 2022). The use of the fission matrix eigenfunction basis as the transfer space could be a solution to improve the performance of these methods. Even if these five cases presented in this paper are not a theoretical proof of the FM-FET method efficiency, there are good hopes that this method will continue to perform better than the classic histogram approach. If used in the framework of a parametric study of perturbation study, the FM-FET method is easy to implement in the Monte-Carlo code SERPENT 2 and offers a 5-10x gain in simulation time to achieve comparable statistics in the power distribution.
Data quality has recently become a critical topic for the research community. European guidelines recommend that scientific data should be made FAIR: findable, accessible, interoperable and reusable. However, as FAIR guidelines do not specify how the stated principles should be implemented, it might not be straightforward for researchers to know how actually to make their data FAIR. This can prevent life-science researchers from sharing their datasets and pipelines, ultimately hindering the progress of research. To address this difficulty, we developed the BIBBOX, which is a platform that supports researchers publishing their datasets and the associated software in a FAIR manner.
However, as these guidelines do not specify how the principles should be implemented nor suggest specific technologies, it might not be clear for researchers how actually to make their data FAIR. To support researchers with this task, we developed the BIBBOX, a platform where datasets and accompanying software (Apps) can be easily published in a FAIR manner, by incorporating a FAIR Data Point (FDP) into the Apps  [4, 5]. An FDP is an additional software layer that allows data and metadata to be exposed in a FAIR compliant manner  [4, 5]  and will be described in detail in the next section. The BIBBOX is therefore a set of tools and services that support the creation and management of FDPs. The architecture of the BIBBOX is designed to be modular and flexible, allowing researchers to add a FAIR Data Point to existing open source software solutions and to publish these in an App Store.
Data and metadata should be retrievable by their identifier. It is important to keep up-to-date accurate metadata and not to delete it, even if the corresponding data is no longer available. • Interoperable: Data should be easy to integrate with other data, using standard data formats and ontologies. This allows datasets to be combined, merged and analyzed together. Additional documents can be created, describing, e. g. , exactly what was recorded and the units of each variable. • Reusable: Data should be designed for reuse, using clear and detailed documentation and metadata and following best practices for data management. Metadata should describe the data using various accurate and meaningful attributes, such as file name, creation date and time, author, and when the file was last updated.
FAIR training and education But why do researchers often not follow the FAIR principles? Making data FAIR is a complex process that requires an early commitment of resources and effort, which only pays off later in the research cycle. It is important for researchers and organizations to consider carefully the costs and benefits of implementing the FAIR principles in order to determine the most appropriate approach for their specific needs and circumstances. The following tools can be used to evaluate one's knowledge of the FAIR principles: • FAIR-Aware is an online tool that helps researchers and data managers assess their knowledge of the FAIR principles, and understand what can be done to make a dataset as FAIR as possible, before uploading it into a data repository. The tool comprises 10 questions, each offering additional information and practical tips that extend users' understanding of the FAIR principles [7].
Some generic 'FAIR enablers' include: Awareness: Researchers and organizations may simply be unaware of the FAIR principles and the tools available to help them implement them. Resources: Implementing the FAIR principles can require resources such as time, expertize and funding, which may not be readily available to all researchers and organizations. Complexity: Making data FAIR can be a complex process, especially for large and diverse datasets. It may be challenging for some researchers and organizations to identify the most appropriate strategies and tools for their specific needs. Incentives: Researchers and organizations may not always have strong incentives to make their data FAIR, especially if they do not anticipate that their data will be heavily used by others. Cultural or organizational barriers: There may be cultural or organizational barriers to making data FAIR, such as a lack of support or infrastructure for data sharing, or concerns about privacy or intellectual property.
Other approaches that can be used to measure the extent to which data adheres to the FAIR principles include e. g. : • FAIRsFAIR Data Object Assessment Metrics proposes 17 core metrics for the systematic assessment of FAIR research data objects in Trustworthy Digital Repositories  [13, 14]. These metrics are based on indicators proposed by the RDA FAIR Data Maturity Model Working Group  [15]  and on the WDS/RDA Assessment of Data Fitness for Use checklist  [16]. • F-UJI is a web service to programmatically assess the FAIRness of digital research datasets. It was developed by FAIRsFAIR and is based on 16 of the FAIRsFAIR Data Object Assessment Metrics  [17] [18] [19] [20]. • CoreTrustSeal þ FAIRenabling Capability Maturity Model is a self-assessment tool that helps repositories evaluate their trustworthiness and FAIR enabling status  [11].
• Creating clear and detailed documentation and metadata to support reuse of the data. • Following best practices for data management, such as properly documenting and versioning data, and providing access to data in a timely manner. Tools and services that can help researchers and organizations implement the FAIR principles may include data repositories, data catalogs, and data management platforms, as well as software tools for creating and managing FAIR data points. Some of such tools are e. g. : • DataCite provides DOIs for research data and other research outputs. Research organizations can join DataCite as members in order to assign DOIs to all their research outputs, making them become discoverable. The associated metadata is also made available to the community. DataCite also offers additional services to improve the DOI management experience  [24].
The FDP main components are the Metadata Provider, the Access Control and the Metadata Schemas. The Metadata Provider enables the access to the metadata available in the FDP. The Access Control controls the access to the metadata content. In general, the metadata content of an FDP can be read by anyone, but specific permissions are required in order to add, delete or edit its content. In some situations, metadata or part of it may also have access restrictions for reading. The Access Control component makes sure that these restrictions are enforced. The FDP exposes metadata about different types of digital objects, such as repositories, catalogs and datasets. The metadata should comply with specific schemas, which describe how the metadata is set up, and usually address standards for common components of metadata such as dates, names, and places.
Community-specific schemas can be used to address elements needed by a given community. The FDP reference implementation provides four Metadata Schemas: for the FDP itself, as a metadata service; for catalogs; for datasets; and for datasets' distributions. These schemas allow the FDP to check and validate metadata content that is added. The FDP reference implementation also supports the addition of metadata schemas for other types of digital objects and extensions to the base schemas. The FDP uses the W3C's Data Catalog Vocabulary (DCAT) version 2  [26]  as the basis for its metadata content. DCAT is a model for describing catalogs, which was designed to facilitate the interoperability between data catalogs published on the Web. It enables to describe datasets and data services in a catalog using a standard model and vocabulary, which facilitates the consumption and aggregation of metadata from multiple catalogs and can increase the discoverability of datasets and data services.
In DCAT, a Dataset represents a collection of data, while DataService represents services accessible through an interface (API) that serve datasets. Catalog, a subclass of Dataset, represents aggregations of metadata records about digital objects. The FDP extends the DCAT model by adding the concept of FAIRDataPoint as a specific type of DataService that serves metadata catalogs and metadata records. FAIRDataPoint is a subclass of MetadataService, which is itself a subclass of DataService. DCAT is frequently augmented by other vocabularies to provide an application profile. The DCAT Application profile (DCAT-AP) for data portals in Europe  [27]  is a DCAT-based specification for describing public sector datasets in Europe. It aims to enable a cross-data portal search for datasets and make public sector data better searchable across borders and sectors.
This can be achieved by the exchange of descriptions of datasets among data portals. There are several extensions of DCAT-APs. For instance, the DCAT-AP-JRC is an extension to DCAT-AP for multidisciplinary research data  [28]. Currently, the Health DCAT-AP is under development. It aims to develop a standardized descriptive metadata respecting the FAIR principles and user needs of the European Health Data Space  [29].
BIBBOX The BIBBOX (Basic Infrastructure Building Box) started to be developed within the framework of the B3Africa project  [30]  and intended to offer an easy installation, deployment and integration of open-source software solutions (Apps) in the fields of bioinformatics and biobanking. Currently, BIBBOX is a locally deployable platform that supports the publication of datasets and accompanying software in a FAIR manner, through the integration of a FAIR Data Point into the Apps. The BIBBOX runs on Linux, and its core architecture strongly relies on the use of Docker  [31]. In the BIBBOX framework, the software tools (Apps) are simply a group of Docker Containers, ready to be used. Therefore, they can run quickly and reliably in any computing environment  [32, 33]. As illustrated in Fig.
5 , all Apps contain a human-readable component and a machine-readable component. Apps that handle data are recommended to be "FAIRified", i. e. , extended with a FAIR Data Point (FDP). The Master FAIR Data Point works as a hub for all FAIRified Apps, as each of these Apps has an individual FDP that connects to the master FAIR Data Point. Obviously, Apps that do not handle data, do not need an FDP. A  Each BIBBOX App must be published in the BIBBOX GitHub repository  [34]. As depicted in Fig. 6 , additional Docker Images that might be necessary, can also be published in the same repository and in the BIBBOX Docker Hub  [36]. Once the App is published in the BIBBOX GitHub repository, it becomes available for installation at the App Store of the BIBBOX Portal. When a user installs an App, its installation will appear on "App Instances".
Conclusion and outlook The redesign of BIBBOX aims to respond to the current need of researchers to make data FAIR. With the help of an FDP, the BIBBOX can support institutions and researchers sharing their findings and increasing the visibility of their research. Furthermore, it can have an important role in the validation of datasets and diagnostic applications. This way, it can promote research collaborations and improve the reproducibility and reliability of research. The availability of BIBBOX and its integrated Apps can also contribute to stimulate global research collaborations, including in low-medium income countries. In future, we plan to incorporate in the BIBBOX the possibility of adding a Provenance Access Point (PAP) to selected Apps. With the incorporation of a PAP, organizations can store and manage provenance information independently.
Acknowledgements Parts of the work have received funding from the  Austrian Science Fund (FWF) , Project:  P-32554 explainable Artificial Intelligence. Part of the implementation work has received funding from the  European Union 's  Horizon 2020 Research and Innovation Program  under Grant agreements nos. 857122  ( CY-Biobank ) and 874662 (HEAP). This publication reflects only the authors' view, and the  European Commission  is not responsible for any use that may be made of the information it contains. Where authors are identified as personnel of the  International Agency for Research on Cancer/World Health Organization , the authors alone are responsible for the views expressed in this article and they do not necessarily represent the decisions, policy or views of the  International Agency for Research on Cancer/World Health Organization.
In other types of systematic reviews, reporting was worse, with 10% of reviews reporting funding and 1% reporting conflicts of interest, although the quality of reporting differed across journals  [5]. Systematic reviews often include older trials that lack information on funding and researchers' conflicts of interest  [6]. While contemporary trial publications more often contain relevant information  [1, 2] , reviewers may sometimes need to retrieve the information elsewhere, for example from trial protocols and public conflicts of interest databases like the US Open Payments Database  [7]. Little is known, however, about the usefulness and effort required for such a strategy. We therefore compared the approach used in contemporary Cochrane reviews for retrieving and reporting funding of included trials and trial researchers' conflicts of interest with a structured information retrieval approach, and assessed the time effort required for such an approach.
Eligibility criteria and inclusion On December 14th, 2020, using the Cochrane Library one author (EF) included the 100 most recently published Cochrane reviews from the Cochrane Database of Systematic Reviews. From each review we then selected one metaanalysis of the primary outcome based on a decision hierarchy (Appendix 2) thereby including 100 unique metaanalyses. Using a random number generator, we then included one trial from each of the 100 included meta-analyses (Appendix 3). For each trial we retrieved the main trial publication (i. e. , primary reference in Cochrane review) and any supplementary documents including separate conflicts of interest disclosure forms (e. g. , ICMJE disclosure form  [8] ).
Data extraction and information retrieval From each Cochrane review, we extracted review characteristics, meta-analysis characteristics and the reviews' characteristics of the included trials as well as any information on trial funding and researchers' conflicts of interest. Using a pilot tested data sheet, one author (EF) extracted data which was verified by another author (AT). In case of disagreements, a third author (AL) acted as arbiter. From each trial publication we also extracted trial characteristics and information on funding and primary trial researchers' conflicts of interest (complete list of data in Appendix 5). For trials where information in trial publications or conflicts of interest disclosure forms was unavailable or was unclear (e. g. , information on funding available but not conflicts of interest), one author (EF) searched other sources relevant to the specific information.
These additional sources were trial protocols, public conflicts of interest databases, trial registry data, secondary publications to the main trial publication and other publications by the same primary trial researchers, as well as publicly available company data (Appendix 6). We also emailed the corresponding authors of trials published within the last decade (and sent reminders after 4 weeks). Lastly, we noted which sources contained the relevant information and any inconsistencies between sources (e. g. , trial publication listing a single funder vs. protocol listing two funders).
g A trial committee is typically an independent committee involved in trial conduct, for example, a data or monitoring board or a steering committee data safety and monitoring board or a steering committee. h A commercial contract research organization is a commercial company, typically contracted by an industry funder to undertake different aspects of a trial such as data management or statistical analysis. Using the comprehensive approach, searching multiple additional information sources, we retrieved funding for two additional trials (from a trial registry and a trial protocol) to a total of 86 trials. We found one instance of discrepant funding information between the trial report and the corresponding Cochrane review (Appendix 7). No discrepancies were found between information retrieved from trial publications and other sources.
e. in some fashion based upon a main trial publication (e. g. , a secondary analysis using the original dataset). gain and time needed for a chosen retrieval approach. We included contemporary Cochrane reviews, the study plan was detailed in our protocol, data extraction was done systematically by two authors, and the guide developed from this work is short and simple. However, we only included one trial from each review as a pragmatic sampling strategy and we solely studied Cochrane reviews, and extrapolation of our results to other systematic reviews requires some caution. Still, it is likely that fewer other systematic reviews retrieve trial funding and researchers' conflicts of interest than Cochrane reviews  [5] , and adaptation and use of a simple structured retrieval approach may provide even greater benefit for other reviews than Cochrane reviews.
For some of these trials, the information can be retrieved, but it requires some degree of effort and our guide describes how to address the issue. Information in trial registries like ClinicalTrials. gov was first made public in 2000  [19]  and the Open Payments' Database was first introduced in 2013 [20], thus information in sources other than main trial publications and disclosure forms are often unavailable for older trials, unless trial researchers are contacted and reply. In a minority of recent trials, the information is unavailable in publications. Potential solutions are enforcement of reporting standards by biomedical journals  [8, 9]  and public conflicts of interest databases  [21] , preferably a single easily searchable international platform. This would increase accessibility, accuracy and transparency of the reporting of conflicts of interest  [22].
To test the model, we explore the effect of the Panama Papers on corruption and capture. We use new survey data provided by Reporters Without Borders, which is the most comprehensive cross-country database that allows the different forms of capture to be identified. For internal capture, we consider conflicts of interest with media owners and the share of owners who have interests in other economic sectors. External capture is proxied by outside payments and in-kind benefits to journalists. We follow a differencein-differences strategy, using the revelation of the Panama Papers as a shock to perceived corruption. We find a larger increase of internal capture in countries exposed to the Panama Papers (where perceived corruption increased) after the revelation, which supports our first prediction.