dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements We thank the participants and investigators of the FinnGen and Whitehall II studies. This study was supported by the  Wellcome Trust  (no. 221854/Z/20/Z ) and the  UK Medical Research Council  (no. S011676 ). For the purpose of open access, the author has applied a CC BY public copyright license to any author-accepted manuscript version arising from this submission. The  FinnGen  project is funded by two grants from  Business Finland  (nos. HUS 4685/31/2016  and  UH 4386/31/2016 ) and by the following industry partners:  AbbVie Inc. ,  AstraZeneca UK Ltd, Biogen MA Inc. ,  Bristol Myers Squibb  (and  Celgene Corporation & Celgene International II Sàrl) ,  Genentech Inc. ,  Merck Sharp & Dohme Corp. ,  Pfizer Inc. ,  GlaxoSmithKline Intellectual Property Development Ltd. ,  Sanofi US Services Inc.
Data availability This study used publicly available data at https://www. mrbase. org/, https://www. uniprot. org/, http://cpdb. molgen. mpg. de/, https://www. genome. jp/kegg/ and https://www. opentargets. org/. Data used in MR are deposited with Zenodo 118 at https://zenodo. org/deposit/7042008. Data, protocols and other metadata of the Whitehall II and FinnGen studies are available according to the data-sharing policies of these studies. The pre-existing data access policy for the Whitehall II study specifies that research data requests can be submitted to the study steering committee, and these will be promptly reviewed for confidentiality or intellectual property restrictions and will not unreasonably be refused. Detailed information on data sharing can be found at https://www. ucl. ac.
uk/epidemiology-health-care/research/epidemiology-and-publichealth/research/whitehall-ii/data-sharing Individual-level patient or protein data may further be restricted by consent, confidentiality or privacy laws/considerations. FinnGen data can be accessed through Finnish Biobanks' FinBB portal (www. finbb. fi). FinnGen summary statistics are freely available at https://www. finngen. fi/en/access_results, with results for new data freezes updated every 6 months.
Study 3: plasma proteomics Whitehall II, UK 6,545 participants; mean age 56 Plasma proteins measured with SomaScan v. 4 National Health Services Hospital Episode Statistics years; 71% men database, the British National mortality register and 5-yearly clinical screening Study 4: PRS FinnGen, Finland 339,233 participants; mean age 54 Illumina and Affymetrix arrays; AxiomGT1 algorithm National hospital discharge (from 1968), death years; 44% men for Affymetrix data; imputation with population- (from 1969), cancer (from 1953) and medication specific SISu v. 3 reimbursement (from 1964) and purchase (from 1995) registries Study 5: HLA analyses FinnGen, Finland 339,233 participants; mean age 54 rSSO-Luminex technology (Labtype, One Lambda); National hospital discharge (from 1968), death years; 44% men PCR-SSP (Micro SSP Generic HLA Class I/II DNA (from 1969), cancer (from 1953) and medication Typing Trays, One Lambda; Olerup SSP genotyping; reimbursement (from 1964) and purchase (from 1995) AlleleSEQR PCR/Sequencing kits, Atria Genetics; registries BI 3130xl genetic analyzer (Applied Biosystems, Thermo Fisher Scientific); Immunochip array (Illumina); imputation HLA*IMP:0240 (The Oxford HLA Imputation Framework) Cohort in study 6: IPW analyses FinnGen, Finland 117,773 participants; mean age 55 years; 55% men ATC codes from medication reimbursement(1997- 2019)and purchase(1997-2019)registries Illumina and Affymetrix arrays; AxiomGT1 algorithm for Affymetrix data; imputation with populationspecific SISu v.
Acknowledgements Financial support from the  German Research Council (DFG)  through the  International Research Training group  ( IRTG 1642 ) Soft Matter Science is gratefully acknowledged. Partial support by the  Deutsche Forschungsgemeinschaft (DFG, German Research Foundation ) under Germany's Excellence Strategy -EXC-2193/1 -390951807 and by the  ZEISS Research Cluster IPROM  is gratefully acknowledged. The authors are thankful to the micro-nanomechanic platform of  Institut Charles Sadron, Strasbourg. Open access funding enabled and organized by Projekt DEAL.
Specifically, we show how BERT (Bidirectional Encoder Representations from Transformers,  Devlin et al. , 2018) , a powerful Natural Language Processing (NLP) technique for calculating contextual word embeddings, can be used to obtain meaningful measures of news media consumption from users' web browsing behaviour, a behaviour often studied with such data. We collected headlines of news articles read online by participants of a web tracking study conducted in France, Germany and the United Kingdom by crawling information from the news websites. Data were collected in the run-up of the 2019 European Parliament election that was held between 23 May and 26 May 2019 in the (back then) 28 member states of the European Union. In addition, participants responded to two surveys that were used to measure their political interest and voting behaviour in the election.
Budak et al. (2016) , for example, study ideological bias in news organizations in the United States by training two algorithms that, first, identify front-section news and then, within front-section news, political news. To understand the actual content of the news classified as front-section and political, they employ human judges recruited from a crowdsourcing platform. These crowdworkers then classified news articles into 15 topics derived from a latent Dirichlet allocation (LDA) topic model and classified each article's ideological position. Similar manual coding is used by Dvir-Gvirsman et al. (  2016 ) for web browsing and survey data from 400 participants in Israel to study selective exposure and by  Peterson et al. (2019)  for measuring partisan selective exposure in online news consumption during the 2016 U.
Moreover, operating systems vary in the amount of information that third parties can access, for example, through research apps. The Android smartphone ecosystem, for example, is much more permissive in the collection of browsing data than Apple's iOS system  (Keusch et al. , 2020). In addition, ownership of smartphone type co-varies with socio-economic status such that differences in the amount of data that can be collected due to smartphone brand will correlate with participant characteristics. Furthermore, one needs to assure that data are collected from all devices of a participant, for example, from their smartphone and from their personal computer to obtain complete records of users' online activities  (Revilla et al. , 2021). In some cases, however, participants cannot install data collection software if, for example, they use devices provided by their employer who may restrict the installation of third-party software  (Barthel et al.
DATA Data for our project are provided by members of a commercial opt-in online access panel maintained by respondi. Participants registered in this panel are occasionally invited to answer to short surveys in exchange for small incentives. In addition, the panel vendor keeps track of the online and app use behaviour of a subset of panel participants who agreed to having their web browsing and app use behaviour on their private devices monitored. Participants' browsing behaviour is logged through a web browser plugin provided by the panel vendor that participants installed in the browsers used on their personal computers. Each time a participant navigates to a website, the plugin records the complete URL of the website visited (e. g. https://en. wikipedia. org/wiki/URL), the domain (wikipedia.
org), the current date and time as well as the time spent on the website. In addition, participants' browsing and app use behaviour on mobile devices (e. g. smartphone and tablets) is recorded through a research app, also provided by the panel vendor. After installing the app on mobile devices, participants' browsing behaviour in the device's native browser (e. g. Safari in iOS and Chrome in Android) is recorded. Likewise, each time a participant opens an app, the name of the app as well as a time stamp are recorded. Information about interactions of participants with websites and apps, the content shown to participants or their actions inside an app are not observed. Moreover, all data were provided to us in pseudonymized and de-identified form. At any time, participants can opt to pause tracking temporarily or opt out of data collection completely.
Researchers can obtain participants' tracking data from the panel vendor, ask the same participants questions through online surveys and, most importantly, link participants' tracking data with their survey responses. We obtained such web browsing data and app use records in the run-up of the 2019 European Parliament elections from panel participants in Germany (n = 1636), France (n = 1748) and the United Kingdom (n = 968). The main objective in selecting participants was to obtain samples with enough variation in respondents' survey reports (see below) and their browsing behaviour in short time. Although these samples match population benchmarks in each country on several socio-demographic characteristics, we note that they are not suited for population inference without further corrections through weighting or similar techniques.
Several criteria had to be fulfilled to be included in the final analysis sample of this study. First, participants had to respond to both surveys and had to provide tracking data during the whole field period from at least one device. Out of 1636 participants originally selected in Germany, 11 broke off the pre-election survey, 14 of 1748 did so in France, and 22 of 968 in the United Kingdom. Four respondents in Germany broke off the post-election survey and one was screened out. In France, six did not complete the post-election survey and three did not complete it in the United Kingdom. Moreover, several participants did not provide tracking data during the whole field period and were therefore excluded from the final analysis sample. Overall, with these two restrictions, our final analysis sample consists of n r = 1061 in Germany, n r = 1155 in France, and n r = 639 in the United Kingdom for whom we observe both survey and web tracking data.
Extracting news media content We employ a three-step processing pipeline to extract content-based measurements of news media consumption from the records of web browsing activities. First, we scrape headlines of news articles that respondents read online during the tracking data collection field period but before the collection of the survey data (26 April-17 May). In addition, we scrape key words or tags that news platforms assigned to the articles on their websites (e. g. News, Employment, Bank of England, UK economy). We use these tags to fine-tune our BERT model. Second, we employ our fine-tuned BERT model to obtain numeric representations of the scraped news headlines. Third, we condense the numeric representations or text encodings into a manageable and interpretable set of features by using dimensionality reduction techniques.
The processing pipeline and the corresponding design choices are outlined below, and summarized in Table  1. Code for implementing the outlined steps as well as for the following analyses is available at https://github. com/ rubac/euro_news. In step 1, we gather headlines of news articles from 1347 websites visited by respondents of all three countries. The list is based on the 'News and Media' category provided by the Website Category API of Webshrinker (https://www. webshrinker. com), with some manual additions and deletions. For example, media sites which do not provide general news but rather specific information regarding, for example, weather, TV programs and gaming were taken out. Overall, selected websites mostly feature national and regional news outlets (e. g. https://www. theguardian.
Moreover, section pages contain multiple headlines and article summaries and we do not know which news were actually read by the respondent when she visited that page. We keep, for example, https://www. glasgowtimes. co. uk/news/17647010. traceywylde-glasgow-takeaway-owner-zhi-min-chen-jailed-life-1997-murder-woman as it points to static content (a news article). The code used for filtering out dynamic pages and for scraping the remaining pages is part of the GitHub repository accompanying this paper (https://github. com/ rubac/euro_news). For each relevant web page visited by at least one respondent during the period of interest, we scrape the headline as well as the tags associated with the news content, if available. As a result, a typical entry for our web tracking data regarding news consumption consists of a URL, for example, https://www.
Space constraints and the complexity of the BERT deep bidirectional transformers architecture do not allow us to present the original BERT model in more detail. Instead, we focus on our fine-tuning procedure of the original BERT language model and the post-processing of the obtained embeddings. Readers unfamiliar with deep learning and neural networks are referred to the excellent introduction to the topic featured in  James et al. (2021)  and to the original BERT paper by  Devlin et al. (2018)  for details on the BERT model architecture. In our application, we fine-tune the original BERT-Base, uncased 12-layer model from Devlin et al. (  2018 ), available at https://github. com/google-research/bert, for topic detection. The goal in our case is to obtain compact sentence-level embeddings of headlines such that headlines dealing with similar topics are represented by similar vectors in a low-dimensional space.
Weights in the BERT layer are kept frozen except for the last two layers of the BERT model itself, which gives a total of 14,252,644 trainable parameters, 14,175,744 within the BERT model and 38,450 for each of the dense layers. 95,306,496 non-trainable parameters remain in the BERT model. Following the recommendations in  Devlin et al. (2018) , we stick to a low learning rate (2e-5), a low number of number of times the fitting algorithm passes through the training set (epochs = 5) and we use binary cross-entropy as the loss function. We achieve a 86. 3% accuracy on test data. To complete step two (obtaining embeddings of the headlines), we compute F I G U R E 2 Extracting content-based features based on BERT encodings. The process starts with a data set with 47,278 headlines (n uh ) and 50-dimensional BERT vectors (e), which are then aggregated into 26 cluster (c * ) and 6 PCA components (d * ).
Lastly, the cluster and components are used to obtain a total of 70 (2c * + 3d * ) BERT-based features on the respondent level (n r ) the 50-dimensional headline vectors for all n uh = 47,278 unique headlines in the time-restricted (26 April-17 May) tracking data using the fine-tuned model. Those vectors are henceforth referred to as the BERT encodings of the headlines. BERT tasks were conducted using Google Research's Tensor Flow implementation of BERT and its Keras integration provided by  Antyukov (2019). In step 3, we utilize the BERT encodings to obtain a reduced set of content-based features on the respondent level. We implement two aggregation approaches for constructing features as shown in Figure  2. In the first approach, the clustering approach, we group all unique news headlines into c * = 26 clusters using K-means clustering and the normalized BERT encodings as clustering variables.
Regression models Figure  3  presents estimates from logistic regressions predicting political outcomes from the survey data (political interest, undecided, voted, changed mind) with BERT features and socio-demographic controls, by country. Specifically, we include the relative browsing time respondents spent while reading political news about their respective country as the main features of interest, that is, the Brexit cluster for the UK models, the French Politics cluster for France and the Elections and German Politics cluster for Germany. We additionally include the median and variance of respondents' scores on the PCA dimensions that match the respective countries. That is, features based on the UK Brexit versus FR crises dimension are included in the models for the United Kingdom and France, whereas corresponding features based on the Entertainment versus GER elections dimension are included in the models for Germany.
B Gain/amplification of TOP3A gene region (17p11) is a genomic signature of ATRX-wt osteosarcomas. For each ATRX-mutated (top part) and ATRX-wt (bottom part) case, from the left to the right, the chromosome 17 and regional genomic profiles were established, with CGH Analytics software. A focus was made within the genomic interval  of the short arm of chromosome 17 (hg38 human genome mapping; Build 38 from NCBI, December 2013 version) including TOP3. Color profiles corresponding to the different tumors are defined at the top of each group. Most ATRX-wt cases show gain or amplification of this region (bottom part), whereas no ATRX-mutated cases display this alteration (top part). C Chromosome 17 copy number variation analysis showing amplification/gain of the TOP3A region that do not extend into the TP53 gene in ALT ATRX-wt tumors (left panel), and amplification of the TOP3A region for which TP53 region is at the edge of the amplification/gain in ALT-positive ATRX-wt tumors (right panel).
Transcriptomic analyses DNA microarrays were used to define the transcriptional profiles of 12 tumor samples with a RIN ranging from 8. 1 to 9. 9. Experiments were done as recommended by the manufacturer (Affymetrix, Thermo Fisher) from 100 ng of total RNA for each sample using the GeneChip TM WT PLUS Reagent Kit and the Affymetrix GeneChip TM HuGene 2. 0 ST arrays. Expression data were normalized by RMA with the nonparametric quantile algorithm in R using Bioconductor and associated packages (version 3. 5. 2; http://www. cran. r-project. org/). Supervised analysis comparing the expression profiles between sample classes was done using a moderated t-test with empirical Bayes statistic 100 included in the limma R package  (version 3. 38. 3). Significantly, DEGs were defined by the following thresholds: P-value < 0.
05 and fold change FC > ¦2x¦. Ontology analysis was done by the GSEA algorithm with the Reactome pathway database included in the MSigDB (version 7. 2) 101. GSEA was done with 1000 gene-set permutations as parameters and significantly DEG sets were defined by the following thresholds: P-value < 0. 05 and qvalue < 0. 25.
Western blot Cells were collected and lysed in radioimmunoprecipitation assay buffer. Proteins were resolved by electrophoresis using a Trisacetate gel or 4-12% Bis-Tris gel (Invitrogen). Transferred membranes were blocked in 5% milk and incubated with primary antibody overnight. Antibodies were used against ATRX (Sigma HPA064684, SC10078, and SC1540; 1:800 dilution), DAXX (Sigma HPA008736; 1:800 dilution), Vinculin (T5191; 1:600 dilution), ctubulin (Sigma, T5192; 1:1,000 dilution), and TOP3A (proteintech 14,525-1-AP; 1:600 dilution).
Quantitative reverse transcription PCR KD experiments using siRNA were verified by qRT-PCR. Total RNA was isolated using the RNeasy mini kit (Qiagen) and DNasetreated to remove genomic DNA. Reverse transcription was performed with 2 lg isolated RNA, 500 ng oligo(dT)15 primer, 40 U RNasin, 0. 5 mM dNTPs, and 20 U M-MLV-Reverse Transcriptase (Promega). Quantitative PCR was performed using SYBR Green qPCR Master Mix (Roche) according to the manufacturer's instructions. Primer sequences are supplied as supplementary data (Table  EV4 ). PCRs were performed on cDNA equivalent to 100 ng total RNA and carried out for 40 amplification cycles, followed by melt curve analysis. For each sample, a replicate omitting the reverse transcription step was included as a negative control. Primer PCR efficiencies were calculated by standard curve.
Realtime data were analyzed using the DDC(t) method, with GAPDH as the reference gene, and expressed as fold-change relative to the appropriate control (AESEM). Immunofluorescence and fluorescence in-situ hybridization Cells were grown on cover slips. Slides were subjected to preextraction by incubation in KCM permeabilization solution (120 mM KCl, 20 mM NaCl, 10 mM Tris, 0. 1% (v/v) Triton X-100) for 10 min. Slides were then washed in PBS, fixed at room temperature for 10 min in PBS with 4% (v/v) formaldehyde. For tumor samples, 4 lm paraffin sections were cut and dewaxed. Slides were rehydrated then microwave heated to 120°C in 90% glycerol, 10 mmol/l Tris (pH 10. 5), 1 mmol/l EDTA, and maintained at 110-120°C for 15 min. The slides were cooled and rinsed in PBS. All subsequent treatments of cell lines and paraffin sections were identical.
Automated image analysis ZEN microscopy images (. czi) were processed into extended projections of z-stacks using ZEN desk 2011 software (Zeiss) and imported into Cellprofiler v2. 1. 1 (29) for analysis. The DAPI channel was used to mask individual nuclei as primary objects. Foci within each segmented nucleus were identified using an intensity thresholdbased mask. Any given object was considered to be overlapping another object when at least 80% of the first object's area was enclosed within the area of a second object.
Acknowledgements AN was supported by the  National Institute of Health and Medical Research  ( Inserm, Poste d'accueil ) and received personal support from the  Swiss National Science Foundation  and the  Fondation ARC Pour la Recherche sur le Cancer. VG's work is supported by the  Ligue Nationale Contre le Cancer , equipe labellis ee. We thank the  CRCM Experimental Histo-pathology platform. This work was supported in part by the  National Health and Medical Research Council of Australia   GNT 1088646. Microscopy was performed in the  ACRF Telomere Analysis Centre , supported by the  Australian Cancer Research Foundation  and the  Ian Potter Foundation.
Data availability The datasets and computer code produced in this study are available at the ArrayExpress database with the following accession number: RNA Affymetrix GeneChip: E-MTAB-11743 (Title: Alternative Lengthening of Telomeres (ALT) and ATRX inactivation in highgrade pediatric osteosarcomas. Permanent link: https://www. ebi. ac. uk/arrayexpress/experiments/E-MTAB-11743); CGH: E-MTAB-11747 (Title: Genome profiling by comparative genomic hybridization (CGH) in pediatric osteosarcoma. Permanent link: https://www. ebi. ac. uk/ arrayexpress/experiments/E-MTAB-11747). Expanded View for this article is available online.
In order to avoid potential bias due to topically administered b-blockers, a sensitivity analysis was performed by excluding patients that were using topical bblockers from the group not using oral b-blockers. Finally, RFS according to use of b1-selective and nonselective b-blockers separately was estimated using the KaplaneMeier estimator, as described previously. For all estimated parameters, point estimates and 95% confidence intervals were used. The proportional hazards assumption was verified using the approach by Lin, Wei and Ying  [14]  based on martingale residuals. All tests were performed at a two-sided significance level of 0. 05. SAS software (SAS Institute) was used for all analyses.
Therefore, there was consistency with our findings regarding non-selective bblockers but not selective b-blockers. There is an ongoing early phase Ib/II trial investigating the effects of propranolol hydrochloride, a nonselective b-blocker, when given with pembrolizumab to patients with stage IIIC-IV melanoma that is not surgically resectable (NCT03384836). Abbreviations: HR (hazard ratio). a Cf model indicated in Table  3 , where an interaction term, treatment group x b-blockers use, was added. The present study has a number of strengths, including a well-characterised cohort of patients with radiologically and biopsy proven resected stage III melanoma, a large sample size, prospective design and rigorous standards of data collection and follow-up of patients in this registration study. It is the first study, to our knowledge, to report quantitative estimates for the effects of b-blockers in melanoma treated with pembrolizumab.