dataesr/scientific-paragraphs-categorization huggingface dataset:
We calculated the percentage agreement for each risk of bias item, as the proportion of studies with a concordant assessment in both reviews (eg, 'low' risk of bias AND 'low' risk of bias). Not all reviews assessed all five key risk of bias items for each RCT included; consequently, the number of RCTs evaluated for discrepancies varies depending on the item considered. selection of studies for in-depth analysis of disagreements For workload reasons, we in-depth evaluated the reasons for disagreements for 50% of the studies analysed in the previous step. In cases of more than one shared RCTs within a given pair of Cochrane reviews, we selected only one RCT at random. To reach 50% of the total sample, we used a simple random selection in the remaining database.
strengths and weaknesses Our study goes beyond previous literature on the topic. 3 12-18 26 As compared with most other studies,  [12] [13] [14] [15] [16] [17]  we used real-world data to explore agreement of risk of bias assessments in real scenarios. We evaluated a very large and comprehensive collection of Cochrane reviews that spanned multiple specialties and topics, including a number of trials about 10 times larger than the largest study on the topic. 12 We completed our analysis by searching individual study reports to give support to our comments on reasons for disagreements, which, to our knowledge, has not been done in previous, smaller works that used a similar methodology. 18 While doing this, we developed a suitable classification scheme for sources of disagreements and conducted, in duplicate, an extensive analysis to understand the risk of bias assessment process and explored the most common reasons for disagreements.
Comparison with other studies Our findings confirm the importance of issues that were previously identified by Jørgensen et al  3  and Savović et al. 26  In particular, Savović et al,  26  surveying users of the risk of bias tool, reported on the possibility of confusion between random sequence generation and allocation Open access concealment and between allocation concealment and blinding; the uncertainty on how to address unfeasibility of blinding; and the difficulties in assessing incomplete outcome data especially regarding the acceptable rate of missing data. More recently, Jørgensen et al,  3  evaluating comments on the use of the risk of bias tool, highlighted how authors complained that judgement often originates from incomplete or missing information. A previous study identified 46 RCTs included in different systematic reviews in the field of fertility and evaluated the percentage agreement in risk of bias assessment.
Acknowledgements We thank  Camila Olarte Parra  for her help during the data management phase and her comments on this manuscript. We thank  David Tovey , editor in chief of the  Cochrane Library , for agreeing to share data from Cochrane reviews; Javier Mayoral Campos, system administrator; the  Cochrane Central Executive  for preparing files; and all Cochrane reviewers who collected data. We also thank  Laura Smales  for English revision of the manuscript. Contributors LB was involved in the study conception, selection of trials, data extraction, data analysis, interpretation of results and drafting the manuscript. PB was involved in the study conception, data analysis, interpretation of results and drafting the manuscript. IA was involved in the study conception, data extraction and drafting the manuscript.
Ledipasvir+sofosbuvir ICUR was not reported as the CE Opinion assigned a major 'réserve' i. e. the method was not accepted  [15]. The corresponding Clinical Opinions  [16] [17] [18] [19] [20]  reported ASMR II-III for sofosbuvir and ASMR IV for all other DAAs. The Pricing decisions were taken in November 2014 (sofosbuvir), May 2015 (simeprevir and daclatasvir), June 2015 (ledipasvir+sofosbuvir) and August 2015 (ombitasvircombo and dasabuvir). The systematic review searched the Medline database from January 2010 to August 2015. The 5year timeframe was aligned with the regulatory timetable in the French setting. For this reason, the literature search period was purposely stopped at the time the pricing decisions were made in order to ensure that the information was available for the pricing negotiation.
Modifications in the recommendations for the use of bepridil -avoiding use with other antiarrhythmics and cautious use with diuretics -resulted in an immediate and dramatic decrease in the incidence of arrhythmias. More and more often, a safety issue detected by spontaneous reporting data could be confirmed by pharmacoepidemiology studies. For example, case reports of acute renal failure with the concurrent use of diuretics, drugs blocking the renin-angiotensin system (angiotensin converting enzyme inhibitors or angiotensin receptor blockers) and nonsteroidal antiinflammatory drugs were particularly observed in older patients in pharmacovigilance  [36] [37]. A nested case-control study in a large primary care database containing longitudinal data on patients' medical history, smoking, alcohol use, body mass index and indication for use of antihypertensive drugs, confirmed the risk of acute kidney injury with triple therapy, particularly during the first 30 days of use  [38].
In another example, deaths occurring in older adults with dementia using antipsychotics were reported in pharmacovigilance; the increased risk of mortality was confirmed by meta-analyses of randomised controlled trials and observational studies  [39] [40] [41]. This detection and confirmation of safety concerns contributes to drug safety through black box warnings, changes in drug labelling and even withdrawal from the market. Although spontaneous reporting provides valuable information in drug safety, it presents inherent limitations, including under-reporting, difficulty in identifying low risks, and the frequent unfeasibility of quantifying risk. Several methods are therefore proposed for safety signal detection in healthcare databases such as disproportionality analysis, traditional pharmacoepidemiology designs (e.
g. self-controlled designs), sequence symmetry analysis (SSA), sequential statistical testing, temporal association rules, supervised machine learning and the tree-based scan statistic  [42]. For example, SSA is a method applied on computerized claims data for detecting adverse drug events based on the concept of prescribing cascade. SSA analyses the sequences of medications; if one medication (drug B) is more often initiated after another medication (drug A) than before, it may be an indication of an adverse effect of drug A  [43]. Founded on a simple principle, SSA has the ability to provide a risk estimate that may be useful to characterise the signal detection in administrative database and support decisions  [42].
However, recent advances in pharmacoepidemiology studies fill the gap left by RCTs, when conducted under rigorous methodological standards  [45]. Pragmatic trials, case-control or cohort studies conducted with healthcare databases have been specially adapted to the constraints of drug assessment in real-life. New approaches have been developed to control for confounding in observational studies such as marginal propensity scores and structural models  [46]. Mixed-effect modelling or time-dependent Cox models are likely to produce biased estimates in the presence of time varying confounders affected by prior exposure (e. g. by a change in drug exposure over time). G estimation, inverse-probability-of-treatmentweighting (IPTW) and parametric G-formula methods are better alternatives  [46]. Network analysis is a novel methodology for analysing effectiveness of drugs and may offer interesting perspectives  [47].
For the elliptic case, we use an Euler-Maruyama scheme and for hypoelliptic models, a 1. 5 order scheme to propagate the noise into the observed state variable  (Kloeden and Platen 1992). The mean and standard error (SE) of the estimators are reported as well as the mean computionnal time. Our method was coded with Matlab, the required time was computed by using the function 'cputime'. Optimization required for inner and outer criteria minimization were both conducted using the function 'fminsearch' (a derivative free simplex search method). The filtering based methods was implemented in R. The simulations have been run on a personal Dell machine using one core Intel(R) Core (TM) i7-4790S CPU @ 3. 20Ghz. Three models are used: the Lotka-Volterra model, the FitzHugh-Nagumo model and the Harmonic Oscillator (HO).
Methods This was a retrospective cohort study of all 2015 discharges from acute-care inpatient hospitalizations in a tertiary-care university center comprising four hospitals. The study endpoint was unplanned 30-day all-cause rehospitalization via emergency departments, and we collected sociodemographic, clinical, and hospital characteristics based on hospitalization database computed for reimbursement of fees. We derived a predictive rehospitalization risk score using a split-sample design and multivariate logistic regression, and we compared the discriminative properties with the LACE index risk-score.
Conclusion Our findings indicate that the possibility of unplanned rehospitalization remains high for some patient characteristics, indicating that targeted interventions could be beneficial for patients at the greatest risk. We developed an easy-to-use predictive rehospitalization riskscore of unplanned 30-day all-cause rehospitalizations with satisfactory discriminant properties. Future works should, however, explore if other data from electronic medical records and other databases could improve the accuracy of our predictive rehospitalization risk score based on medico-administrative data.
Study design This was a retrospective cohort study of all acute-care inpatient hospitalization cases discharged from January 1 to December 31, 2015, from the largest university health center in south France (Assistance Publique-Hôpitaux de Marseille, APHM). All data were collected from the APHM Hospital database computed for reimbursement of fees including both the PMSI (PMSI-Programme de Médicalisation des Systèmes d'Information) database as well as administrative data. The PMSI is the French medico-administrative database for all hospitalizations based on diagnosis related-groups (DRG) that we can group into significant diagnostic categories.
Study outcome The study outcome was unplanned 30-day all-cause rehospitalization, defined as any cause of admission via emergency departments in any acute care wards within 30 days of discharge. To calculate this outcome, the unique and individual identifying variable was used to track rehospitalizations 30 days following discharge. We excluded patients with identification problems in the database. No more than one rehospitalization for each discharge was taken into account. Readmission via the emergency department was employed to identify unplanned rehospitalizations  [30].
The AUC of the predictive rehospitalization risk-score was then compared to the AUC of the LACE index score  [38] , computed on the same database. To compare these two AUCs, we used Chi-square statistic developed by Go ¨nen  [39]. In addition to the AUC metric, we used other metrics based on a threshold value determined as the best trade-off between sensitivity and specificity. For this cutoff, we provide the following metrics for each score (i. e. , the predictive rehospitalization risk-score and the LACE index): sensitivity, specificity, the Youden Index, the accuracy and the F1 score: Sensitivity ¼ true positive true positive þ false negative Specificity ¼ true negative true negative þ false positive Youden index score ¼ sensitivity þ specificity À 1 Accuracy ¼ true positive þ true negative N Precision ¼ true positive true positive þ false positive F1 score ¼ 2 � sensitivity � precision sensitivity þ precision Then, we assessed the robustness of the predictive rehospitalization risk score.
Following the methodology proposed by Tuffery  [37] , the data were split into two samples: a training sample including 2/3 of the data and a test sample including the remaining 1/3. Using the training datasets, risk scores were built for each independently-associated factors of 30-day allcause rehospitalization, previously determined and using the multivariate logistic regression. The AUCs obtained for the training and testing datasets were then compared. The model was considered robust if the AUCs between the testing and training datasets were similar. Lastly, we computed the 30-day rehospitalization rates to each class (10 by classes) of the predictive risk score and test the association using the Chi-square test. Statistical significance was defined as p <0. 05. The statistical analyses were performed with Statistical Analysis Software (SAS), Version 9.
Ethics and consent to participate The datasets generated and/or analyzed during the current study are issued from the Assistance Publique des Hopitaux de Marseille (APHM) hospitalization database computed for reimbursement of fees. Patients are informed by the hospital that their data may be analyzed for research purpose, consequently respecting the French law for research that does not require explicit or written consent of the patient. No additional data out of this database has been computed for the study. The use of such database is governed by a local ethic committee (named CIL-APHM) and declared under the following authorization numbers 1305855 for medical data and 2012-1 for administrative ones. According to the French law, there is no need to ask to another relevant ethical review board or relevant regulatory body.
Rates of unplanned 30-day all-cause rehospitalization A total of 289,358 hospitalizations (112,662 patients) were recorded in 2015 in this French University Hospital. After excluding mortalities and hospitalization in ambulatory hospitalizations care unit (ambulatory surgery, radiotherapy, dialysis, chemotherapy, transfusions), 118,650 hospitalizations (82,862 distinct patients) were included. The most common diseases were digestive disease, nervous system conditions, and cardiovascular and pulmonological diseases. We excluded 4 patients with identification problems in the database. In total, 4,127 (3,294 distinct patients) hospitalizations resulted in rehospitalizations via emergency departments 30 days after discharge (30-day re-hospitalization rate equal to 3. 5%). Thirty-days rehospitalization rates according to socio-demographic, clinical, and hospitalization characteristics are presented in Table  1.
Lastly, return to home was associated with more rehospitalizations, thus confirming the necessity for clinicians to better prepare discharges, check the availability of home-based services, and carefully plan the transition of care. This is clearly a weak point of the FHS  [16, 43]. Lastly, our final aim was to propose a predictive rehospitalization risk score and, to our knowledge, this was the first unplanned 30-day rehospitalization risk model to use an understanding set of factors sourced directly from the French hospital medico-administrative database. This score is easy to use, accurate in predicting the risk of rehospitalization via emergency departments, it already presents higher discriminative properties than the LACE score (c statistics = 0. 74 vs. 0. 66 for the LACE index), despite being recommended by the French Health Authority.
The French policy is mainly based on the publication of guidelines by the French Health Authority (HAS)  [2] , which recommends identifying patients at risk using the LACE index  [34]  or 8Ps risk assessment tools used in the BOOST (Better Outcomes for Older adults through Safe Transitions) program involving 11 hospitals in the USA  [58]. Despite the interest of these two instruments, they have not yet been rigorously validated in France. The LACE index presented poor discriminative ability starting from its construction  [18, 34]  and the authors themselves later improved it (AUC for 30-day urgent readmission between 0. 743 and 0. 753, depending on the inclusion of case mix groups) by adding other covariates closer than those used in our model but validated on the Ontario administrative database  [35].
The 8Ps check list issued from the BOOST could be cumbersome in routine practice if performed for every hospitalized patient since it requires physicians to identify and address each of these factors then propose an appropriate intervention  [59]. An important advantage of our predictive rehospitalization risk score is that it does not require additional completion by physicians to that already required for the PMSI; consequently, even if analyzing such a medico-administrative database may require computational aid, it is important to use the data already available and not to increase doctors' medico-administrative work burdens  [60].
An external validation in addition to the internal validation performed in this study will guaranty the generalizability of this score. Our model does not take into account deaths outside the hospital since we do not dispose of this information in our database. Other studies with available data on outpatient events are needed to investigate to what extent this could impact our predictive risk score using a competing risk model as an example. In this study, we excluded from the analyses the ambulatory surgery. However, this specific topic should be better studied in the French context, strongly marked by pressures for reducing length of stay and consequently cost of care. Even if our predictive risk-score led to better AUC score than other scores already performed, other and more advanced methods like machine learning are advocated to investigate whether they can give better predictive power than our score derived from the logistic regression.
The advantages of these methods is their ability to use more information that cannot be used with classical statistical methods such as logistic regressions, and in particular textual information in electronic patient records. Future works should use all the data relevant in hospital databases and these new methods to improve the level of prediction. A substantial amount of data (e. g. , polypharmacy, socio-economic status, and self-reported functional status) has been reported as predictive of rehospitalization  [9, 20, [61] [62] [63] , though it is not currently available in the French PMSI database. Future works should explore how to systematically collect this data in the other available databases in hospitals (e. g. , electronic medical records) and if such data could improve the accuracy of our predictive rehospitalization risk score based on the PMSI database.
Conclusion The 3. 5% unplanned rehospitalization rate was substantially lower in our study than that of studies performed in other countries, suggesting that universal insurance coverage may be a key factor for controlling rehospitalization. Despite this low unplanned rehospitalization rate, our findings likewise indicate that the possibility of unplanned rehospitalization remains high for patients with certain characteristics, suggesting the interest of proposing targeted interventions for patients at the greatest risk. We also developed an easy-to-use predictive rehospitalization risk-score of unplanned 30-day all-cause rehospitalizations with satisfactory discriminant properties. Future works should, however, explore if other data available in electronic medical records and other databases could improve the accuracy of our predictive rehospitalization risk score based on medico-administrative data.
This workstream highlighted the potential use of stated-preference methods, patient-reported outcome (PRO) questionnaires, and qualitative research methods to gather data from individuals. Stated-preference methods are used to assess patient preference for medical interventions or certain attributes of these interventions. The workstream acknowledged the strength of such methods in that they can be used with large and diverse populations to identify subgroups of individuals with similar preferences and also rare benefits and harms. However, these methods gather information on risks (anticipated harms) based on hypothetical scenarios as opposed to actual patient experience  [1]. To capture patients' actual experiences of medical interventions, the workstream suggested the use of PRO questionnaires, qualitative research, or a combination of both methods as part of a mixed-methods approach  [1].
Use of the PQAT in the DRI13940 Study The revised PQAT was administered as an exploratory endpoint in a dose-ranging phase II clinical trial (DRI13940, ClinicalTrials. gov identifier: NCT02973321  [32] ). The purpose of this study was to assess a novel dual glucagon-like peptide-1 receptor (GLP-1R)/glucagon receptor (GCGR) agonist developed for the treatment of overweight/obese patients with T2DM inadequately controlled by diet/exercise and/or metformin.
Participant Sample Eligible patients had been diagnosed with T2DM at least 3 months prior to screening and had experienced an inadequate response to diet/exercise and/or treatment with metformin (≥ 1500 mg/day or at least the maximal tolerated dose). Trial treatment was self-administered via subcutaneous injection. All patients were between 18 and 80 years of age. Full exclusion criteria for the study can be found at ClinicalTrials. gov (NCT02973321). The PQAT was only available in English and so was limited to English-speaking participants in the USA and Canada.
Analysis of Qualitative Data All identifiable information was removed from the PQAT data prior to analysis. Participant free-text answers were coded and grouped according to concept using methods derived from thematic analysis  [21] [22] [23]. Codes were used to capture the relevant concepts described by participants in relation to each of the three qualitative questions. The data were coded manually by two experienced qualitative researchers (authors KB and AF) in Microsoft Excel ® (Microsoft Corp. , Redmond, WA, USA). Data from the first 10% of participants (n = 6) were coded by both researchers independently. Assigned codes were then compared and discussed to reach a consensus on a provisional code list. The accuracy and consistency of the coding across data were assessed by an additional researcher (author AG).
Analysis of Quantitative Data Data extraction tables were populated with categorical responses to PQAT quantitative items at the 1 As recommended for any PRO measure capturing aspects of drug toxicity, investigators from the trial did not have access to the PQAT data of patients to avoid bias/influencing patient responses of adverse events. Unlike traditional safety information, PRO data do not involve clinical judgment  [19]. The PQAT information was therefore considered tolerability information, reflecting "the extent to which overt adverse events impact patient's willingness to remain on the current treatment dose"  [20]. Subsequently, data from the PQAT were considered complementary to usual safety/adverse event reporting made by investigators directly, and not included in the safety clinical database.
Ethical approval and informed consent The DRI13940 study protocol and subsequent substantial amendments were approved by local independent ethics committees and the competent regulatory authority in each country for each investigational site. The study was registered with EudraCT (2016-001328-77) and ClinicalTrials. gov (NCT02973321), and conducted according to Good Clinical Practice guidelines, any local guidelines, the Declaration of Helsinki, and European Directive. The study did not include any procedures performed on animals. Written informed consent was obtained from all individual participants included in the study. Open Access This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4. 0 International License (http://creat iveco mmons. org/licen ses/by-nc/4.