dataesr/scientific-paragraphs-categorization huggingface dataset:
Most of the literature relates to CT and MR images, and there is limited research on PET image quality assessment. Varying image qualities were generated by reconstructing images with different simulated injected doses such that all other factors that may affect image quality, including reconstruction algorithm and system design, were fixed, and only the dose level determined the differences in image quality. The performance of an established VGG16 network  [24]  pre-trained on the ImageNet-1K database  [21]  was assessed and compared to a pre-text learning task (using an easily extracted label, the injected dose) as a form of pre-training to predict three clinically scored metrics. These metrics are: global quality rating (GQR), which evaluates the aesthetic component of the reconstruction, and takes into consideration qualities such as noise level and resolution; pattern recognition (PR), which determines whether the reconstruction allows for the clinician to determine the presence of any pathological patterns suggesting a particular diagnosis; and diagnostic confidence (DC), which concerns the certainty with which the clinician can use the reconstruction to observe these patterns and make a diagnosis.
As low count reconstruction causes the degradation of PET image quality, these metrics were designed by the authors to encompass the necessary considerations involved in using these low count clinical PET images for making a confident and accurate diagnosis. These three metrics are scored as: 0 = "unacceptable", 1 = "poor but usable", 2 = "acceptable" and 3 = "good/excellent", with 0. 5 ratings accepted. The aim was to reduce the need for clinicallyassessed training data by the use of pre-trained networks and transfer learning.
II. METHODS AND MATERIALS A. Real Patient Data Data from PET-MR scans of 21 memory clinic patients with suspected dementia were used. Each patient underwent an [ 18 F]FDG-PET and a simultaneous 3D T1-magnetisationprepared gradient-echo (MP-RAGE) MRI scan on a 3T PET-MR scanner (Biograph mMR, Siemens Healthcare, Erlangen, Germany). Siemens e7 tools was used for image reconstruction, using the clinical standard of the ordered subset expectation maximisation (OSEM) algorithm with 2 iterations and 21 subsets to reconstruct 3D images of 344 × 344 × 127 voxels. It is possible for the list-mode PET data to be resampled  [45] ,  [46] , and such methods can be used to simulate seven lower injected doses (Fig. 1 ). To simulate the lower injected doses, the prompts and random coincidences in the corresponding listmode PET data for the 100% count reconstruction were randomly sampled.
Subsequently, the corresponding emission, normalisation, randoms and sinograms were re-calculated along with the attenuation sinograms using Siemens e7 tools. The listmode PET data from all datasets were resampled  [46] , simulating seven lower injected doses (Fig. 1 ). Images were normalised between 0-1 to reduce standard deviations, suppressing the impact of outliers, then blinded and randomised prior to clinician scoring. The simulated dose levels were grouped into "low quality" (0. 5% and 1% of counts), "medium quality" (5% and 10% of counts), and "high quality" (25%, 50% and 100% of counts) categories to decrease clinical assessment times. In total, ten patients at three different count levels (one per category) were assessed by an experienced clinician. For calibration, nine images were scored twice by the same clinician in different scoring sessions (Fig.
This number of patches was chosen as it is proportional to the brain:background ratio in the whole image volume. Fig. 1. Cropped example transverse slices from "low", "medium" and "high" quality reconstructions for two independent datasets. 3. A clinically assessed image with a 20-point colour scale with a lower global quality rating (1), but higher pattern recognition (2) and diagnostic confidence (2) scores (corresponding to the top-right subplot in Fig. 1 ). The colourscale is the technique by which the clinician determines the pattern recognition and diagnostic confidence ratings.
5 ). The three original fully connected layers of the VGG16 architecture were replaced with a single fully connected layer with three outputs relating to each of the clinical metrics (GQR, PR and DC) with ReLU activation (Fig. 5 ) in Section II. D. This is because the original VGG16 network is used for classification of the ImageNet database into 1000 different classes. However, this is a regression task, with the output a prediction of three clinical quality scores. Therefore, the original projection head was replaced by another fully connected layer to achieve this task. In Section II. E, the replacement fully connected layer had a single output corresponding to the simulated dose. The same seven patients were used in the transfer learning dataset, reconstructed at three simulated dose levels each.
Greyscale input patches were replicated along one axis to obtain three channels as VGG16 inputs are RGB images. The VGG16 network (Fig. 5 ) was initialised with random weights, with a learning rate of either 10 -3 (default) or 10 -4 as a baseline comparison model. The VGG16 network was then also initialised using ImageNet weights, and trained by sequentially unfreezing weights from consecutive blocks starting from training zero convolutional blocks (only the fully connected layer) up to and including all five convolutional blocks. Each model was trained seven times and seven patients were used in the transfer learning dataset. This model was also tested as a function of the number of patients in the transfer learning dataset; a maximum of 6300 patches were used for transfer learning.
G. Implementation Reconstructions via Siemens e7 tools were carried out in MATLAB (The Mathworks, Inc. ). All models were trained using the Keras  [18]  application programming interface implemented in TensorFlow  [48]. Training was accelerated using either an NVIDIA Tesla K40 12GB GPU or NVIDIA Quadro RTX6000 24GB GPU. All models used the Adam optimiser with a learning rate of 10 -4 (unless otherwise stated), a mean-squared-error loss function and batch size of 10. Training occurred for 1000 epochs, and models were saved at the lowest validation loss which was used for analysis.
A. Influence of the Number of Retrained Blocks Based on the three test patients, Fig. 8  shows a series of boxplots describing the evaluation metrics as a function of the number of pre-fully connected layer blocks when seven patients at three dose levels each were used in the transfer learning dataset. Re-training the last one or two blocks outperformed training the fully connected layer only, but there was no clear benefit to retraining the last 3-5 blocks, as percentage close agreement decreased. Re-training a higher number of convolutional blocks, generally increased the variation across separate training runs, without further decreases in the mean or median absolute error. There is a plateau in the MAE around 0. 5, in line with Fig. 2.
B. Influence of the Number of Patients in the Transfer Learning Dataset Fig. 9  shows a series of boxplots describing the same metrics as in Fig. 8. Based on Section III. A, the last two convolutional blocks of the VGG16 backbone were re-trained. Generally, the MAE and the associated variation decreased with increasing numbers of independent training datasets, but plateaued around 0. 5, again in line with Fig. 2. Using one patient in the transfer learning dataset was consistently outperformed across all evaluation metrics by more independent patients in the transfer learning. There was clear improvement in the percentage exact agreement with increasing numbers of independent training patients.
This is because when none of the convolutional blocks are trained, there remain high-level features specific to the ImageNet dataset and not to medical images, and similarly, when all parameters are trainable, this negates any pre-training benefit  [49]. Unfreezing the final two convolutional blocks' weights resulted in the best percentage exact and close agreements, suggesting a balance between the number of high level features and pre-training benefit, agreeing with the work of Yosinski et al. [49]. Fig. 8  informed the investigation into the number of patients in the transfer learning dataset where the last two convolutional blocks of the VGG16 backbone were re-trained. The expected trend was that with more patients in the transfer learning dataset, the better the network performance on unseen datasets as there are more examples from which the model can learn  [50].
Fig. 9  indeed shows that increasing the number of patients in the training dataset improved the percentage exact agreement and decreased the MAE, supporting the idea that a larger dataset can improve generalisability to unseen datasets. Dunnmon et al. [25]  showed that increasing the number of 2D chest X-ray images from 2000 to 20,000 improved the average area under the receiver operating characteristic curve (AUC) from 0. 84 to 0. 95, but improvements plateaued afterwards, as using 200,000 images, only achieved a non-significant increase in AUC to 0. 96. Results from the use of just one patient in the transfer learning dataset were consistently outperformed by using ≥2 patients, suggesting that use of one patient in the dataset was insufficient to generalise to unseen data. The percentage close agreement started to plateau at ~70% beyond ≥3 independent training patients, but the percentage exact agreement continually improved.
This suggests that, whilst three patients are sufficient for predictions within 0. 5 of the clinician-scores, a larger dataset (i. e. >7) may be required to achieve exact agreements between the true and predicted scores. For both Fig. 8  and Fig. 9 , the MAE plateaued at ~0. 5 corresponding to the inter-session scoring uncertainty within the clinical assessments of the images (Fig. 2 ). A total of 27 images were re-assessed by the same clinician, with 44% of the readings scored at least 0. 5 different to the original assessments. Thus, obtaining a MAE of approximately 0. 5 may be the optimum achievable for this task. Table  II  shows that using pre-training via ImageNet outperformed no pre-training for dose inference. Informed by Fig. 8 , pre-training VGG16 with ImageNet weights and retraining the last two blocks was used as the pre-trained network.
It has also been shown in Table  III  that it is not possible to only use injected dose standardised by patient weight as input into a machine learning network. Despite this model being much smaller, and using an easily available label, the best results were achieved when using PET Future work may include training and testing with a larger number of clinically-scored images, to observe the trend beyond seven separate transfer learning datasets. This work could also be applied to other low-dose reconstruction algorithms such as KEM  [7]  and MR-guided methods  [52]  or novel regularisation strategies  [53]. As these methods are superior to standard OSEM, they should give higher quality predictions, even at lower simulated doses. However, similarly to  [31] , this may need to involve the inclusion of reconstructions generated via these algorithms into the training dataset.
V. CONCLUSION This study shows that pre-training can help improve network performance in the task of automatically predicting clinical image quality assessments of PET images. By transferlearning using ImageNet weight initialisation and retraining the last two convolutional blocks of a VGG16 backbone, a superior model performance is achieved for this clinical task. Additionally, this method does not require further independent patient datasets, unlike for using a pretext learning task. Overall, this work shows that pre-training on natural images for application to medical images has the potential to provide benefit for regressing from images to clinically-scored metrics.
ACKNOWLEDGMENT The authors would like to thank  Dr Veerle Kersemans  for her guidance and support,  Dr Zuzana Balazova  for her work on data acquisition and processing, and  Dr Abolfazl Mehranian  for his work on low-count PET reconstruction. All authors declare that they have no known conflicts of interest in terms of competing financial interests or personal relationships that could have an influence or are relevant to the work reported in this paper.
This work was funded by the  EPSRC Centre   for Doctoral Training in Smart Medical Imaging  [# EP/S032789/1 ] and  Siemens Healthcare, Erlangen, Germany. This work was supported by the  Wellcome/EPSRC Centre for Medical Engineering at King's College London  ( WT 203148/Z/16/Z ) and the  Department of Health  via the  National Institute for Health Research (NIHR)  comprehensive  Biomedical Research Centre award  to Guy's & St Thomas'  NHS Foundation  Trust in partnership with  King's College London  and  King's College Hospital NHS Foundation Trust. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. This work involved human subjects in its research. Approval of all ethical and experimental procedures and protocols was granted by the institution's  Research and Development department , the  North East -York  research ethics committee (reference  15/NE/0203 ), and the radiation protection committee (ARSAC).
MetricPre-trained Network Dose/Weight to Clinical Scores (LR = 10 -3 ) Dose/Weight to Clinical Scores (LR = 10 -4 ) in the transfer learning dataset, evident by the number of green boxes in the table. Whilst most prior research is on binary classification and not in PET imaging, our maximum Spearman's rank correlation coefficients of 0. 96 for GQR, 0. 67 for PR and 0. 59 for DC are comparable to that achieved by Coyner et al. [43]  (0. 86 -0. 92). GQR PR DC GQR PR DC GQR PR DC % Exact Agreement 56 22 11 0 0 0 0 0 0 % Close Agreement 89 44 33 89 56 33 78 56 33 MAE 0. 20 0. 64 0. 71 0. 29 0. 65 0. 89 0. 28 0. 64 0. 88 Spearman's Rank 0. 97 0. 64 0. 52 0. 39 0. 66 0. 64 0. 39 0. 66 0. 64 images.
The Benjamini-Hochberg procedure adjusted p-values to control the false discovery rate and pairs of anomalies with adjusted p-values < 0. 05 were identi ed. A total of 1,386 combinations of two anomalies were analysed. Out of the 31 statistically signi cant positive associations identi ed, 20 were found to be known associations or sequences already described in the literature and 11 were considered "potential new associations" by the EUROCAT Coding and Classi cation Committee. After review of the literature and detailed examination of the individual cases with the anomaly pairs, six pairs remained classi ed as new associations. In summary, systematically searching for congenital anomalies occurring together more frequently than expected using the EUROCAT database is worthwhile and has identi ed six new associations that merit further investigation.
Multiple congenital anomalies may also occur as a consequence of a single primary anomaly (for example, the Potter sequence resulting from renal agenesis and with secondary lung hypoplasia and clubfoot). It is therefore important to identify cases with two or more congenital anomalies in different organ systems, where the pattern of anomalies has not been recognised as part of a syndrome, known association or sequence as these could indicate unknown teratogens or new associations. Around 2% of all births have a congenital anomaly, but multiple congenital anomalies occur in around 16 per 10,000 births, with speci c combinations of anomalies being even rarer  (7). Therefore, it is necessary to analyse the data from large datasets with su cient cases. The aim of this study was to analyse data from the EUROCAT network of congenital anomaly registries collected from births between 2008 and 2016.
A web-based system for review of cases has been developed, which allows easy and fast review of many cases and transfer of the nal decision back to the central database. If two geneticists agreed on a case classi cation, this was considered the nal decision. If all three geneticists disagreed or one of them classi ed the case for query, the moderator made the nal decision. Thirty-two full member registries covering 6,599,765 births provided 154,154 cases with one or more major congenital anomalies born between 2008 to 2016. Cases with chromosomal and genetic syndromes, skeletal dysplasias or hereditary skin disorders were excluded resulting in 123,566 cases for inclusion in this analysis.
The data analysed in our study rstly only compared the occurrence of a pair of anomalies within cases that had at least two anomalies and therefore the tendency to cluster did not need to be adjusted for in the rst set of analyses. The second analyses did compare pairs of anomalies to cases with only one anomaly and as expected the odds ratios were higher. However, when adjusted p-values were calculated, a similar set of anomalies was statistically signi cant at p < 0. 01. A second important difference between the method adopted by Benjamin et al (2019)  (9) , was that in this analysis we excluded any cases with known chromosome or genetic anomalies. We wanted to identify any new anomaly clusters -we were not interested in identifying known associations. The strength of this study was that it was based on data from 32 EUROCAT full member congenital anomaly registries covering 6,599,765 births between 2008 to 2016.
EUROCAT has standardised methods of coding and data cleaning which are adopted by all member registries and the data quality is monitored by the use of data quality indicators. The EUROCAT multiple congenital anomaly algorithm identi es all cases with two or more non-genetic major congenital anomalies in different organ systems, where the pattern of anomalies has not been recognised as part of an association or sequence. A limitation of the study was that it was not possible to obtain detailed genetic information on all cases -the researchers were.
ACKNOWLEDGEMENTS This work has received funding from the  European Unions   Horizon 2020 research and innovation programme  under grant agreement No. 101004159  (SERPENTINE, www. serpentine-h2020. eu). Part of this work was performed using the DiRAC Data Intensive service at Leicester, operated by the  University of Leicester IT Services , which forms part of the  STFC DiRAC HPC Facility  (www. dirac. ac. uk), under the project "dp031  Turbulence, Shocks and Dissipation in Space Plasmas ". MHD simulations have been performed on the Newton HPC cluster at the  University of Calabria , supported by " Progetto STAR 2-PIR01 00008 " ( Italian Ministry of University and Research ). L. S. -V. is supported by the  Swedish Research Council (VR)   Research Grant   N. 2022-03352. D. B. is supported by  STFC  grants.
The 2018-2019 National Prostate Cancer Audit data show that low-risk prostate can-cer comprises 6% of new diagnoses  [5]. However, in the USA, where PSA testing is much more widespread, low-risk disease comprises one-third of new diagnoses, and Gleason 6 disease accounts for more than half of the diagnoses of localised prostate cancer in a series assessing the risk profile of prostate cancer at diagnosis over 10 years  [6]. The United States Preventative Services Taskforce recommendations in 2008  [7]  led to a reduction in the proportion of men seen with low-risk disease, according to the National Cancer Database, with a reduction from 38% in 2004 to 27% in 2014. The biopsy and PSA thresholds for entry to AS vary, and there is also significant variation between published protocols and usual practice, particularly in the uptake of routine biopsies after the 1st year or 2nd year of AS.
Although all established guidelines include the use of clinical staging by digital rectal examination (DRE), it is common in some centres for MRI to be used for staging, with DRE not done where MRI is used. We know that there is significant variation in adherence to different protocols  [8]. The use of MRI, and subsequent MRI-targeted biopsies, before enrolment and during follow-up differs significantly between different settings. Even for those who regularly use MRI, there are no agreed MRI features that denote magnetic resonance characteristics unsuitable for surveillance. Movember is a global men's health charity that has invested in prostate cancer research since 2005. Movember's investment in AS to date has largely focussed on the Global Action Plan (GAP3) Prostate Cancer Active Surveillance consortium and database  [9] [10] [11] [12] [13].
Drawing on incomplete contracting and financial intermediation theory, we explore two potential frictionreducing mechanisms: (1) increased liquidity in the secondary market for patent assets, which could alter lender expectations of salvage value, and (2) the ability of an intermediary (here, the venture capitalist) to credibly convey to a lender that he/she will support the fledgling company. Examining the market for venture lending poses a number of challenges. The most significant of these is the nature of the available data. As  Ibrahim (2010)  notes, venture debt transactions and information about the parties involved are sparsely reported. Regulators do not require venture lenders to publicly disclose information about their loans. Venture loans also are rarely syndicated, which reduces coverage in lending databases such as DealScan.
Conversations with lenders further suggest that these transactions are not systematically reported in standard venture capital databases. In light of these challenges, we use an alternative approach developed in  Serrano (2010)  that identifies debt through patents, a common form of collateral used to secure the loans. Venture lenders typically require a blanket lien on assets, including but not limited to patents  (Mann, 1997; Gordon, 2013). When the collateral includes patents, lenders have strong incentives to record the security interest with the U. S. Patent and Trademark Office (USPTO) both in order to establish their secured-lender status and to convey the lien on assets to other potential lenders  (Haemmeli, 1996; Mann, 1997). 2  We use this paper trail of recordation to map startups to loans through patents, thus revealing startup-level lending difficult to glean from other sources.
Serrano (2010)  and  Galasso et al. (2013)  document active trading in the modern market for patents, particularly for inventions originating from individuals and small companies. The implications of patent trading for innovation financing is unexplored in prior research, a gap that this study helps fill. The remainder of this paper is organized as follows. We first discuss relevant insights from the incomplete contracting literature and their relevance for venture lending. We then describe the sample and data used in the study. Finally, we present and discuss our empirical analysis and findings.
Sample Construction Our sample is drawn from the universe of U. S. venture capital-backed firms reported in Dow Jones' VentureSource (aka "VentureOne") database in three innovation-intensive sectors: software, semiconductor devices, and medical devices. Software and semiconductor device startups were sectors hard-hit by the technology bubble's collapse, while medical device startups were relatively shielded from its effect. Focusing on startups that eventually receive VC financing allows us to observe when each company first received a VC equity infusion and from whom they received such investment. We then select all startups founded from 1987, the first year of comprehensive reporting in VentureOne, through 1999. The latter cut-off captures the youngest cohort at risk of being affected by the market crash in early 2000, and provides a common decadelong window for tracking the startups' activities and outcomes.