dataesr/scientific-paragraphs-categorization huggingface dataset:
Semantic edge detection is a challenging problem which aims to produce multiple edge maps corresponding to different object categories. The design and the training of the related network are the two most powerful levers to optimize its accuracy. To this end, we first refine existing semantic edge detection architectures with low and high level features inside our Edge Pyramid Network, which already outperforms state-of-the-art networks on popular benchmark datasets. Then, we apply an ingenious fine-tuning strategy, which incrementally improves edge detection accuracy through a few alternated training cycles, using various loss functions to perform thick and thin edge detection. We observe that applying separately any of these refinements outperforms such networks as SEAL and STEAL for category-aware edge detection.
More remarkable is the fact that these two refinements provide accumulated benefits on the edge detection performance (+0. 85% and +1. 78% on MF-scores compared to STEAL on SBD and Cityscapes datasets respectively). This improvement is visible in the edge maps in which EPN architecture results in smoother edges and the fine-tuning strategy is responsible for thinner edges. Furthermore, we show that EPN architecture achieves competitive performance (ODS F-measure of 0. 828) against the state-of-the-art category-agnostic edge detection networks on the BSDS500 dataset.
The first one is the class-imbalance between edge and non-edge pixels. To address this problem, using the weighted cross entropy loss is a common choice of edge learning networks in  Xie and Tu (2015)   [2, [6] [7] [8] [9]. However, this strategy has a pernicious side effect: it makes detected edges thicker than ground truth, i. e. more false posi-tive edges are produced. The second issue is the noisy humanannotated labels in annotation dataset. Correcting label noise during training is needed to deal with misalignment, as described in  [3, 4]. In this paper, we aim to detect thin and accurate edges using an effective fine-tuning strategy, in which a novel weightednegative loss is proposed to address the misalignment issue. Our strategy is able to capture thin edges and gradually improve the accuracy after each training cycle.
1. From left to right: original image, ground truth label corresponding to the category of motorbike, thinned edge maps after NMS of CASENet  [2]  , SEAL  [3]  , STEAL  [4]  and the proposed network EPN. The aliasing effect is reduced by our network. against the state-of-the-art networks on benchmark datasets such as SBD, Cityscapes and BSDS500 are given in Section 4 , before to conclude.
Because of their small P r(y = 0 | X; W ) , the weighted-negative loss will reduce their contribution to the total loss. Thus, they will be easily accepted as positive edges. As a result, the weighted-negative model M (i +1) w -acts as a thick edge detection module, as shown in Fig. 4. Thin edge detection step Similarly to the thick edge detection step, we fine-tune EPN with the unweighted cross-entropy loss from the weighted-negative model M (i )  w -. Without the modulating factor P r(y j = 0 | X; W ) 2 , M (i )   u focus on the misclassified non-edge pixels which are almost ignored in M (i )  w -. They are also true edge pixels which are not marked as ground truth as green contours in Fig. 3. In other words, the network attempts to eliminate false positive and find more accurate pixels inside thick edges, as illustrated in Fig.
4. From the model obtained in the initialization step M (0) u , we perform alternately and iteratively the thick and thin edge detection steps. The idea of this process is to facilitate the training and to help the network to get a better model after each cycle. There are two reasons behind our strategy: Firstly, there are always misaligned labels in training dataset as illustrated in Fig. 3. Training with the unweighted loss which takes into account all ground truth pixels can be affected by noisy labels. Thus, we believe that an unweighted model fine-tuned from the weighted loss ( M (0) u ) may be not optimal and can still be improved. Our idea is to exclude the loss on noisy negative labels from the total loss by using the weighted-negative loss, as described in Section 3. 3. Training without these noisy pixels can help network get a better optimum ( M (1)  w -) on the remaining pixels.
Datasets and evaluation protocol Semantic Boundary Dataset (SBD) This dataset is composed of 11,355 images from PASCAL VOC2011  [12]  , divided into 8498 training and 2857 test images. This dataset contains 20 categories for semantic boundary detection. Due to the label misalignment, SEAL  [3]  re-annotated 1059 test images with high quality labels for more reliable results. Cityscapes Dataset This dataset consists of 2975 training images, 500 for validation and 1525 for test. The validation images are used for test set because of the unavailability of test labels. In our experiments, the same 19 semantic classes among 30 categories are selected for the benchmark, as others semantic edge detection algorithms  [2] [3] [4]. Evaluation Protocol We use the standard performance metric F-measure (MF) at optimal dataset scale (ODS) for evaluation.
Data augmentation We follow  [3]  to generate ground truth labels for training our model. For data augmentation, we resize images with scaling factors 0. 5, 0. 75, 1. 0, 1. 25, 1. 5 in SBD and keep original resolution in Cityscapes. Hyper-parameters For training, we choose stochastic gradient descent (SGD) for both SBD and Cityscapes datasets with the same hyper-parameters: batch size (1), learning rate (1e-7), gamma (0. 1), iteration size  (10) , step size (10k), momentum (0. 9), weight decay (0. 0 0 05), crop size (352x352). We stop training cycles when there is no more improvement of accuracy on the test set. The iteration numbers of fine-tuning steps are set to 22,0 0 0 and 20,0 0 0 on SBD and Cityscapes, respectively. Each fine-tuning step takes about a day on a single GeForce GTX 1080 Ti GPU.
Results on SBD Table  1  reports the MF scores of semantic edge detection algorithms on the re-annotated SBD test set under Instance-sensitive (IS) and Non-Instance-sensitive (non-IS) modes, respectively. Note that we train and evaluate our network under two different modes for fair comparison with SEAL  [3]  and STEAL  [4]. After three training cycles, our model M (3)   u outperforms SEAL and STEAL by 1. 9% and 0. 85% respectively in MF (ODS). We also report the performance of our network on the original SBD test set following the evaluation protocol from  [12]  , as in Table  2  where our model M (3)   u achieves comparable performance to SEAL (+0. 1%), but worse than STEAL (-1. 1%). The accuracy of our model degrades compared to the edge alignment algorithms due to the noisy ground truth test labels from the original SBD dataset.
Fine-tuning strategy. Fig. 4  shows the intermediate results of our fine-tuning strategy on the SBD dataset. While the weightednegative models reduce gradually false positives, the unweighted MF scores on the re-annotated SBD test set with Instance-sensitive (IS) and Non Instance-sensitive (Non-IS) modes following the evaluation protocol from  [3]. Qualitative results on the SBD dataset can be found in  [25]. Several variants. We investigated several variants of our network such as addition of feature map from the first stage to the pyramid, keeping the 3x3 convolution layer on each map as in FPN, imposing deep semantic supervision on other side outputs, replacing the initialization model CASENet by CASENet -or DSN  [2]. However, they yielded no improvement.
Results on cityscapes As done for SBD, we also trained and evaluated EPN with IS and non-IS modes on the Cityscapes dataset. In Table  4  , EPN achieves better performance compared to SEAL (3. 7%) and STEAL (1. 78%) after three and five training cycles. Note that our first model M (0) u gets MF scores 71. 1% and 71. 5% under IS and non-IS modes, higher than SEAL (69. 1%) and STEAL (71. 42%), respectively. Compared to SBD, our network gives better results in Cityscapes due to the higher label annotation quality in this dataset. Matching distance tolerance We evaluate the crispness of predictions by varying the maximum tolerance allowed for matching contours as illustrated in Table  5. When decreasing the matching distance tolerance from 0. 0035 to 0. 0015, the MF difference between EPN and STEAL increases from 1.
Fine-tuning strategy with CASENet architecture Additionally, we evaluated our training strategy with CASENet architecture on the SBD and Cityscapes datasets using the same training hyper-parameters as in  [2]. As reported in Table  6  , the performance of CASENet is significantly improved on both datasets. On the Cityscapes dataset, CASENet achieves MF = 72. 3% after one training cycle, 4. 2% and 3. 2% higher than CASENet original and SEAL respectively. We note that using the same training strategy, our architecture EPN outperforms CASENet on the SBD (68. 9% compared to 67. 1%) and Cityscapes (72. 8% compared to 72. 3%) datasets.
During the data augmentation, we follow the same procedure as described in  Xie and Tu (2015). Inspired by  [7,8 , 22 , 23]  , we pre-train our network with the PASCAL VOC Context dataset  [24]  before fine-tuning with the BSDS500. Finally, we also use the NMS and the multi-scale testing for evaluation as in other works  [8 , 9 , 22 , 23]. Results. Table  8  reports the two standard measures: ODS and OIS (per-image best threshold) of the deep learning based contour detection algorithms on BSDS500 dataset. Our network gives ODS =. 828 and OIS =. 844 in multi-scale testing, competing with others. Note that if we only use BSDS500 dataset for training without additional data from PASCAL VOC, EPN achieves state-of-theart results on F-measure ODS (. 814) and OIS (. 829) in single scale prediction.
Testing of different backbones On the other hand, the second setting S2 outperforms S1 for 12 out of 18 configurations. For the others, both settings obtain the same results. Thanks to its larger coefficients, the S2 setting allows the network weights to fluctuate more than S1 during training, thus offering a greater chance of finding a better optimum than with the S1 setting. On the other hand, if these coefficients are too large, the network may diverge or converge to a local minimum. As illustrated in 7 , the S2 setting helps the network to find more contours inside the aircraft than S1. Ablation study Here we train the networks on the BSDS500 base without using the PASCAL VOC dataset and the fine-tuning strategy. The ablation concerns the convolution and deep supervision layers and several options have been tested ( Table  10  ): EPNL1: if we keep only one deep supervision layer applied on the P1 output map, EPN-L1 network produces worse results than EPN network on all metrics EPNP532: if we remove the side connection from the 1st stage as for the semantic segmentation task, keeping the deep supervision layers applied on all output maps, we observe a slight degradation of the results.
Conclusion The improvement of state-of-the art semantic edge detection algorithms has been achieved here, both by relevant refinements of network architecture and a fine-tuning strategy during the learning step. Inspired by CASENet (resp. FPN) networks in its bottomup and top-down (resp. FPN) pathways to combine low and high features, Edge Pyramid Network already outperforms CASENet and STEAL on SBD and Cityscapes datasets, providing smoother edges without aliasing effect. The EPN architecture was carefully validated by testing different backbones and by making an ablation study. If we apply our fine-tuning strategy, alternating weightednegative and unweighted cross-entropy losses over only two training cycles, the performance of the above cited networks is increased, as thinner edges are produced.
Finally, combining EPN architecture and fine-tuning strategy provides cumulative improvements, as more accurate and crisper edges are obtained, as attested by the highest MF-scores. This evidence is also reinforced by the fact that decreasing the tolerance allowed for matching the contours leads to a larger performance gap between our implementation and STEAL network. Our training strategy also makes a significant improvement on another structure as CASENet. Additionally, EPN architecture achieves competitive performance against stateof-the-art algorithms on the BSDS500 dataset. It demonstrates the remarkable versatility of our approach, ranging from categoryaware to category-agnostic edge detection networks.
Sjögren's disease is a complex autoimmune disease with twelve established susceptibility loci. This genome-wide association study (GWAS) identifies ten novel genome-wide significant (GWS) regions in Sjögren's cases of European ancestry: CD247, NAB1, PTTG1-MIR146A, PRDM1-ATG5, TNFAIP3, XKR6, MAPT-CRHR1, RPTOR-CHMP6-BAIAP6, TYK2, SYNGR1. Polygenic risk scores yield predictability (AUROC = 0. 71) and relative risk of 12. 08. Interrogation of bioinformatics databases refine the associations, define local regulatory networks of GWS SNPs from the 95% credible set, and expand the implicated gene list to >40. Many GWS SNPs are eQTLs for genes within topologically associated domains in immune cells and/or eQTLs in the main target tissue, salivary glands.
All genotyped Sjögren's individuals (Sjögren-All) and population control individuals were randomly divided into a training dataset (2/3 individuals; 2166 cases, 11,638 controls) or testing dataset (1/3 individuals; 1076 cases, 5826 controls) (Supplementary Fig. 2A, B ). Sjögren-All individuals with reported Ro autoantibody status were further subset into Ro autoantibody positive (Sjögren-Ro + ) or negative (Sjögren-Ro -) datasets, then the Sjögren-Ro + subset was similarly divided into training (1100 cases, 11,544 controls) and testing (618 cases, 5896 controls) datasets for PRS calculations. Unfortunately, the Sjögren-Ro -subset was too small for accurate PRS calculations. Sjögren-All and Sjögren-Ro + training sets most accurately predicted case-control status at a P value threshold P T = 0.
0 [http://www. enhanceratlas. org/] to evaluate the enrichment of GWS Sjögren-SNPs in the enhancers of different immune cell subsets  35. Significant enrichment of GWS Sjögren-SNPs was observed in the enhancers of T helper, T regulatory, dendritic, and B cells, with T helper 1 cells showing the most significant enrichment (P = 1. 83 × 10 -5 ), followed by GM18486 and GM12878 (P = 1. 58 × 10 -4 and P = 4. 48 × 10 -4 , respectively) (Fig. 4b ). Last, Regulatory Element Locus Intersection (RELI) analysis [https://github. com/WeirauchLab/RELI] was used to identify positional overlap between Sjögren's risk loci and transcription factor binding sites, as well as the potential role for Epstein-Barr virus (EBV) in Sjögren's, as had been reported in other related diseases  36, 37. The transcription factors, MYCassociated factor X (MAX), basic leucine zipper ATF-like  ) Figure  1 , Table  1  7 Established Loci (P <5x10 -8 ) Figure  1  Meta-Analysis (METAL)*: 7 Novel Loci (P <5x10  -8  ) Figure  1 , Table  1  4 Established Loci (P <5x10 -8 ) Figure  1  76 Suggestive Loci (P <5.
5x10 -5 ) Supplementary Data 2 GWAS: Step 1: Characterization of risk loci i. Conditional analysis: Logistic regression was performed adjusting for the genotype covariates and the most significant variant to define independent risk loci using PLINK. ii. Bayesian analysis: Posterior probability estimates were calculated for each SNP adjusting for genotype covariates in each risk loci using Trinculo. iii. Haplotype analysis: LD and probable haplotype blocks were determined using solid spine of LD algorithms in HAPLOVIEW. Step 2: Annotation of candidate functional variants in risk loci i. RegulomeDB score: RegulomeDB database was used to identify scores for SNPs in 95% credible sets for each loci. ii. HaploReg: HaploReg database was used to query the enrichment of regulatory chromatin states from DNAse and histone ChIP-Seq experiments in primary immune cells and EBV transformed B lymphocytes, proteins bound in ChiP-Seq experiments, reported eQTLs and regulatory motifs altered for each SNP in 95% credible sets for each loci.
iii. IMPACT score: IMPACT was used to identify the SNPs with the highest transcription factor binding activity in the 95% credible set. Step 3: Cis-eQTL mapping and Promoter capture Hi-C analyses i. eQTL mapping: FUMA GWAS was used to map SNPs to genes up to 1Mb apart that show a significant (p<0. 05) cis-eQTL association in different databases: eQTL catalogue, single-cell RNA sequencing eQTLs, Database of Immune Cell eQTLs and Epigenomics (DICE), and minor salivary gland (GTEx v08). Another online tool, QTLbase was also used to map SNPs in 95% credible sets to cis-eQTLs. ii. Promoter capture Hi-C analysis: Pre-processed capture Hi-C data from 3D Genome Browser for each candidate functional variants from 14 immune cells and EBV transformed B lymphocytes were used to find promoter and enhancer sites located 5kb either side of each SNP.
The start and end coordinates of each consensus loop were then queried in USCS genome browser to identify overlapping and/or nearby genes. iii. EcholocatoR: R package for automated fine-mapping, annotation and plotting of SNPs to identify most probable causal SNPs. The post-imputation merged dataset (PI1) containing DS1-6 was used to perform the SNP-Sjögren's single marker trait analysis (orange), the polygenic risk score (PRS) analysis (yellow), the genetic correlation analyses (blue), and the epigenetic enrichment analyses (blue). Meta-analysis was performed using the genotyped PI1 merged dataset and DS7 ImmunoChip dataset (green) merged using the DS7 genotyping platform. See Supplementary Data 1 for detailed information for each dataset. b Statistical and bioinformatic analysis workflow applied to each novel risk locus to identify and predict functionality of likely functional SNPs.
transcription factor (BATF), and MYC proto-oncogene, BHLH transcription factor (MYC), exhibited the highest number of intersections with Sjögren's risk loci (Fig. 4c ). Further, seven of the ten transcription factors yielding the highest number of intersection counts with Sjögren's risk loci were previously reported to be associated with EBV  38. Moreover, the EBVencoded protein, Epstein-Barr Virus nuclear protein 2 (EBNA2), occupied 8 of the 17 GWS Sjögren's risk loci. Refinement of the novel Sjögren's genetic associations. Publicly available Hi-C data from the EBV B cell line, GM12878, and eQTL data from the DICE database in FUMA [http://fuma. ctglab. nl/] 39,40 were used to map TADs and eQTLs reported for genes positioned in each Sjögren's risk locus. A majority of the Sjögren's risk loci exhibited strong linkage disequilibrium with many SNPs and enrichment of reported eQTLs and TADs in GM12878 (Fig.
Logistic regression and Bayesian analyses were used to refine the association signals from each of the 10 novel GWS regions, then posterior probability analyses were used to identify the 95% credible set of likely functional SNPs present in each region (Fig. 2b-k ; Supplementary Figs. 3 4 5 6 7 8 9 10 11 12 ; Supplementary Data 5-34). SNPs from each credible set were independently interrogated for evidence of functionality using a variety of bioinformatics databases, including RegulomeDB [https://www. regulomedb. org/regulome-search/]  41, 42 nd HaploReg [https://pubs. broadinstitute. org/mammals/haploreg/ haploreg. php]  43 , and annotated accordingly. SNPs with the strong evidence of known and predicted regulatory functions, including transcription factor binding sites, reported promotor and enhancer activities, DNAse hypersensitivity, and eQTLs were selected for further functional dissection (Fig.
1b ). Coalescence of epigenetic marks with eQTLs mapped to TADs identified potential functional SNPs from each locus that most likely  Fig. 2  Summary of the SNP-Sjögren associations in a European population. a Manhattan plot shows the summary data from the meta-analysis of the 7. 3 × 10 5 SNPs shared between the GWAS and ImmunoChip dataset (Supplementary Data 1) after imputation. The -log 10 (P) for each variant is plotted according to chromosome and base pair position. A total of seven novel loci (indicated in red) exceeded genome-wide significance (GWS) of P GWAS < 5 × 10 -8 (red dashed line). Three additional novel loci (indicated in royal blue) exceed GWS after meta-analysis (P META < 5 × 10 -8 ). Several previously established loci were replicated (indicated in light blue). The suggestive GWAS and meta-analysis threshold (P Suggestive < 5 × 10 -5 ) is indicated by the blue dashed line.
b-h Logistical regression analysis was performed on GWAS dataset PI1 after imputation, identifying the top SNP associations (e. g. , index SNPs) of the novel GWS regions of association: NAB1 (b), PTTG1-MIR146A (c), XKR6 (d), MAPT-CRHR1 (e), RPTOR-CHMP6-BAIAP6 (f), TYK2 (g), SYNGR1 (h). i-k Logistical regression analysis was performed after meta-analysis of the GWAS and ImmunoChip data, identifying the top SNP associations (e. g. , index SNPs) of the novel GWS regions of association: CD247 (i), PRDM1-ATG5 (j), TNFAIP3 (k). Table  1  Summary results of the GWAS and meta-analysis of Sjögren's of European Ancestry. 7. 17 × 10   Functional analysis of Sjögren-SNPs in intronic enhancers. Four variants at the first intron of CD247 molecule gene (CD247; encodes the T cell receptor ζ-chain) showed regulatory potential: rs7523907 (index SNP), rs2949661, rs1214595, and rs1723018 (Fig.
5b ; Supplementary Fig. 4H, J ; Supplementary Data 10). Finally, EcholocatoR  44  , an R package for automated statistical and functional fine-mapping [https://github. com/ RajLabMSSM/echolocatoR], was used to further evaluate the expanded regulatory network of the CD247 region and identify additional SNPs with potential functional significance. Independent EcholocatoR analyses, specifying each of the SNPs identified above as an index SNP, supported the identification of rs7523907 as a likely functional SNP and further identified rs2462552 and rs10800313 as additional SNPs of interest (Supplementary Fig. 4K ). The association region peaking at rs4841466 positioned in the intron of XK-related 6 (XKR6) is an independent association signal from the previously reported FAM167A-BLK region (Fig.
Interestingly, removal of SNPs from the HLA region significantly reduced the predictability of the Sjögren's PRS calculations in both the Sjögren-All and Sjögren-Ro + datasets, thus indicating that the SNPs carried on the HLA are strong genetic risk factors for Sjögren's and the anti-Ro autoantibody positive sub-phenotype. Due to the limited number of Sjögren-Ro -individuals available in this study, we could not assess the impact of HLA on the genetic risk of the anti-Ro autoantibody negative phenotype. However, the residual effect observed in the Sjögren-All PRS predictions after HLA removal suggests that the HLA association may have a lower impact on the genetic risk of the Sjögren-Ro -subphenotype  62. Future studies focusing on the genetic risk of Sjögren-Ro -patients are needed to assess these subphenotypic genetic differences.
5b ; Supplementary Data 25), leading us to hypothesize that the risk allele of rs533733 may also contribute to Sjögren's pathogenesis by modulating autophagy in the salivary gland. Interpretations of our bioinformatic analyses must consider the following limitations: (1) leveraged eQTL databases do not always report directionality, which limits the ability to interpret whether an implicated Sjögren-SNP increases or decreases gene expression; (2) the promoter-capture Hi-C data used for TAD mapping were obtained from quiescent cell types. Cellular microenvironments, including the presence/absence of inflammatory cytokines, can modulate gene expression in several ways, including altered transcription factor binding, regulatory element activity, and TADs between regulatory elements and target gene promoters  [28] [29] [30].