dataesr/scientific-paragraphs-categorization huggingface dataset:
Image processing Our home-made Hi-M microscope produced z-stacks in DCIMG format, which were converted to TIFF using proprietary software from Hamamatsu. TIFF images were then deconvolved using Huygens Professional version 20. 04 (Scientific Volume Imaging, the Netherlands, https://svi. nl/). Further analysis steps were performed using a homemade analysis software that implemented the steps described previously  49. Briefly, images were first z-projected using either sum (DAPI channel) or maximum intensity projection (barcodes, fiducials). Image-based cross-correlation was used to align the fiducial channels. These corrections were then used to align DAPI and barcode images. Next, the positions of the XY centers of barcodes were detected with sub-pixel resolution using local maximum fitting functions from the ASTROPY package  95.
ShRec3D Structures Three-dimensional topological representations were obtained from Hi-M pairwise distance maps using our own Python implementation of the approaches described by Lesne et al. and Morlot et al. for ShRec3D  63, 96. Starting from the single-cell pairwise distance matrix, an ensemble pairwise distance matrix was calculated using the first maximum of the kernel density estimation. These pairwise distances were converted into 3D coordinates for each barcode using nonclassical metric multidimensional scaling. When necessary, structures were mirrored and a ball-and-stick representation was rendered with PyMOL (The PyMOL Molecular Graphics System, Version 2. 3 Schrödinger, LLC. ).
Genome-wide analysis of Zld-mediated interactions First, we extracted lists of Zld peaks genome-wide. For Zld peaks used in Figure  4k -o, Extended Data Figure  7i -k: Datasets from Harrison et al. 55  (GSM763062) were used to extract the autosomal regions bound by Zelda at 3 hpf using their corresponding ChIP-Seq intensity  55. For Zld peaks used in Extended Data Figure  7l : a list of putative enhancers of pre-MBT genes (N = 62) was obtained by selecting the Zld peaks nearest to the TSS of pre-MBT genes  97  (Supplementary Table  8 ). BED coordinates were remapped from dm3 (BDGP R5) to dm6 (BDGP R6) using FlyBase's sequence coordinates converter (FB2020_05, released Oct 14, 2020). Second, we characterized interactions between Zld-bound regions using the 5-kb resolution Hi-C dataset from Hug.
Acknowledgements We are grateful to  N. Benabdallah ,  G. Cavalli ,  T. Forne ,  T. Robert ,  J. Bonnet  and members of the  Lagha and Nollmann laboratories  for their critical reading of the manuscript. We thank  A. Makrini  and  D. Cattoni  for help with bioinformatic analysis. This project was funded by the  European Union 's  Horizon 2020 Research and Innovation Program  (Grant  ID 724429 ) (M. N. ). We acknowledge the  Bettencourt-Schueller Foundation  for their prize 'Coup d'élan pour la recherche Française', the  France-BioImaging  infrastructure supported by the  French National Research Agency  (grant ID  ANR-10-INBS-04 , '' Investments for the Future ''), and the  Drosophila facility  ( BioCampus Montpellier ,  CNRS ,  INSERM ,  Université de Montpellier , Montpellier, France). M.
Data availability Oligopaint public database (http://genetics. med. harvard. ) was used to select oligopaints. Publicly available datasets used in this study (GSE86966, GSE25180,E-MTAB-4918, GSM763062, GSE58935, GSE16245, GSE68983, GSE68654, E-MTAB-1673, GSE62904, GSE65441) are detailed in Supplementary Table  9. Uncropped version of agarose gel in Extended Data Figure  7a  is provided in Source Data Extended Data Figure  7. Data for matrices in Figures  1 2 3 4 and in Extended Data Figures are publicly available at https://github. com/NollmannLab/Espinola-Goetz-2021.
We instantiate our framework with two new formulations of GAT and GCN models, namely ML-GCN and ML-GAT , specifically devised for general, attributed multilayer networks. The proposed approaches are evaluated on an entity classification task on nine widely used real-world network datasets coming from different domains and with different structural characteristics. Results show that both our proposed ML-GAT and ML-GCN methods provide effective and efficient solutions to the problem of entity classification in multilayer attributed networks, being faster to learn and offering better accuracy than the competitors. Furthermore, results show how our methods are able to take advantage of the presence of real attributes for the entities, in addition to arbitrary inter-layer connections between the nodes in the various layers.
4. Our designed GNN components in the proposed framework are able to incorporate external information associated with the multilayer network, in the form of attributes that can be available at entity-level or at node-level for each particular layer of the input network. 5. Experimental evidence from widely used multiplex networks and from a real-world attributed multilayer network dataset has shown that both the GAT and the GCN instances of our framework represent effective and efficient solutions to the problem of entity classification in multilayer attributed networks. Our methods were also compared with two recently proposed methods for multirelational networks based on a GAT model, named  GrAMME-SG and GrAMME-Fusion (Shanthamallu et al. 2020) : our methods are able to achieve accuracy as good as or better than the competitors (up to 13% of accuracy improvement), while outperforming them in terms of efficiency (with a training time which is two orders of magnitude lower than GrAMME methods in most cases).
In order to show the outcomes of the representation learning process, a two-dimensional projection of the embeddings produced by the proposed approaches is reported in Fig. 3  (ML-GAT ) and Fig. 4  (ML-GCN). The representation is obtained via the t-distributed stochastic neighbor embedding (t-SNE) method, which is a widely used nonlinear dimensionality reduction technique for embedding high-dimensional data for visualization in a low-dimensional space (van der Maaten and Hinton 2008). More specifically, (  16 ) the plots show the embeddings produced by t-SNE on Z for the training entities ( 25% of total entities), i. e. , the entity representation obtained through the cross-layer aggregation defined in Eq. (  15 ). The embeddings refer to the Koumbia-2 network including its real-world node-attributes (cf.
Data We considered 9 network datasets, from which we derived a total of 19 networks for evaluation (plus their monoplex flattened versions). These datasets come from different domains and present very different structural characteristics. Moreover, all datasets are publicly available and most of them have been previously exploited as benchmarks for a variety of network analysis tasks, including node classification and link prediction. This is a major criterion for our choice, since it allows comparison of our results with the ones reported in previous literature. Eight of such datasets are originally provided as multiplex networks, i. e. , networks in which inter-layer connections are coupling edges only, connecting a node and its counterparts in other layers. These networks also do not provide attributes associated to the entities.
Therefore, in order to stress the ability of our framework of dealing with generic multilayer attributed networks, we introduced the Koumbia network dataset  (Interdonato et al. 2020 ), which comes with unconstrained inter-layer connections and real-world properties associated to the entities. In the following we briefly describe our evaluation network datasets. Balance  (Siegler 1976 ) models psychological experimental results on a set of individuals. According to  Shanthamallu et al. (2020) , four attributes characterize the subjects (left weight, the left distance, the right weight, and the right distance). The classes correspond to the balance scale of the subjects (tip to the right, tip to the left, or being balanced). CKM-Social  (Coleman et al. 1957 ) contains social information from physicians (entities) in four towns in Illinois, Peoria, Bloomington, Quincy and Galesburg.
It consists of 3 directed layers generated from different sociometric matrices, where the cities are used to label the entities. Congress  (Schlimmer 1987 ) models the results of bills obtained from the 1984 United States Congressional Voting Records Database. The network has 16 layers corresponding to votes, where for each layer two congressmen are linked if they voted the same. Each congressman is labeled as either democrat or republican. DKPol  (Magnani et al. 2021 ) (Dansk Politik) is a network with three types of online relations between Danish Members of the Parliament on Twitter. It comes with a ground truth corresponding to affiliations to 10 political parties, that we used as labels. Leskovec-Ng (Chen and III 2016) is a 4-layer temporal collaboration network, which contains coauthors of Prof.
Jure Leskovec or Prof. Andrew Ng over 20 years, partitioned into 4 different 5-year intervals. Entities are researchers, and on each layer there is an edge between two researchers if they co-authored at least one paper in the 5-year interval. Each researcher is labeled as Leskovec's collaborator or Ng's collaborator. Starwars 2 is comprised of 6 layers, each corresponding to interactions between Starwars characters in the first 6 episodes of the saga. We manually labeled each character as male, female or droid and used this information as entity labels. Note that the resulting class distribution is very unbalanced as there are 76 males, 12 females and 4 droids. Terrorist  (Everton 2012 ) models interactions between 79 terrorists drawn from the Noordin's Network dataset. Similarly to  Liu et al.
In this dataset, entities represent segments of the satellite image, and classes correspond either to crop (i. e. , segments containing pixels related to cultivated area) or no-crop (i. e. , segments containing pixels related to uncultivated areas, such as forests) segments. The network is originally associated with inter-layer edges and realworld attributes for the entities, corresponding to the segment statistics of the radiometric bands of the satellite images. We created the input feature matrix by concatenating the average values of ten different radiometric bands for 21 timestamps, obtaining a feature vector of size 210 for each entity. Fig. 5a  displays the feature distribution obtained by linearizing the input feature matrix. Note that the geo2net framework 4 presented in  Interdonato et al.
(2020)  is designed to build a multilayer network with an arbitrary number of layers, which model the association of nodes to an arbitrary number of functional classes (e. g. , temporal radiometric profiles) by producing fuzzy layer memberships using 2 https:// github. com/ eveli nag/ StarW ars-social-netwo rk/ tree/ master/ netwo rks. 3 https:// senti nel. esa. int/ web/ senti nel/ missi ons/ senti nel-2. 4 https:// gitlab. irstea. fr/ raffa ele. gaeta no/ geo2n et. the fuzzy c-means algorithm  (Ross 2009). In our case study, we exploit this functionality in order to take into account versions of the network with a varying number of layers  (i. e. , 2, 5, 10, 15, 20). We denote each of these networks as Koumbia-l (e. g. , Koumbia-5 will denote the version with 5 layers). Figure  5  shows Koumbia-2 (b) and Koumbia-5 (c) graphs, whereas Table  4  reports detailed information about inter-layer edges.
Since the competitors are specifically designed for multiplex networks, for a fair comparison with those methods we will also use a multiplex version of this dataset (i. e. , obtained discarding inter-layer connections other than coupling ones), named Koumbia-l-mpx. Table  2  summarizes the structural properties of the all network datasets, plus the multiplex versions of Koumbia. For the latter, inter-layer edge information is reported in Table  4. Note that the graph homophily in Table  2  corresponds to the fraction of edges in a graph connecting nodes with the same class label  (Zhu et al. 2020 ), formally |{(v i , v j ) : (v i , v j ) ∈ E ∧ y i = y j }|/|E|, with v i , v j ∈ V and y i , y j class labels of v i , v j , respec- tively. This statistic was calculated excluding the inter-layer edges, which otherwise would lead to biased homophily scores.
Results We present the results of the experimental evaluation of the proposed framework, which are organized into two evaluation stages: 1. In "Evaluation with competing methods" section, we compare the proposed methods and competitors (i. e. , GrAMME-SG, GrAMME-Fusion, GAT and GCN), on the multiplex networks introduced in "Data" section, in terms of prediction performance, impact due to the setting of the training set size and early-stopping technique, and of the initial input node-features. We also analyze the training execution times of the methods, and we discuss aspects of their computational complexity. 2. In "Evaluation on real-world node-attributes and arbitrary inter-layer edges: the Koumbia multilayer network testbed" section, we conduct a thorough analysis of the Koumbia dataset, focusing on the impact of using real node-features.
Evaluation with competing methods Table  5  reports the average accuracy scores achieved by the proposed ML-GAT and ML-GCN approaches, and the competing methods (GrAMME-SG, GrAMME-Fusion, GAT and GCN). For each network and method, the average accuracy was computed over 10 independent runs, where each run corresponded to a different train-test split, with 25% of training entities, and the input features of the entities were randomly initialized with a normal distribution for all the networks. At a first glance, our proposed methods are able to achieve very high, or even nearly optimal accuracy, on most networks. This also mostly holds for GrAMME-Fusion and, to a less extent, for GrAMME-SG. Moreover, the results obtained by the baselines on the corresponding monoplex versions of the network datasets reveal some situations in which the flattening process is actually beneficial for the entity classification task: particularly, as it can be observed in Fig.
Furthermore, according to  Zhu et al. (2020)  and  Qian et al. (2021) , all models have relatively low performance on the network with the lowest homophily score, i. e. , Terrorist-status (0. 469). Likewise, all models have good performance on networks with strong homophily, such as Leskovec-Ng (0. 994) and CKM-Social (1. 0). On the other hand, note how multilayer models can still perform well on networks with lower homophily. Finally, a major remark that stands out from Table  5  concerns the results obtained on Koumbia networks, for increasing number of layers. Note that Koumbia is the network dataset including the highest number of entities (  2246 ), and that the Koumbia-2 network has no edge overlap between the two layers (i. e. , the node set on the two layers is completely disjoint).
Conclusions: discussion and future work We proposed a GNN framework for representation learning and semi-supervised classification in multilayer networks with attributed entities, and with arbitrary number of layers and intra-layer and inter-layer connections between nodes. We instantiated our framework through two new formulations of GAT and GCN models, specifically designed for the above general, attributed multilayer networks. We evaluated our ML-GAT and ML-GCN methods on real-world network datasets coming from different domains and with different structural characteristics. Our results showed that ML-GAT and ML-GCN models are significantly faster learners than the competitors, and they outperform in accuracy both the competitors and baseline methods especially on arbitrary multilayer networks, with large number of entities and layers.
Accuracy (mean and standard deviation over 10 runs) obtained by initializing the entity features with mixed distributionsBold values refer to the best results on each network GrAMME-Fusion and GCN have worse performance, while on the latter, GrAMME methods even improve w. r. t. Table5. These can be explained as both network datasets have highly unbalanced distribution of class labels, therefore for the least covered class (i. e. , 'droid' for Starwars and 'dead' for Terrorist-status) the number of selected training instances does not significantly change as the percentage of training set size decreases from 25 to 5%. More interestingly, on the largest networks other than Koumbia, i. e. , Congress and Balance, ML-GCN and ML-GAT outperform the other competing models.
Domain Name System The DNS translates a name of the form www. example. com into an Internet destination address to use in the packet header to forward the packet. Structurally, the DNS is a hierarchical, distributed database with built-in redundancy. Assignment of responsibility for domains occurs through a process of delegation, in which an entity at a higher level in the hierarchy assigns responsibility for a subset of names to another party. The hierarchy starts at the root of the DNS, which delegates top-level domains (TLDs) such as. com,. net,. nl, and so on. These TLDs in turn delegate to second-level domains, which may further delegate parts of the name space. Administration of these delegations can be a complex task involving many stakeholders, most obviously registries, registrars, and registrants.
A DNS registry administers a TLD. The registrar provides an interface between registrant and registry, managing purchase, renewal, changes, expiration, and billing. The registrant is the customer that registers a domain. Other players, for example, Cloudflare, may buy and host domain resources on behalf of registrants. The organization with overall responsibility for the stewardship of the DNS namespace is ICANN. 14. Rosen. 15. For a survey of the history of proposed schemes to secure BGP, see Testart, "Reviewing a Historical Internet Vulnerability. " Today, harms that leverage the DNS protocols and supply chain represent some of the most pernicious security threats on the Internet. Malicious actors can subvert existing names or register their own names by penetrating databases operated by either registries or registrars, and then use those names for malicious purposes.
By penetrating a registry or registrar database, one can add invalid registrant information, or change the binding from a name to an address. Lack of oversight of the competitive for-profit DNS supply chain contributes to these security risks. But the complexity of the DNS also leads to misconfiguration of the name resolution mechanisms by owners of domain names, which can allow malicious actors to take control of them. Finally, and most challenging, is the registration of domain names intended for malicious use such as phishing or malware delivery. Every month, the ICANN reports the number of active domain names associated with abusive practices. 16 Since the beginning of 2020, the numbers range from a low of 572K in July to a high of 926K in October. Some registrars support operational practices that seem tailored to the needs of malicious actors, such as automatic registration of bulk, meaningless domain names, the creation on demand of "look-alike" or "impersonation" names, or lax attention to capturing the identity of the registrant.
The Role of Measurement In tactical, real-time defense, defenders gather security-related data on what the bad guys are doing at a given moment. Maintainers of blocklists try to infer which address and naming resources attackers are using, on an ongoing basis. This data is evanescent. Interdiction and forensics may be useful to respond to ongoing attacks, but they do not shift the playing field toward the defenders. For example, defenders who attempt to deal with the registration of domain names for malicious purposes are locked in an endless battle with the malicious actors, who adapt to interdictions as fast as they appear. So long as the defenders only try to find the bad guys and chase them from where they currently are to some new place, the generality of the Internet works against the defenders.
Engagement with Stakeholders This approach targets turning technical knowledge derived from understanding system characteristics and ongoing data analysis into open, actionable knowledge that is relevant and meaningful to various actors in the ecosystem, including those responsible for protecting infrastructure. The design of proposed operational practices must rely on a pragmatic recognition of what various actors are willing and able to undertake. Developing an understanding of incentives, costs, and externalities is as important as developing an accurate model of system operation. This means a large component of such an effort must be transferring knowledge generated by this effort to policy development and cybercrime communities. Examples from the DNS abuse community include the Anti-Phishing Working Group (APWG), the Mail, Messaging, Mobile Anti-Abuse Working Group (M3AAWG), and other operational and technical forums.
Measurement to Reduce Abuse of Internet Routing System (BGP) We have recently seen growing acceptance of a step toward better BGP security. The RIRs, which maintain databases of address block ownership, and the IP standards community (IETF) have developed a protocol for voluntary use of Route Origin Authorization (ROA), a mechanism to establish definitive authority to originate a specified prefix into the global routing system. Uptake of this technology is low, although growing. Globally, about 19% of the Internet address space is protected by ROAs as of January 2021. 26 The MANRS code of conduct (see section "Measurement to Reduce Abuse of Internet CA System") includes a requirement that member ISPs will check the BGP origin announcements of their customers to make sure that the AS and prefix are valid.
But there is growing evidence that security and consumer protection concerns have been a casualty of the multistakeholder model, where low-cost operational practices take priority over secure ones. Before we investigate the development of DNS operational practices, we must evaluate how to define, discover, quantify, and continuously monitor aspects and trends of DNS abuse, in order to find patterns of abusive behavior that can motivate operational practices. An equally important goal is to develop a model that illuminates the incentives of the various actors in the ecosystem, which requires understanding the money flows, contributions of the bad actors to the flows, and degrees of freedom for all parties. This is an ambitious, long-term goal. The starting point is to develop data collection infrastructure that enables mapping of trust dependencies, relationships among domains and name server infrastructure, and unintended attack surfaces that result from current operational practices.
This effort is relevant to two classes of threat mentioned earlier: domains registered for legitimate use but exploited by miscreants for illegitimate purposes; and those explicitly registered for malicious use. Researchers have already explored the use of existing data sets (lists of domain names called zone files for different parts of the name hierarchy) to study operational practices reflected in those zone files, including anomalous patterns in the DNS that reflect suspicious or risky registrar or registrant behavior. Such patterns include orphan DNS name servers, bulk registrations, 29 delayed registration of domains, and new, changed, or deleted domains/name servers, and their implications for resilience and large-scale vulnerabilities, that is, potential attack surfaces that encompass many domains.