dataesr/scientific-paragraphs-categorization huggingface dataset:
SpykeTorch's functional module provides several functions for different kinds of lateral inhibitions. sf. feature inhibition is useful for complete inhibition of the selected feature maps. This function comes in handy to apply dropout to a layer. sf. pointwise inhibition employs competition among features. In other words, at each location, only the neuron corresponding to the most salient feature will be allowed to emit a spike (the earliest spike with the highest potential). Lateral inhibition is also helpful to be applied on input intensities before conversion to spike-waves. This will decrease the redundancy in each region of the input. To apply this kind of inhibition, sf. intensity lateral inhibition is provided. It takes intensities and a lateral inhibition kernel by which it decreases the surrounding intensities (thus increases the latency of the corresponding spike) of each salient point.
utils Module utils module provides several utilities to ease the implementation of ideas with SpykeTorch. For example, utils. generate inhibition kernel generates an inhibition kernel based on a series of inhibition factors in a form that can be properly used by sf. intensity lateral inhibition. There exist several transformation utilities that are suitable for filtering inputs and converting them to spike-waves. Current utilities are mostly designed for vision purposes. utils. LateralIntencityInhibition objects do the sf. intensity lateral inhibition as a transform object. utils. FilterKernel is a base class to define filter kernel generators. SpykeTorch has already provided utils. DoGKernel and utils. GaborKernel in order to generate DoG and Gabor filter kernels, respectively. Objects of utils.
FilterKernel can be packed into a multi-channel filter kernel by utils. Filter objects and applied to the inputs. The most important utility provided by utils is utils. Intensity2Latency. Objects of utils. Intensity2L are used as transforms in PyTorch's datasets to transform intensities into latencies, i. e. spike-wave tensors. Usually, utils. Intensity2Latency is the final transform applied to inputs. Since the application of a series of transformations and the conversion to spike-waves can be timeconsuming, SpykeTorch provides a wrapper class, called utils. CacheDataset, which is inherited from PyTorch's dataset class. Objects of utils. CacheDataset take a dataset as their input and cache the data after applying all of the transformations. They can cache either on primary memory or secondary storage.
visualization Module The ability to visualize deep networks is of great importance since it gives a better understanding of how the network's components are working. However, visualization is not a straightforward procedure and depends highly on the target problem and the input data. Due to the fact that SpykeTorch is developed mainly for vision tasks, its visualization module contains useful functions to reconstruct the learned visual features. The user should note that these functions are not perfect and cannot be used in every situation. In general, we recommend the user to define his/her own visualization functions to get the most out of the data.
Tutorial In this section, we show how to design, build, train, and test a SNN with SpykeTorch in a tutorial format. The network in this tutorial is adopted from the deep convolutional SNN proposed by  [10]  which recognizes handwritten digits (tested on MNIST dataset). This network has a deep structure and uses both STDP and Reward-Modulated STDP (R-STDP), which makes it a suitable choice for a complete tutorial. In order to make the tutorial as simple as possible, we present code snippets with reduced contents. For the complete source code, please check SpykeTorch's GitHub 1  web page. The best way to design a SNN is to define a class inherited from torch. nn. Module. The network proposed in  [10] , has an input layer which applies DoG filters to the input image and converts it to spike-wave.
Step 2. Input Layer and Transformations SNNs work with spikes, thus, we need to transform images into spike-waves before feeding them into the network. PyTorch's datasets accept a function as a transformation which is called automatically on each input sample. We make use of this feature together with the provided transform functions and objects by PyTorch and SpykeTorch. According to the network proposed in  [10] , each image is convolved by six DoG filters, locally normalized, and transformed into spike-wave. As appeared in Listing 6, a new class is defined to handle the required transformations. Each InputTransform object converts the input image into a tensor (line 9), adds an extra dimension for time (line 10), applies provided filters (line 11), applies local normalization (line 12), and generates spike-wave tensor (line 13).
Step 3. Data Preparation Due to the PyTorch and SpykeTorch compatibility, all of the PyTorch's dataset utilities work here. As illustrated in Listing 7, we use torchvision. datasets. MNIST to load MNIST dataset with our previously defined transform. Moreover, we use SpykeTorch's dataset wrapper, utils. CacheDataset to enable caching the transformed data after its first presentation. When the dataset gets ready, we use PyTorch's DataLoader to manage data loading.
Source Code Through this tutorial, we omitted many parts of the actual implementation such as adaptive learning rates, multiplication of learning rates, and saving/loading the best state of the network, for the sake of simplicity and clearance. The complete reimplementation is available on SpykeTorch's GitHub web page. We have also provided scripts for other works  [26, 9]  that achieve almost the same results as the main implementations. However, due to technical and computational differences between SpykeTorch and the original versions, tiny differences in performance are expected. A comparison between SpykeTorch and one of the previous implementations is provided in the next section.
We agree that SpykeTorch has hard limitations on type of SNNs, however, there is always a trade-off between computational efficiency and generalization. Apart from the increase of computational efficiency, this particular type of SNNs are bio-realistic, energy-efficient, and hardware-friendly that are getting more and more popular recently. We provided a tutorial on how to build, train, and evaluate a DCSNN for digit recognition using SpykeTorch. However, the resources are not limited to this paper and additional scripts and documentations can be found on SpykeTorch's GitHub page. We reimplemented various works  [26, 9, 10]  by SpykeTorch and reproduced their results with negligible difference. Although the current version of SpykeTorch is functional and provides the main modules and utilities for DCSNNs (with at most one spike per neuron), we will not stop here and our plan is to extend and improve it gradually.
Acknowledgments This research was partially supported by the  Iranian Cognitive Sciences and Technologies Council  (Grant no. 5898 ) and by the  French Agence Nationale de la Recherche  (grant:  Beating Roger Federer   ANR-16-CE28-0017-01 ). The authors would like to thank  Dr. Jean-Pierre Jaffrézou  for proofreading this manuscript and  NVIDIA GPU Grant Program  for supporting computations by providing a high-tech GPU.
1(c) ]. For the WAXS data, the integration sectors from the low-q SAXS data were used. The azimuthal integration and other treatments were done with the aid of the pyFAI  (Ashiotis et al. , 2015)  and FabIO  (Knudsen et al. , 2013)  Python packages. The integrated SAXS intensities from different detector distances were merged and rebinned using the SAXSutilities software (http:// www. sztucki. de/SAXSutilities), whereas the same procedures for the SANS intensities and the rebinning of SAXS data from drying samples were done with Python scripts.
, 1996; Nishiyama et al. , 2014) , whereas the Gaussian function centred at q = 0 A ˚À1 (term with constant B) was added to approximate scattering from larger pores or other unspecified structural features and to reach a good fit for all data except the SAXS data from wet softwoods. The relative contributions of the different terms in a fit to a real wood sample are illustrated in Fig. 2(b). The model fitting of equatorial SAXS and SANS intensities of wet and dry samples, after subtracting the isotropic component as described in Section 2. 4, was done in the SasView 4. 1. 0 software  (Doucet et al. , 2017)  using a plugin written in Python 2, whereas the SAXS data measured during drying were fitted using customized Python scripts. An error of 2% for the intensity data points was used to weight the fits in all cases.
Determination of crystallite dimensions from WAXS data Crystal size was calculated from equatorial WAXS data from the dry wood samples. The intensity between q values of 0. 5 and 2. 25 A ˚À1 was fitted with three Gaussian functions corresponding to the reflections hkl = 110, 110 and 200 of cellulose I  (Nishiyama et al. , 2002) , and one broad Gaussian function centred around q = 1. 4 A ˚À1 plus a linear function corresponding to the amorphous background and other reflections. The crystal size L hkl in each of the three directions was calculated using the Scherrer equation: L hkl ¼ 2=Áq hkl ; ð2Þ where Áq hkl is the integral breadth of the diffraction peak. The values of L hkl are often regarded as lower limits for the true crystal size, because other factors such as lattice distortions also contribute to peak broadening  (Fink et al.
Moreover, the equatorial SAXS intensities exhibit a similar weakening of the shoulder feature and strengthening of the power-law scattering at low q to that seen with the SANS data (Section 3. 1. 1), except that the onset of the power-law scattering is shifted even more clearly from about 0. 03 A ˚À1 (20 nm) to about 0. 06 A ˚À1 (10 nm) as a result of drying. The strong increase in the power-law scattering with almost unchanged exponent ' 4 has previously been explained by the increased contrast between larger pores and the cell wall when water is gradually exchanged for air in the pores and voids of the wood tissue  (Jakob et al. , 1996; Leppa ¨nen et al. , 2011; Suzuki & Kamiyama, 2004). 3. 1. 3. SAXS results from drying wood samples. In order to examine the gradual changes in the SAXS data with decreasing moisture content, the wood samples were allowed to dry in the sample holder and SAXS patterns at the shorter sample-to-detector distance (39 cm) were measured at about 30 min time intervals during drying.
WAXS results WAXS data were measured in order to obtain an independent estimate for the lateral dimensions of the cellulose microfibrils in the wood samples. Representative equatorial WAXS intensities of dry wood samples with an example fit are presented in Fig. 6. The lateral crystallite dimensions obtained by peak fitting (Table  3 ) correspond to a roughly circular or elliptic cross section of the cellulose microfibril in all samples. The diameter of the microfibril cross section, averaged over the different lateral directions, is between 3. 2 and 3. 6 nm in all samples, with a slightly larger value for birch than the softwoods. However, the crystallite thickness in the direction perpendicular to the plane of the glucose units, calculated from the best-resolved 200 reflection, is close to 3.
The intrinsic differences between SANS and SAXS data obtained from the same wood sample are probably related to the different scattering length density contrasts seen by the two probes, which make them sensitive to slightly different structures depending on the chemical or isotopic composition and the local distribution of the different components in the system. In order to demonstrate the performance of the model further, drying of wood samples was monitored with in situ SAXS measurements. The fits to the data (Fig. 5 ) were able to capture behaviour in the nanoscale structure of the wood cell wall similar to what has been reported before, most importantly the gradual change in the spacing between the microfibrils with changing moisture content  (Fernandes et al. , 2011; Leppa ¨nen et al.
Results from such experiments will be reported in the near future. The model of equation (1) will be made publicly available as a plugin (WoodSAS) to the SasView fitting software at the SasView Marketplace (http://marketplace. sasview. org/). where I ? ðq; Þ / f 2 À f 2 þ f 2 Z 1 Z 2 : ð4Þ The form factor of an infinitely long cylinder in equation (  4 ) is f ðq; RÞ / A cyl J 1 ðqRÞ qR ¼ R J 1 ðqRÞ q ; ð5Þ with J 1 denoting the Bessel function of the first kind and A cyl the cross-sectional area of the cylinder. The terms with averaging in equation (  4 The paracrystal lattice factors Z 1 and Z 2 for a hexagonal lattice with lattice vectors a 1 and a 2 are obtained from Z k ðqÞ ¼ 1 À F k 2 1 À 2 F k cosðq Á a k Þ þ F k 2 ; ð9Þ where F k ¼ exp À 1 2 Áa a 2 q Á a 1 ð Þ 2 þ q Á a 2 ð Þ 2 Â Ã ( ) ; ð10Þ q Á a 1 ¼ Àaq cos À 6 ; ð11Þ q Á a 2 ¼ aq sin : A2.
Acknowledgements The ILL (proposal No. TEST-2747 ) and  ESRF  (final numbers  02-01-882  and  02-01-883 ) are thanked for providing beamtime. The SAXS and WAXS experiments were performed on the French CRG beamline D2AM at the  ESRF. The authors thank  Dr Isabelle Morfin  from  CNRS/University of Grenoble Alpes, LIPhy , for providing assistance in using the beamline, and  Dr Patrik Ahvenainen  from the  University of Helsinki, Department of Physics , for help during the measurements. Dr Yoshiharu Nishiyama  from  CERMAV-CNRS  is thanked for valuable discussions and suggestions.
Funding information The following funding is acknowledged:  Emil Aaltosen Sa  ¨a ¨tio (scholarship to  Paavo A. Penttila  ¨);  Agence Nationale de la Recherche  (grant No. ANR-11-EQPX-0010 );  US National Science Foundation  (award No. DMR-0520547 );  EU Horizon 2020  (grant No. 654000 ).
There is a massive amount of information in the literature about molecular mechanisms implicated in innate immune cells polarization in TME. However, most of the studies are focused on individual molecular components and pathways. They do not integrate the complexity of multiple crosstalks between innate immune cells and tumor cells. To create a holistic picture of the diversity and integrity of innate immune system in TME, the knowledge about molecular circuits should be gathered together and systematically represented  21. To address these challenges, a systems biology approach is needed  22. Formalization of biological knowledge in a form of comprehensive signaling maps, both at the intra-and intercellular levels, helps to integrate information from multiple research papers  23. There are numerous public databases containing signaling pathways related to innate-immune response such as KEGG  24  and REACTOME  25  , which are quiet comprehensive, but contain mostly generic mechanisms.
Results Principles of innate immune response in cancer. The molecular mechanisms regulating six major innate immune cell types found in the TME were gathered and depicted in the form of network maps. To cope with a massive body of literature on innate immune response in cancer we followed a systematic procedure of literature selection, knowledge organization, and integration of information in a visual and understandable manner (Fig. 1 ). The network maps were constructed as two-dimensional maps to facilitate the graphical representation of molecular mechanisms that drive biological processes. The maps possess a particular layout that reflects the accepted vision of spatial organization and propagation of biological processes. The information about molecular mechanisms was manually retrieved by the researchers from the scientific literature along with the information presented in general pathway databases or in the immune systemspecialized resources.
3b ). Each molecular player and reaction in the maps was annotated in the NaviCell format. The NaviCell annotations include PubMed references, cross-references with other databases, and notes of the map manager. In addition, molecular complexes and reactions were assigned with confidence scores and tags indicating their involvement in different biological processes on the maps. Finally, the correspondence of each molecular player on the map to different cell types is also indicated, indicated by cell-type-specific tags (Supplementary Fig. 1 )  35. The principles and procedure for map construction are provided in the Methods. Content and structure of the innate immune maps. Macrophages are the major immune component of leukocyte infiltration in the tumor. The anti-tumor polarization of macrophages is related to their ability to recognize and to reject tumor cells by phagocytosis, represent tumor antigens on the cell surface and induce a T cell response and attract immune cells into the TME.
INHERITS FEATURES OF CELL SPECIFC MAPS AND ALSO CONTAINS: Fig. 1  Map construction workflow and map structure. The scheme demonstrates the steps of meta-map construction starting from collection of cancerspecific and innate-immune specific information about individual molecular interactions from scientific publications and databases, manual annotation and curation of this information (steps 1-4), then organization of this formalized knowledge in form of cell-type specific maps (step 5), and finally integration the cell-type specific networks in one global meta-map of innate immune response in cancer with areas corresponding to biological processes, modules, pro-and anti-tumor polarization (step 6) prognosis. To escape NK control, tumor cells express immunosuppressive cytokines and downregulate NK ligands expression that collectively inhibit cytotoxic activity of NK cells  41.
curie. fr/ pages/maps_innateimmune. html. Each map is presented under three independent platforms, namely NaviCell, MINERVA, and NDEx. All map components are clickable, making the map interactive. The extended annotations of map components contain rich tagging system converted to links and confidence scores. This allows tracing the involvement of molecules into different map sub-structures as pathways, modules, and biological processes (meta-modules) (Fig. 3 ). Tagging system also allows to use the meta-map as a source of annotated signatures (Supplementary Fig. 1 ). The semantic zooming feature of NaviCell  35  simplifies the navigation through large maps of molecular interactions, showing readable amount of details at each zoom level. Comparison of meta-map with existing pathway databases.
The meta-map content (Supplementary Fig. 3 ) and the coverage of literature used to annotate the entities (Supplementary Fig. 4 ) were compared to a sub-set of pathways related to the innate immune system from the existing molecular interaction databases (Supplementary Table  2 ). The InnateDB database contains a detailed description of the innate-immune signaling, even though more general databases as KEGG and REACTOME also include immune pathways. A description of comparison procedure is provided in the Methods. We further compared the major features of innate immune response representation in different pathway databases. The Structure and content of innate immune meta-map innate immune response in cancer resource contains cell-typespecific maps in contrast to other databases. The comparison indicates that the cross-talk between the pathways is visually represented at the maps of immune response in cancer resource.
Finally, the combination of hierarchical organization of knowledge and possibility of navigation through the layers of the maps thanks to semantic zooming feature makes the innate immune resource more suitable for meaningful data visualization. The visualization tool box is built into the NaviCell environment which allows easy data integration and visualization in the context of the innate immune maps (Figs. 4  and 5 ). Taken together, the results of database comparisons indicate that the innate immune response in cancer resource is topicspecific, and describes immune-related and cancer-relevant signaling processes based on the latest publications about innate immune component in TME. The thoughtful layout and visual organization of the biological knowledge on the maps makes it a distinguished resource for data analysis and interpretation.
The results of the enrichment study for the two Macrophage groups were also represented as heatmaps with a significance level of p value for Student's t-test (see Methods) (Supplementary Fig. 5 ). The module activity values were plotted on the maps using BiNoM plugin of Cytoscape  47. Visualization of the module activity scores in the context of macrophage cell-type-specific demonstrates that the module Antigen Presentation is upregulated in Macrophage Group 1 (Fig. 4b ) comparing to Macrophage Group 2 (Fig. 4c ). Whereas, Macrophage Group 2 (Fig. 4c ) shows upregulated modules Core Signaling Pathways and Immunosuppressive Cytokines Pathways comparing to Macrophage Group 1 (Fig. 4b ). Then, the module activity scores for the two Macrophage cell groups were analyzed in the context of the meta-map that allowed to detect several additional modules differentially regulated between the two groups.
Methods Map and model. The maps sere drawn in CellDesigner diagram editor  34  using Process Description (PD) dialect of Systems Biology Graphical Notation (SBGN) syntax which is based on the Systems Biology Markup Language (SBML)  33. The data model used includes the following molecular objects: proteins, genes, RNAs, antisense RNAs, simple molecules, ions, drugs, phenotypes, complexes. These objects can play the role of reactants, products, and regulators in a connected reaction network. The objects phenotypes play a role biological process outcome or readout (e. g. Migration, Tumor killing, ROS production, etc). Edges on the maps represent biochemical reactions or reaction regulations of various types. Different reaction types represent post-translational modifications, translation, transcription, complex formation or dissociation, transport, degradation and so on.
In addition, pathway databases were used to retrieve information of the canonical pathways reported for the innate immune signaling general pathway databases (e. g. KEGG, REACTOME, SPIKE SignaLink, EndoNET) or in the immune system-specialized resources such as VirtuallyImmune (http:// www. virtuallyimmune. org) and InnateDB (www. innatedb. com). Map structure and tag×ging system. The annotation of each molecular object on the maps (protein, gene, RNA, small molecule, etc. ) includes several tags indicating participation of the object in signaling pathways (tag PATHWAY:NAME), functional modules (tag MODULE:NAME), and cell-type-specific map (tag: MAP: NAME). Each PATHWAY obtains the name of the initiating ligand or receptor, in case when several ligands are acting through the same receptor. The tags are converted into the links by the NaviCell factory in the process of online map version generation.
The score reflects an average distance in the PPI graph between all proteins participating in the reaction (reactants, products, or regulators). Map entity annotation in NaviCell format. The annotation panel followed the NaviCell annotation format of each entity of the maps includes sections Identifiers, Maps_Modules, References, and Confidence as detailed in ref. 32. Identifiers section provides standard identifiers and links to the corresponding entity descriptions in HGNC, UniProt, Entrez, SBO, GeneCards, and cross-references in REACTOME, KEGG, Wiki Pathways, and other databases. Maps_Modules section includes tags of modules, meta-modules, and cell-type-specific maps in which the entity is implicated (see above). References section contains links to related publications. Each entity annotation is represented as a post with extended information on the entity.