dataesr/scientific-paragraphs-categorization huggingface dataset:
A New Typology: Gradience and Context-Sensitivity As shown in § 4. 2, most of the typology-savvy algorithms thus far exploited features extracted from manually crafted databases. However, this approach is riddled with several shortcomings, which are reflected in the small performance improvements observed in § 5. 4. Luckily, these shortcomings may potentially be averted through the use of methods that allow typological information to emerge from the data in a bottomup fashion, rather than being predetermined. In what follows we advocate for such a data-driven approach, based on several considerations. Firstly, typological databases provide incomplete documentation of the cross-lingual variation, in terms of features and languages. Raw textual data, which is easily accessible for many languages and is cost-effective, may provide a valid alternative that can facilitate automatic learning of more complete knowledge.
Secondly, database information is approximate, as it is restricted to the majority strategy within a language. However, in theory each language allows for multiple strategies in different contexts and with different frequencies, hence databases risk hindering models from learning less-likely but plausible patterns  (Sproat 2016). Inferring typological information from text would enable a system to discover patterns within individual examples, including both the frequent and the infrequent ones. Thirdly, typological features in databases are discrete, utilizing predefined categories devised to make high-level generalizations across languages. However, several categories in natural language are gradient (see for instance the discussion on semantic categorization in § 2), hence they are better captured by continuous features.
In addition to being psychologically motivated, this sort of gradient representation is also more compatible with machine learning algorithms and particularly with deep neural models that naturally operate with real-valued multidimensional word embeddings and hidden states. To sum up, the automatic development of typological information and its possible integration into machine learning algorithms have the potential to solve an important bottleneck in polyglot NLP. Current manually curated databases consist of incomplete, approximate, and discrete features that are intended to reflect contextual and gradient information implicitly present in text. These features are fed to continuous, probabilistic, and contextual machine learning models-which do not form a natural fit for the typological features.
The selection of similar sentences rather than similar languages as source data in language transfer is likely to yield large improvements, as demonstrated by  Agić (2017)  for parsing in an oracle setting. Finally, the bottom-up development of typological features may address also radically resource-less languages that lack even raw textual data in a digital format. For this group, which still constitutes a large portion of the world's languages, there are often available reference grammars written by field linguists, which are the ultimate source for typological databases. These grammars could be queried automatically, and fine-grained typological information could be harvested through information extraction techniques.
Conclusions In this article, we surveyed a wide range of approaches integrating typological information, derived from the empirical and systematic comparison of the world's languages, and NLP algorithms. The most fundamental problem for the advancement of this line of research is bridging the gap between the interpretable, language-wide, and discrete features of linguistic typology found in database documentation, and the opaque, contextual, and probabilistic models of NLP. We addressed this problem by exploring a series of questions: (i) for which tasks and applications is typology useful? (ii) Which are the advantages and limitations of currently available typological databases? Can data-driven inference of typological features offer an alternative source of information? (iii) Which methods allow us to inject typological information from external resources, and how should such information be encoded? (iv) By which margin do typology-savvy methods surpass typology-agnostic baselines? How does typology compare to other criteria of language classification, such as genealogy? (v) In addition to augmenting machine learning algorithms, which other purposes do typology serve for NLP? We summarize our key findings here: 1.
Typological information is currently used predominantly for morphosyntactic tasks, in particular dependency parsing. As a consequence, these approaches typically select a limited subset of features from a single data set (WALS) and focus on a single aspect of variation (typically word order). However, typological databases also cover other important features, related to predicate-argument structure (ValPaL), phonology (LAPSyD, PHOIBLE, StressTyp2) and lexical semantics (IDS, ASJP), which are currently largely neglected by the multilingual NLP community. In fact, these features have the potential to benefit many tasks addressed by language transfer or joint multilingual learning techniques, such as semantic role labeling, word sense disambiguation, or sentiment analysis.
2. Typological databases tend to be incomplete, containing missing values for individual languages or features. This hinders the integration of the information in such databases into NLP models; and therefore, several techniques have been developed to predict missing values automatically. They include heuristics derived from morphosyntactic annotation; propagation from other languages based on hierarchical clusters or similarity metrics; supervised models; and distributional methods applied to multi-parallel texts. However, none of these techniques surpasses the others across the board in prediction accuracy, as each excels in different feature types. A challenge left for future work is creating ensembles of techniques to offset their individual disadvantages.
In fact, typological database documentation is incomplete, approximate, and discrete. As a consequence, it does not fit well with the gradient and contextual models of machine learning. However, typological databases are originally created from raw linguistic data. An alternative approach could involve learning typology from such data automatically (i. e. , from scratch). This would capture the variation within languages at the level of individual examples, and to naturally encode typological information into continuous representations. These goals have already been partly achieved by methods involving language vectors, heuristics derived from morphosyntactic annotation, or distributional information from multi-parallel texts. The main future challenge is the integration of these methods into machine learning models, as opposed to sourcing typological features from databases.
Georgi, Xia, and Lewis (2010) for prediction of typological features. Unsupervised Propagation. Another line of research seeks to increase the coverage of typological databases borrowing missing values from the known values in other languages. One approach is clustering languages according to some criterion and propagating the majority value within each cluster. Hierarchical clusters can be created either according to typological features (e. g. ,Teh, Daumé III, and Roy 2007)or based on language genus(Coke, King, and Radev 2016). Through extensive evaluation,Georgi, Xia, and Lewis (2010)demonstrate that typology based clustering outperforms genealogical clustering for unsupervised propagation of typological features. Among the clustering techniques examined, k-means appears to be the most reliable as compared to k-medoids, the Unweighted Pair Group Method with Arithmetic mean, repeated bisection, and hierarchical methods with partitional clusters.
Methods Patients with lower-risk MDS (IPSS risk: low or intermediate-1) 3 from 16 European countries and Israel were included in the EUMDS Registry, after providing signed informed consent, within 100 days of their initial diagnosis of a MDS which was made according to the World Health Organization (WHO) 2001 criteria. 18 Patients with an IPSS intermediate-2 or high risk, or with therapy-related MDS were excluded, but MDS-specific treatment, started before registration within 100 days after diagnosis, was not a reason for exclusion. Data were collected at baseline and at each 6-monthly outpatient routine follow-up visit. Clinical information was collected on: demographics, anthropometrics, co-morbidities, performance status, quality of life (EQ-5D), concomitant medication, laboratory parameters, diagnostics including information on bone marrow morphology, histology, cytogenetics, RBCT episodes, total number of transfused units and simultaneous therapeutic interventions.
All subjects were followed prospectively by full reports every 6 months until death, progression to high risk MDS or leukemia, loss to follow-up or withdrawal of informed consent. The Registry was approved by each institution's ethics committee in accordance with national legislation. Transfusion data available in the EUMDS Registry consists of the number of units received between each reported visit, usually at 6-month intervals. In order to assess the association between transfusions received and PFS, proportional hazards regression with time-varying covariates was employed, adjusting the effect of transfusions by appropriate baseline and time-varying variables. For the purposes of the time-to-event analyses, time was measured from the date of diagnosis with MDS to the date of disease progression or date of death.
In conclusion, the negative effect of transfusion treatment on PFS already occurs at transfusion densities below 3 units/16 weeks. This indicates that transfusion dependency, even at relatively low dose densities, may be considered as an indicator of inferior PFS. This trial was registered at www. clinicaltrials. gov as #NCT00600860. analysis was performed in the survival regressions to take into account that the population was not homogeneous but distributed over different centers in several countries, using a random effects frailty term. The random effect, called "frailty", is the term that describes the common risk or the individual heterogeneity, acting as a factor on the hazard function. Missing values in adjustment variables were imputed with last observation carried forward or next observation carried backward.
Patients' characteristics The EUMDS Registry contained data from 2,192 patients diagnosed between December 3, 2007 and March 14, 2017 of whom 1,504 patients had data recorded from three or more visits (visit 3 = landmark at the 1-year follow-up). Two patients with refractory anemia with excess blasts-2 were excluded, resulting in the inclusion of 1,502 patients. An additional 235 patients were excluded, as one or more of the following variables had never been measured or the test failed throughout the study: cytogenetics (n=112), EQ-5D (n=101), blast count (n=60), platelet count (n=1), and neutrophil count (n=2). The final cohort consisted of 1,267 patients, unselected for any type of treatment. In 162 patients the disease had progressed to higher-risk MDS or acute myeloid leukemia and 317 patients had died without progression.
The EUMDS Registry is collecting its observational data at registration of each new patient (within 100 days after diagnosis) and follow-up data at 6-monthly intervals. This practice leads to regular visit intervals of 6 months. For many patients in this dataset, the value of the recorded transfusion rate varied strongly over time, as shown in the Online Supplementary Files. We therefore calculated the RBCT rate at each reported visit during all preceding visit intervals between the date of the first RBCT and the date of the last visit, leading to a "smoothed" variable, defined as dose density. This reflects an average rate of receiving transfusions during the whole observation period with transfusions. The relatively low number of red cell units transfused per month can be explained by the remarkable variation of the transfusion rate over time, even when using interval visit reports of 6 months' duration.
Funding The work of the EUMDS Registry is supported by an educational grant from  Novartis Pharmacy B. V. Oncology Europe, and Amgen Limited. This work is part of the  MDS-RIGHT activities , which has received funding from the  European Union 's  Horizon 2020 research and innovation programme  under grant agreement n. 634789 -"Providing the right care to the right patient with  MyeloDysplastic Syndrome  at the right time".
Reciprocal Deletion and Duplication of 16p11. 2 Have Mirror Effects on Craniofacial Structure 3D morphometric facial imaging was performed on subjects with 16p11. 2 duplications or deletions and controls recruited to the Simons VIP study  (Simons VIP Consortium, 2012;  see STAR Methods). The final dataset (N = 228; Table  S1 ) included 45 with deletions, 44 with duplications, and 139 familial non-carrier controls. A total of 24 landmarks were placed on each image (Figure  1A ; descriptions of landmarks are provided in Table  S2 ). "Features," defined as pairwise distances between landmarks, were normalized to the geometric mean. Differences in deletion and duplication groups relative to controls were detected by linear regression, and covariates were included in the model to control for known factors that influence dimensions of head and face in this genetic disorder, including age, head circumference, BMI, sex, and ancestry principal components obtained from genetic data, with a random intercept allowed to account for within-family correlation.
EXPERIMENTAL MODEL AND SUBJECT DETAILS Human subjects-Subjects were recruited in conjunction with the Simons VIP study  (Simons VIP Consortium). 3D morphometric facial imaging was performed on a subset (N = 359) of subjects with 16p11. 2 duplications or deletions from the Simons VIP cohort at 3 sites (University of Washington, Harvard University, and Baylor College of Medicine) using the 3DMD 3-pod camera system. Data analysis was restricted to subjects of European ancestry older than 3 years of age. Additional subjects were excluded due to facial hair, image quality or landmark visibility (i. e. , obscured by clothing, hair or makeup). The final dataset (N = 228) included 45 deletions, 44 duplications and 139 familial non-carrier controls (Table  S1 ). Rodent models of 16p11. 2 deletion and duplication-The 0.
PCR products were denatured, reannealed slowly and separated on a 20% TBE 1. 0-mm precast polyacrylamide gel (ThermoFisher), which was then incubated in ethidium bromide and imaged on a ChemiDoc system (Bio-Rad) to visualize hetero-and homoduplexes. To estimate the percentage of mosaicism of F0 mutants (n = 5/condition), PCR products were gel purified (QIAGEN), and cloned into a pCR8/GW/ TOPO-TA vector (Thermo Fisher). Plasmid was prepped from individual colonies (n = 9-12 colonies/embryo) and Sanger sequenced according to standard procedures. Automated zebrafish imaging: Larvae were maintained under standard conditions at 28. 5°C until 3 dpf and were positioned and imaged live as described  (Isrie et al. , 2015). Automated imaging was conducted with an AxioScope. A1 microscope and Axiocam 503 monochromatic camera facilitated by Zen Pro software (Zeiss), to capture dorsal images of GFP signal.
Larval batches were positioned and imaged live using the Vertebrate Automated Screening Technology (VAST; software version 1. 2. 5. 4; Union Biometrica) BioImager. Larvae from each experimental condition were anesthetized with 0. 2 mg/mLTricaine prior to being loaded into the sample reservoir. Dorsal and lateral image templates of uninjected controls and experimental larvae were created and we acquired images at a > 70% minimum similarity for the pattern-recognition algorithms. Larvae were rotated to 180° to acquire ventral images via a 10x objective and fluorescent excitation at 470nm to detect GFP to capture fluorescent images of the pharyngeal skeleton. ImageJ software (NIH) was used to measure the angle of the ceratohyal cartilage. All experimental conditions were normalized to uninjected controls and set to 100 degrees.
Examining levels of gene expression during murine craniofacial development: We examined whether the significant effects of seven genes (SPN, C16orf54, SEZ6L2, ASPHD1, TAOK2, INO80E and FAM57B) on shape of the mandible could be attributable to the differential regulation of these genes. A published dataset was obtained consisting of Affymetrix (Mouse Gene ST 1. 0 array) gene expression analysis of the major craniofacial processes of the developing mouse embryo (E10. 5-E12. 5) (Hooper etal. , 2017)  (accession # FB00000803, Facebase. org). Samples included mesenchymal and ectodermal cells of the frontonasal and mandibular processes of embryos at E10. 5, E11. 5 and E12. 5 in triplicate, and samples of the maxillary process at E11. 5 and E12. 5. Summary gene level expression data was obtained (https://www.
Analysis of human craniofacial features: Quantitative measurement of all pairwise distances between 24 landmarks were calculated using the 3dMDvultus -Analysis software, version 2. 5. 0. 1 (http://www. 3dmd. com). Symmetric distances were averaged, yielding 156 facial distance measurements. Each distance was normalized to the overall size of the individual's face, by dividing by the geometric mean of the 156 distances for that individual. Angular measurements of the nose and chin, the nasomental (NMA) and labiomental (LMA) angles respectively, were calculated by triangulating the relevant landmarks. A series of linear mixed-effects models, using package lme4 in R (version 3. 4. 1), was used to separately test for the effect of deletion and of duplication on each angle or normalized facial distance.
Generating 3D models by averaging faces of deletion, duplication and control subjects: To visualize the respective facial gestalts of controls, deletion carriers and duplication carriers, a 3D-model of each was generated by averaging of the surface topography of faces from multiple subjects using the 3dMDvultus-Analysis software, version 2. 5. 0. 1 (www. 3dmd. com). To maximize the number of unrelated subjects that were closely matched in age within each group, selection criteria for averaging of faces differed slightly from that of the overall dataset. Only unrelated individuals were included, additional subjects were removed due to image quality (gaps in the surface topography), and the requirement for landmark visibility was relaxed, allowing for frontotemporal landmarks (landmarks 1 and 9, see Figure  1 ) to be covered in some cases by hair or headwear.
We tested all 91 distances on the dorsal skull and two distances on the mandible. Effects that were significant at an FDR of 5% were identified. In addition, as we did previously in human, we identified a set of features that distinguish CNV models from controls by performing least absolute shrinkage and selection operator (LASSO) based on all univariate significant distances by generalized linear model, with AIC as the criteria rule. DATA AND CODE AVAILABILITY-Human 3D images and were obtained from the Simons VIP Consortium  (Simons VIP Consortium, 2012). Clinical phenotypes and SNP and CNV genotype data on Simons VIP subjects were obtained from SFARI base (https:// base. sfari. org/). Quantitative data extracted from 3D surface images and CT scans of human, rat, and mouse are available by request from Jonathan Sebat (jsebat@ucsd.
ACKNOWLEDGMENTS This study was funded by grants to J. S. from the  Simons Foundation  (SFARI # 178088 ) and the  NIH  ( MH119746 ); a grant to E. E. D. from the  NIMH  ( MH106826 ); a  NIMH   Silvio O. Conte Center grant  to  N. K. (P50MH094268 ); a grant to Y. H. from the  French Foundation for Rare Diseases ; and grants to L. M. I. from the  Simons Foundation  (SFARI # 345469 ) and  NIMH  ( MH109885 ;  MH108528 ;  MH104766 ). We are grateful to all of the families at the participating Simons Variation in Individuals Project ( Simons VIP ) sites, as well as the  Simons VIP Consortium. We appreciate obtaining access to 3D human imaging and genetic data from SFARI. Approved researchers can obtain the Simons VIP datasets described in this study (http://www. sfari. org/resource/simons-searchlight/)by applying at https://base.
INTRODUCTION Segmentation of images acquired with different lighting conditions is a challenging task in image analysis that can occur in many settings such as visual inspection for industry  (Cord et al. , 2010; Noyel, 2011; Noyel et al. , 2013; Parra-Denis et al. , 2011) , medical images  (Noyel et al. , 2014; 2017) , security  (Foresti et al. , 2005) , driving assistance  (Hautière et al. , 2006) , etc. Lighting variations obviously represent a substantial obstacle for the development of reliable image processing algorithms. To overcome this difficulty, we can determine the two following approaches: (i) Firstly, a practical approach consisting of circumventing the problem, e. g. by a learning process. This is often the case for face recognition, where a database is created by varying together the acquisition angle and the intensity of the lighting source  (Ramaiah et al.
and Jourlin (1997). The new region R n+1 is the contraction of D n+1 : Ctrct(D n+1 ). 1. If H f (D n+1 ) ≤ t, The algorithm stops when the region R n+1 does not grow any more. For a fully automatic segmentation method, the seed points could be chosen as the minima of a function such as: a gradient of the image -like for the watershed transform  (Beucher and Meyer, 1992 ) -or a map of Asplund's distances of the image  (Jourlin et al. , 2012; 2014; Noyel and Jourlin, 2015; 2017b; a). The seed points could also be automatically determined by a previous classification  (Noyel et al. , 2007; 2010; 2014). As the homogeneity criterion is invariant to contrast changes with an optical cause, the threshold could be learnt from a training database  (Lu et al. , 2017). It could also be automatically chosen as the most frequent contrast on every pixel neighbourhoods  (Jourlin, 2016).
Simulated images For this experiment, we have selected a colour image of a butterfly (Fig. 1a ) from the dataset YFCC100M (Yahoo Flickr Creative Commons 100M)  (Thomee et al. , 2016; Butterfly, 2010). We have extracted its luminance as a grey-level image f (Fig. 1b ). In order to verify the respective insensitivities of the criteria H + + + and H × × × to lighting variations simulated by a LIP-addition + + + or a LIPmultiplication × × × , we have darkened and brightened this luminance image f by using each of the LIPoperations. For the LIP-additive homogeneity criterion H + + + f (R), a dark image f dk,a is obtained by a LIPaddition of a constant k = 120 to the complement f c of the image f , f dk,a = ( f c + + + k) c (Fig. 1c ). The image complement, which is defined by f c = M -1f , allows to take advantage of the LIP-scale.
2b ), f 1 (Fig. 2d ) and f 2 (Fig. 2f ). The user selected regions G are the same seed points which are used for our approach after a dilation by a square of length side 3 pixels. As the user selected regions G are much smaller than the letters that we want to segment, we choose a parameter α = 0 in order to minimise the number of false-negatives d 0 (∪ N∈K N, G) in those regions G. After minimisation, this number is equal to zero. In order, to verify the low sensitivity to object opacity of the LIP-multiplicative homogeneity criterion H × × × , we have selected three X-ray images of a luggage acquired under different exposures for security purpose (Fig. 3 ). The images are coming from a publicly available database, namely GDXray  (Mery et al. , 2015; GDXray, 2015). The object to be detected is a razor blade.