Miracle-dz/UniFAR-FTUs huggingface dataset:
Generalized Holographic Principle, Gauge Invariance and the Emergence of Gravity à la Wilczek

We show that a generalized version of the holographic principle can be derived from the Hamiltonian description of information flow within a quantum system that maintains a separable state. We then show that this generalized holographic principle entails a general principle of gauge invariance. When this is realized in an ambient Lorentzian space-time, gauge invariance under the Poincaré group is immediately achieved. We apply this pathway to retrieve the action of gravity. The latter is cast à la Wilczek through a similar formulation derived by MacDowell and Mansouri, which involves the representation theory of the Lie groups SO ( 3,2 ) and SO ( 4,1 ) .
Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection

We study the detection of propagandistic text fragments in news articles. Instead of merely learning from input-output datapoints in training data, we introduce an approach to inject declarative knowledge of fine-grained propaganda techniques. We leverage declarative knowledge expressed in both natural language and first-order logic. The former refers to the literal definition of each propaganda technique, which is utilized to get class representations for regularizing the model parameters. The latter refers to logical consistency between coarse- and fine- grained predictions, which is used to regularize the training process with propositional Boolean expressions. We conduct experiments on Propaganda Techniques Corpus, a large manually annotated dataset for fine-grained propaganda detection. Experiments show that our method achieves superior performance, demonstrating that injecting declarative knowledge expressed in both natural language and first-order logic can help the model to make more accurate predictions.
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension

Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages. However, the transfer quality for multilingual Machine Reading Comprehension (MRC) is significantly worse than sentence classification tasks mainly due to the requirement of MRC to detect the word level answer boundary. In this paper, we propose two auxiliary tasks in the fine-tuning stage to create additional phrase boundary supervision: (1) A mixed MRC task, which translates the question or passage to other languages and builds cross-lingual question-passage pairs; (2) A language-agnostic knowledge masking task by leveraging knowledge phrases mined from web. Besides, extensive experiments on two cross-lingual MRC datasets show the effectiveness of our proposed approach.
AxCell: Automatic Extraction of Results from Machine Learning Papers

Tracking progress in machine learning has become increasingly difficult with the recent explosion in the number of papers. In this paper, we present AxCell, an automatic machine learning pipeline for extracting results from papers. AxCell uses several novel components, including a table segmentation subtask, to learn relevant structural knowledge that aids extraction. When compared with existing methods, our approach significantly improves the state of the art for results extraction. We also release a structured, annotated dataset for training models for results extraction, and a dataset for evaluating the performance of models on this task. Lastly, we show the viability of our approach enables it to be used for semi-automated results extraction in production, suggesting our improvements make this task practically viable for the first time. Code is available on GitHub.
LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction through Non-Linear Mapping in Latent Space

Most of the successful and predominant methods for bilingual lexicon induction (BLI) are mapping-based, where a linear mapping function is learned with the assumption that the word embedding spaces of different languages exhibit similar geometric structures (i.e., approximately isomorphic). However, several recent studies have criticized this simplified assumption showing that it does not hold in general even for closely related languages. In this work, we propose a novel semi-supervised method to learn cross-lingual word embeddings for BLI. Our model is independent of the isomorphic assumption and uses nonlinear mapping in the latent space of two independently trained auto-encoders. Through extensive experiments on fifteen (15) different language pairs (in both directions) comprising resource-rich and low-resource languages from two different datasets, we demonstrate that our method outperforms existing models by a good margin. Ablation studies show the importance of different model components and the necessity of non-linear mapping.
CoronaVis: A Real-time COVID-19 Tweets Analyzer

The goal of ConronaVis is to use tweets as the information shared by the people to visualize topic modeling, study subjectivity and to model the human emotions during the COVID-19 pandemic. The main objective is to explore the psychology and behavior of the societies at large which can assist in managing the economic and social crisis during the ongoing pandemic as well as the after-effects of it. The novel coronavirus (COVID-19) pandemic forced people to stay at home to reduce the spread of the virus by maintaining the social distancing. However, social media is keeping people connected both locally and globally. People are sharing information (e.g. personal opinions, some facts, news, status, etc.) on social media platforms which can be helpful to understand the various public behavior such as emotions, sentiments, and mobility during the ongoing pandemic. In this paper, we describe the CoronaVis Twitter dataset (focused on the United States) that we have been collecting from early March 2020. The dataset is available to the research community at this https URL. We would like to share this data with the hope that it will enable the community to find out more useful insights and create different applications and models to fight with COVID-19 pandemic and the future pandemics as well.
Pyramid Attention Networks for Image Restoration

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their "clean" correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code will be available at this https URL
Multiscale Collaborative Deep Models for Neural Machine Translation

Recent evidence reveals that Neural Machine Translation (NMT) models with deeper neural networks can be more effective but are difficult to train. In this paper, we present a MultiScale Collaborative (MSC) framework to ease the training of NMT models that are substantially deeper than those used previously. We explicitly boost the gradient back-propagation from top to bottom levels by introducing a block-scale collaboration mechanism into deep NMT models. Then, instead of forcing the whole encoder stack directly learns a desired representation, we let each encoder block learns a fine-grained representation and enhance it by encoding spatial dependencies using a context-scale collaboration. We provide empirical evidence showing that the MSC nets are easy to optimize and can obtain improvements of translation quality from considerably increased depth. On IWSLT translation tasks with three translation directions, our extremely deep models (with 72-layer encoders) surpass strong baselines by +2.2~+3.1 BLEU points. In addition, our deep MSC achieves a BLEU score of 30.56 on WMT14 English-to-German task that significantly outperforms state-of-the-art deep NMT models. We have included the source code in supplementary materials.
Conditional Spoken Digit Generation with StyleGAN

This paper adapts a StyleGAN model for speech generation with minimal or no conditioning on text. StyleGAN is a multi-scale convolutional GAN capable of hierarchically capturing data structure and latent variation on multiple spatial (or temporal) levels. The model has previously achieved impressive results on facial image generation, and it is appealing to audio applications due to similar multi-level structures present in the data. In this paper, we train a StyleGAN to generate mel-frequency spectrograms on the Speech Commands dataset, which contains spoken digits uttered by multiple speakers in varying acoustic conditions. In a conditional setting our model is conditioned on the digit identity, while learning the remaining data variation remains an unsupervised task. We compare our model to the current unsupervised state-of-the-art speech synthesis GAN architecture, the WaveGAN, and show that the proposed model outperforms according to numerical measures and subjective evaluation by listening tests.
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning

Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. Most of the existing approaches rely on a randomly initialized classifier on top of such networks. We argue that this fine-tuning procedure is sub-optimal as the pre-trained model has no prior on the specific classifier labels, while it might have already learned an intrinsic textual representation of the task. In this paper, we introduce a new scoring method that casts a plausibility ranking task in a full-text format and leverages the masked language modeling head tuned during the pre-training phase. We study commonsense reasoning tasks where the model must rank a set of hypotheses given a premise, focusing on the COPA, Swag, HellaSwag and CommonsenseQA datasets. By exploiting our scoring method without fine-tuning, we are able to produce strong baselines (e.g. 80% test accuracy on COPA) that are comparable to supervised approaches. Moreover, when fine-tuning directly on the proposed scoring function, we show that our method provides a much more stable training phase across random restarts (e.g x10 standard deviation reduction on COPA test accuracy) and requires less annotated data than the standard classifier approach to reach equivalent performances.
Revisiting Pre-Trained Models for Chinese Natural Language Processing

Bidirectional Encoder Representations from Transformers (BERT) has shown marvelous improvements across various NLP tasks, and consecutive variants have been proposed to further improve the performance of the pre-trained language models. In this paper, we target on revisiting Chinese pre-trained language models to examine their effectiveness in a non-English language and release the Chinese pre-trained language model series to the community. We also propose a simple but effective model called MacBERT, which improves upon RoBERTa in several ways, especially the masking strategy that adopts MLM as correction (Mac). We carried out extensive experiments on eight Chinese NLP tasks to revisit the existing pre-trained language models as well as the proposed MacBERT. Experimental results show that MacBERT could achieve state-of-the-art performances on many NLP tasks, and we also ablate details with several findings that may help future research. https://github.com/ymcui/MacBERT
The Rotation Curve, Mass Distribution, and Dark Matter Content of the Milky Way from Classical Cepheids

With the increasing number of large stellar survey projects, the quality and quantity of excellent tracers for studying the Milky Way are rapidly growing, one of which is the classical Cepheids. Classical Cepheids are high-precision standard candles with very low typical uncertainties (<3%) available via the mid-infrared period–luminosity relation. About 3500 classical Cepheids identified from the Optical Gravitational Lensing Experiment, All-Sky Automated Survey for Supernova, Gaia, Wide-field Infrared Survey Explorer, and Zwicky Transient Facility survey data have been analyzed in this work, and their spatial distributions show a clear signature of Galactic warp. Two kinematical methods are adopted to measure the Galactic rotation curve (RC) in the Galactocentric distance range of . Gently declining RCs are derived by both the proper motion (PM) method and three-dimensional velocity vector (3DV) method. The largest sample of classical Cepheids with the most accurate 6D phase-space coordinates available to date are modeled in the 3DV method, and the resulting RC is found to decline at the relatively smaller gradient of (−1.33 ± 0.1) . Comparing to results from the PM method, a higher rotation velocity ((232.5 ± 0.83) ) is derived at the position of the Sun in the 3DV method. The virial mass and local dark matter density are estimated from the 3DV method, which is the more reliable method, and GeV, respectively.
Deep Reinforcement Learning with Graph-based State Representations

Deep RL approaches build much of their success on the ability of the deep neural network to generate useful internal representations. Nevertheless, they suffer from a high sample-complexity and starting with a good input representation can have a significant impact on the performance. In this paper, we exploit the fact that the underlying Markov decision process (MDP) represents a graph, which enables us to incorporate the topological information for effective state representation learning. Motivated by the recent success of node representations for several graph analytical tasks we specifically investigate the capability of node representation learning methods to effectively encode the topology of the underlying MDP in Deep RL. To this end we perform a comparative analysis of several models chosen from 4 different classes of representation learning algorithms for policy learning in grid-world navigation tasks, which are representative of a large class of RL problems. We find that all embedding methods outperform the commonly used matrix representation of grid-world environments in all of the studied cases. Moreoever, graph convolution based methods are outperformed by simpler random walk based methods and graph linear autoencoders.
Exponential Decay for Damped Klein-Gordon Equations on Asymptotically Cylindrical and Conic Manifolds

We study the decay of the global energy for the damped Klein-Gordon equation on non-compact manifolds with finitely many cylindrical and subconic ends up to bounded perturbation. We prove that under the Geometric Control Condition, the decay is exponential, and that under the weaker Network Control Condition, the decay is logarithmic, by developing the global Carleman estimate with multiple weights.
Span-based Localizing Network for Natural Language Video Localization

Given an untrimmed video and a text query, natural language video localization (NLVL) is to locate a matching span from the video that semantically corresponds to the query. Existing solutions formulate NLVL either as a ranking task and apply multimodal matching architecture, or as a regression task to directly regress the target video span. In this work, we address NLVL task with a span-based QA approach by treating the input video as text passage. We propose a video span localizing network (VSLNet), on top of the standard span-based QA framework, to address NLVL. The proposed VSLNet tackles the differences between NLVL and span-based QA through a simple and yet effective query-guided highlighting (QGH) strategy. The QGH guides VSLNet to search for matching video span within a highlighted region. Through extensive experiments on three benchmark datasets, we show that the proposed VSLNet outperforms the state-of-the-art methods; and adopting span-based QA framework is a promising direction to solve NLVL.
WoodFisher: Efficient second-order approximations for model compression

Second-order information, in the form of Hessian- or Inverse-Hessian-vector products, is a fundamental tool for solving optimization problems. Recently, there has been significant interest in utilizing this information in the context of deep neural networks; however, relatively little is known about the quality of existing approximations in this context. Our work examines this question, identifies issues with existing approaches, and proposes a method called WoodFisher to compute a faithful and efficient estimate of the inverse Hessian. Our main application is to neural network compression, where we build on the classic Optimal Brain Damage/Surgeon framework. We demonstrate that WoodFisher significantly outperforms popular state-of-the-art methods for one-shot pruning. Further, even when iterative, gradual pruning is considered, our method results in a gain in test accuracy over the state-of-the-art approaches, for pruning popular neural networks (like ResNet-50, MobileNetV1) trained on standard image classification datasets such as ImageNet ILSVRC. We examine how our method can be extended to take into account first-order information, as well as illustrate its ability to automatically set layer-wise pruning thresholds and perform compression in the limited-data regime. The code is available at the following link, this https URL.
Research on the Effects of Different Phosphorus Doses on Root, Nodule and Plant Growth in Some Vetch Species

This research was conducted to determine the effects of different phosphorus doses (0, 4, 8 and 12 kg P2O5/da) on root, above-ground biomass and nodule development of vetch species (Vicia villosa Roth., V. pannonica Crantz. and V. sativa L.) in the experimental field of the Agricultural Faculty of Gaziosmanpasa University in the 1997-98 growing season. Common vetch had the highest above-ground biomass, number of nodules on the lateral roots and total nodule number, whereas Hungarian vetch had the highest root biomass, number of nodules on the main root and nodule dry weight. Phosphorus fertilization enhanced all the parameters investigated. It was concluded that a phosphorus dose of 12 kg/da could be recommended for optimal root, stem and nodule development in the vetch species studied.
Role of microbial adaptation in the biodegradation of chemical pollutants : extrapolation from laboratory to rivers

The Organisation for Economic Cooperation and Development (OECD) established standardized tests to predict the chemical behaviour in the environment. Difficulties exist in the extrapolation of data from laboratory OECD tests into the environment, and prediction of possible scenarios regarding environmental pollution is not accurate. The current project aimed to increase the realism of OECD tests and to investigate the microbial populations involved in the biodegradation of paranitrophenol (PNP). River water, sediment and biofilms were sampled from a stretch of River Dene with an effluent discharge point (Wellesbourne WWTP), and were used as microbial inocula in biodegradation studies. The effect of light, PNP concentration and inoculum preparation on PNP biodegradation was determined. Culture dependent techniques were used for isolation of PNP-degrading bacteria while culture independent techniques; including 16S rRNA Terminal Restriction Fragment Length Polymorphism, QPCR, and high throughput sequencing targeting the PNP functional genes (pnpA and mar), enabled detection and characterization of PNP-degrading bacteria. Light incubation lead to increased river water pH which inhibited PNP degradation. A threshold PNP concentration was determined around 42 μg/L. Application of biofilm inocula improved the reproducibility of PNP biodegradation at a concentration of 2 mg/L, and increased the amount of microbial biomass in test systems. Pseudomonas syringae was identified as a key PNPdegrader. Additional groups of PNP-degrading bacteria were detected based on the analysis of pnpA and mar functional markers. It was shown that location of sampling site for inoculum collection had no impact on biodegradation test outcome but variation of microbial inocula between sampling dates may affect the biodegradation of PNP. Core and satellite taxa analysis demonstrated that ‘biodegradation lottery’ is not the major process which determines the successful chemical biodegradation, and that probably microbial interactions within inoculum affect proliferation of PNPdegraders, and therefore impact on test results. This project revealed the lack of consideration of microbial inocula in the OECD biodegradation test guidelines. Further experimental work was suggested to expand current studies to different chemicals and rivers, and to develop more predictable approaches for better chemical risk management by Industry and Regulatory bodies.
Towards Transparent and Explainable Attention Models

Recent studies on interpretability of attention distributions have led to notions of faithful and plausible explanations for a model’s predictions. Attention distributions can be considered a faithful explanation if a higher attention weight implies a greater impact on the model’s prediction. They can be considered a plausible explanation if they provide a human-understandable justification for the model’s predictions. In this work, we first explain why current attention mechanisms in LSTM based encoders can neither provide a faithful nor a plausible explanation of the model’s predictions. We observe that in LSTM based encoders the hidden representations at different time-steps are very similar to each other (high conicity) and attention weights in these situations do not carry much meaning because even a random permutation of the attention weights does not affect the model’s predictions. Based on experiments on a wide variety of tasks and datasets, we observe attention distributions often attribute the model’s predictions to unimportant words such as punctuation and fail to offer a plausible explanation for the predictions. To make attention mechanisms more faithful and plausible, we propose a modified LSTM cell with a diversity-driven training objective that ensures that the hidden representations learned at different time steps are diverse. We show that the resulting attention distributions offer more transparency as they (i) provide a more precise importance ranking of the hidden states (ii) are better indicative of words important for the model’s predictions (iii) correlate better with gradient-based attribution methods. Human evaluations indicate that the attention distributions learned by our model offer a plausible explanation of the model’s predictions. Our code has been made publicly available at https://github.com/akashkm99/Interpretable-Attention