Miracle-dz/UniFAR-FTUs huggingface dataset:
El Departamento de Nosotros: How Machine Translated Corpora Affects Language Models in MRC Tasks

Pre-training large-scale language models (LMs) requires huge amounts of text corpora. LMs for English enjoy ever growing corpora of diverse language resources. However, less resourced languages and their mono- and multilingual LMs often struggle to obtain bigger datasets. A typical approach in this case implies using machine translation of English corpora to a target language. In this work, we study the caveats of applying directly translated corpora for fine-tuning LMs for downstream natural language processing tasks and demonstrate that careful curation along with post-processing lead to improved performance and overall LMs robustness. In the empirical evaluation, we perform a comparison of directly translated against curated Spanish SQuAD datasets on both user and system levels. Further experimental results on XQuAD and MLQA transfer-learning evaluation question answering tasks show that presumably multilingual LMs exhibit more resilience to machine translation artifacts in terms of the exact match score.
Towards Efficient Connected and Automated Driving System via Multi-agent Graph Reinforcement Learning

Connected and automated vehicles (CAVs) have attracted more and more attention recently. The fast actuation time allows them having the potential to promote the efficiency and safety of the whole transportation system. Due to technical challenges, there will be a proportion of vehicles that can be equipped with automation while other vehicles are without automation. Instead of learning a reliable behavior for ego automated vehicle, we focus on how to improve the outcomes of the total transportation system by allowing each automated vehicle to learn cooperation with each other and regulate human-driven traffic flow. One of state of the art method is using reinforcement learning to learn intelligent decision making policy. However, direct reinforcement learning framework cannot improve the performance of the whole system. In this article, we demonstrate that considering the problem in multi-agent setting with shared policy can help achieve better system performance than non-shared policy in single-agent setting. Furthermore, we find that utilization of attention mechanism on interaction features can capture the interplay between each agent in order to boost cooperation. To the best of our knowledge, while previous automated driving studies mainly focus on enhancing individual's driving performance, this work serves as a starting point for research on system-level multi-agent cooperation performance using graph information sharing. We conduct extensive experiments in car-following and unsignalized intersection settings. The results demonstrate that CAVs controlled by our method can achieve the best performance against several state of the art baselines.
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion

Code autocompletion is an integral feature of modern code editors and IDEs. The latest generation of autocompleters uses neural language models, trained on public open-source code repositories, to suggest likely (not just statically feasible) completions given the current context. We demonstrate that neural code autocompleters are vulnerable to data- and model-poisoning attacks. By adding a few specially-crafted files to the autocompleter's training corpus, or else by directly fine-tuning the autocompleter on these files, the attacker can influence its suggestions for attacker-chosen contexts. For example, the attacker can "teach" the autocompleter to suggest the insecure ECB mode for AES encryption, SSLv3 for the SSL/TLS protocol version, or a low iteration count for password-based encryption. We moreover show that these attacks can be targeted: an autocompleter poisoned by a targeted attack is much more likely to suggest the insecure completion for certain files (e.g., those from a specific repo). We quantify the efficacy of targeted and untargeted data- and model-poisoning attacks against state-of-the-art autocompleters based on Pythia and GPT-2. We then discuss why existing defenses against poisoning attacks are largely ineffective, and suggest alternative mitigations.
Smart Home, security concerns of IoT

The IoT (Internet of Things) has become widely popular in the domestic environments. People are renewing their homes into smart homes; however, the privacy concerns of owning many Internet connected devices with always-on environmental sensors remain insufficiently addressed. Default and weak passwords, cheap materials and hardware, and unencrypted communication are identified as the principal threats and vulnerabilities of IoT devices. Solutions and countermeasures are also provided: choosing a strong password, strong authentication mechanisms, check online databases of exposed or default credentials to mitigate the first threat; a selection of smart home devices from reputable companies and the implementation of the SDN for the Dos/DDoS threat; and finally IDS, HTTPS protocol and VPN for eavesdropping. The paper concludes dealing with a further challenge, "the lack of technical support", by which an auto-configuration approach should be analysed; this could both ease the installation/maintenance and enhance the security in the self configuration step of Smart Home devices.
Efficient excitations and spectra within a perturbative renormalization approach.

We present a self-consistent approach for computing the correlated quasiparticle spectrum of charged excitations in iterative $\mathcal{O}[N^5]$ computational time. This is based on the auxiliary second-order Green's function approach [O. Backhouse \textit{et al.}, JCTC (2020)], in which a self-consistent effective Hamiltonian is constructed by systematically renormalizing the dynamical effects of the self-energy at second-order perturbation theory. From extensive benchmarking across the W4-11 molecular test set, we show that the iterative renormalization and truncation of the effective dynamical resolution arising from the $2h1p$ and $1h2p$ spaces can substantially improve the quality of the resulting ionization potential and electron affinity predictions compared to benchmark values. The resulting method is shown to be superior in accuracy to similarly scaling quantum chemical methods for charged excitations in EOM-CC2 and ADC(2), across this test set, while the self-consistency also removes the dependence on the underlying mean-field reference. The approach also allows for single-shot computation of the entire quasiparticle spectrum, which is applied to the benzoquinone molecule and demonstrates the reduction in the single-particle gap due to the correlated physics, and gives direct access to the localization of the Dyson orbitals.
Relevance Transformer: Generating Concise Code Snippets with Relevance Feedback

Tools capable of automatic code generation have the potential to augment programmer's capabilities. While straightforward code retrieval is incorporated into many IDEs, an emerging area is explicit code generation. Code generation is currently approached as a Machine Translation task, with Recurrent Neural Network (RNN) based encoder-decoder architectures trained on code-description pairs. In this work we introduce and study modern Transformer architectures for this task. We further propose a new model called the Relevance Transformer that incorporates external knowledge using pseudo-relevance feedback. The Relevance Transformer biases the decoding process to be similar to existing retrieved code while enforcing diversity. We perform experiments on multiple standard benchmark datasets for code generation including Django, Hearthstone, and CoNaLa. The results show improvements over state-of-the-art methods based on BLEU evaluation. The Relevance Transformer model shows the potential of Transformer-based architectures for code generation and introduces a method of incorporating pseudo-relevance feedback during inference.
Parameter Identifiability for a Profile Mixture Model of Protein Evolution

A profile mixture (PM) model is a model of protein evolution, describing sequence data in which sites are assumed to follow many related substitution processes on a single evolutionary tree. The processes depend, in part, on different amino acid distributions, or profiles, varying over sites in aligned sequences. A fundamental question for any stochastic model, which must be answered positively to justify model-based inference, is whether the parameters are identifiable from the probability distribution they determine. Here, using algebraic methods, we show that a PM model has identifiable parameters under circumstances in which it is likely to be used for empirical analyses. In particular, for a tree relating 9 or more taxa, both the tree topology and all numerical parameters are generically identifiable when the number of profiles is less than 74.
MORUS SPECIES THROUGH CENTURIES IN PHARMACY AND AS FOOD

Coloured fruits are good sources of phenolic com-pounds including anthocyanins, flavonoids and carot-enoids [1-4]. Mulberry fruits are rich in phenols and have a unique sour and refreshing taste [5]. They are used as a traditional medicine in curing dental diseases, dia-betes, hypertension, arthritis and anemia [6]. With the aim of finding new sources of natural antioxidants, fruits, vegetables and other plants with the antioxidant activity were investigated [7-12]. Genus Morus belongs to Moraceae family, and the names of over 150 species have been published. Differ-ent sources usually cite different selections of accepted names. Only 10-16 are generally cited as being accept-ed by the vast majority of botanical authorities [13]. The best known mulberry species are white mulberry (
The network structure of cultural distances

This paper proposes a novel measure of cultural distances between countries. Making use of the information coming from the World Value Survey (Wave 6), and considering the interdependence among cultural traits, the paper proposes a methodology to define the cultural distance between countries, that takes into account the network structure of national cultural traits. Exploiting the possibilities offered by Copula graphical models for ordinal and categorical data, the paper infers the network structure of 54 countries and proposes a new summary measure of national cultural distances. The DBRV Cultural Distance index shows that, as for 2010-2014, compared to Inglehart and Welzel (2005) the world appears to be more culturally heterogeneous than what it was previously thought.
Evaluating the Calibration of SN Ia Anchor Datasets with a Bayesian Hierarchical Model

Inter-survey calibration remains an important systematic uncertainty in cosmological studies using type Ia supernova (SNe Ia). Ideally, each survey would measure its system throughputs, for instance with bandpass measurements combined with observations of well-characterized spectrophotometric standard stars; however, many important nearby-SN surveys have not done this. We recalibrate these surveys by tying their tertiary survey stars to Pan-STARRS1 g, r, and i, and SDSS/CSP u. This improves upon previous recalibration efforts by taking the spatially variable zeropoints of each telescope/camera into account, and applying improved color transformations in the surveys' natural instrumental photometric systems. Our analysis uses a global hierarchical model of the data which produces a covariance matrix of magnitude offsets and bandpass shifts, quantifying and reducing the systematic uncertainties in the calibration. We call our method CROSS-CALIBration with a Uniform Reanalysis (X-CALIBUR). This approach gains not only from a sophisticated analysis, but also from simply tying our calibration to more color calibrators, rather than just the one color calibrator (BD+17 4708) as many previous efforts have done. The results presented here have the potential to help understand and improve calibration uncertainties upcoming SN Ia cosmological analyses.
Exploration of Optimized Semantic Segmentation Architectures for edge-Deployment on Drones

In this paper, we present an analysis on the impact of network parameters for semantic segmentation architectures in context of UAV data processing. We present the analysis on the DroneDeploy Segmentation benchmark. Based on the comparative analysis we identify the optimal network architecture to be FPN-EfficientNetB3 with pretrained encoder backbones based on Imagenet Dataset. The network achieves IoU score of 0.65 and F1-score of 0.71 over the validation dataset. We also compare the various architectures in terms of their memory footprint and inference latency with further exploration of the impact of TensorRT based optimizations. We achieve memory savings of ~4.1x and latency improvement of 10% compared to Model: FPN and Backbone: InceptionResnetV2.
GCN for HIN via Implicit Utilization of Attention and Meta-Paths

Heterogeneous information network (HIN) embedding, aiming to map the structure and semantic information in a HIN to distributed representations, has drawn considerable research attention. Graph neural networks for HIN embeddings typically adopt a hierarchical attention (including node-level and meta-path-level attentions) to capture the information from meta-path-based neighbors. However, this complicated attention structure often cannot achieve the function of selecting meta-paths due to severe overfitting. Moreover, when propagating information, these methods do not distinguish direct (one-hop) meta-paths from indirect (multi-hop) ones. But from the perspective of network science, direct relationships are often believed to be more essential, which can only be used to model direct information propagation. To address these limitations, we propose a novel neural network method via implicitly utilizing attention and meta-paths, which can relieve the severe overfitting brought by the current over-parameterized attention mechanisms on HIN. We first use the multi-layer graph convolutional network (GCN) framework, which performs a discriminative aggregation at each layer, along with stacking the information propagation of direct linked meta-paths layer-by-layer, realizing the function of attentions for selecting meta-paths in an indirect way. We then give an effective relaxation and improvement via introducing a new propagation operation which can be separated from aggregation. That is, we first model the whole propagation process with well-defined probabilistic diffusion dynamics, and then introduce a random graph-based constraint which allows it to reduce noise with the increase of layers. Extensive experiments demonstrate the superiority of the new approach over state-of-the-art methods.
GROVER: Self-supervised Message Passing Transformer on Large-scale Molecular Data

How to obtain informative representations of molecules is a crucial prerequisite in AI-driven drug design and discovery. Recent researches abstract molecules as graphs and employ Graph Neural Networks (GNNs) for task-specific and data-driven molecular representation learning. Nevertheless, two "dark clouds" impede the usage of GNNs in real scenarios: (1) insufficient labeled molecules for supervised training; (2) poor generalization capabilities to new-synthesized molecules. To address them both, we propose a novel molecular representation framework, GROVER, which stands for Graph Representation frOm self-superVised mEssage passing tRansformer. With carefully designed self-supervised tasks in node, edge and graph-level, GROVER can learn rich structural and semantic information of molecules from enormous unlabelled molecular data. Rather, to encode such complex information, GROVER integrates Message Passing Networks with the Transformer-style architecture to deliver a class of more expressive encoders of molecules. The flexibility of GROVER allows it to be trained efficiently on large-scale molecular dataset without requiring any supervision, thus being immunized to the two issues mentioned above. We pre-train GROVER with 100 million parameters on 10 million unlabelled molecules---the biggest GNN and the largest training dataset that we have ever met. We then leverage the pre-trained GROVER to downstream molecular property prediction tasks followed by task-specific fine-tuning, where we observe a huge improvement (more than 6% on average) over current state-of-the-art methods on 11 challenging benchmarks. The insights we gained are that well-designed self-supervision losses and largely-expressive pre-trained models enjoy the significant potential on performance boosting.
Jump Operator Planning: Goal-Conditioned Policy Ensembles and Zero-Shot Transfer

In Hierarchical Control, compositionality, abstraction, and task-transfer are crucial for designing versatile algorithms which can solve a variety of problems with maximal representational reuse. We propose a novel hierarchical and compositional framework called Jump-Operator Dynamic Programming for quickly computing solutions within a super-exponential space of sequential sub-goal tasks with ordering constraints, while also providing a fast linearly-solvable algorithm as an implementation. This approach involves controlling over an ensemble of reusable goal-conditioned polices functioning as temporally extended actions, and utilizes transition operators called feasibility functions, which are used to summarize initial-to-final state dynamics of the polices. Consequently, the added complexity of grounding a high-level task space onto a larger ambient state-space can be mitigated by optimizing in a lower-dimensional subspace defined by the grounding, substantially improving the scalability of the algorithm while effecting transferable solutions. We then identify classes of objective functions on this subspace whose solutions are invariant to the grounding, resulting in optimal zero-shot transfer.
CareCall: a Call-Based Active Monitoring Dialog Agent for Managing COVID-19 Pandemic

Tracking suspected cases of COVID-19 is crucial to suppressing the spread of COVID-19 pandemic. Active monitoring and proactive inspection are indispensable to mitigate COVID-19 spread, though these require considerable social and economic expense. To address this issue, we introduce CareCall, a call-based dialog agent which is deployed for active monitoring in Korea and Japan. We describe our system with a case study with statistics to show how the system works. Finally, we discuss a simple idea which uses CareCall to support proactive inspection.
Stable Higgs mode in anisotropic quantum magnets

Low-energy excitations associated with the amplitude fluctuation of an order parameter in condensed matter systems can mimic the Higgs boson, an elementary particle in the standard model, and are dubbed as Higgs modes. Identifying the condensed-matter Higgs mode is challenging because it is known in many cases to decay rapidly into other low-energy bosonic modes, which renders the Higgs mode invisible. Therefore, it is desirable to find a way to stabilize the Higgs mode, which can offer an insight into the stabilization mechanism of the Higgs mode in condensed matter physics. In quantum magnets, magnetic order caused by spontaneous symmetry breaking supports transverse (magnons) and longitudinal (Higgs modes) fluctuations. When a continuous symmetry is broken, the Goldstone magnon mode generally has a lower excitation energy than the Higgs mode, causing a rapid decay of the latter. In this work, we show that a stable Higgs mode exists in anisotropic quantum magnets near the quantum critical point between the dimerized and magnetically ordered phases. We find that an easy axis anisotropy increases the magnon gap such that the magnon mode is above the Higgs mode near the quantum critical point, and the decay of the Higgs mode into the magnon mode is forbidden kinematically. Our results suggest that the anisotropic quantum magnets provide ideal platforms to explore the Higgs physics in condensed matter systems.
The Rice Brown Planthopper: Feeding Physiology and Host Plant Interactions

Rice, Oryza sativa L. , is an ancient cereal crop domesticated about 10,000 years ago along the hilly tracts south of the Himalayas (8). Because of the long history of its cultivation and its widespread dispersal, rice is probably the most genetically diversified among the world's cereal crops. In all Asian countries, rice is by far the most important staple food crop, and approxi­ mately half of the people on earth obtain the majority of their caloric intake from rice. Among the rice herbivorous insects, the brown planthopper, Nilaparvata lugens (Stal) (hereafter BPH), is one of the most notorious pests. It causes economic damage to the rice crop directly by feeding ( 116) and also indi­ rectly by transmitting grassy stunt (8 1) and ragged stunt virus diseases (54). The BPH is a small (2-3.5 mm in body length), brownish, sucking insect, belonging to the suborder Homoptera, series Auchenorrhyncha, super­ family Fulgoridae, and family Delphacidae. Members of the genus Nilapar­ vata are characterized by several lateral spines on the hind basi-tarsus (69). The adult shows density-dependent wing dimorphism with macropterous and brachypterous forms (20). Macropterous adults have the ability to migrate over long distances. The BPH is endemic to the tropical oriental region, but its habitat can temporarily expand as far north as Japan and Korea every summer through long-distance massive migrations from the tropics (46). The BPH is a monophagous insect restricted to cultivated rice and its allied wild forms such as Oryza perennis and O. spontanea (45). Thus, the seasonal occurrence and relative abundance of the BPH depend largely upon cropping patterns and intensification of rice cultivation.
Matrix Riccati Equation Solution of the 1D Radiative Transfer Equation

Abstract In recent years, the first author has developed three successful numerical methods to solve the 1D radiative transport equation yielding highly precise benchmarks. The second author has shown a keen interest in novel solution methodologies and an ability for their implementation. Here, we combine talents to generate yet another high precision solution, the Matrix Riccati Equation Method (MREM). MREM features the solution to two of the four matrix Riccati ODEs that arise from the interaction principle of particle transport. Through interaction coefficients, the interaction principle describes how particles reflect from- and transmit through- a single slab. On combination with Taylor series and doubling, a high-quality numerical benchmark, to nearly seven places, is established.
WoodFisher: Efficient Second-Order Approximation for Neural Network Compression

Second-order information, in the form of Hessian- or Inverse-Hessian-vector products, is a fundamental tool for solving optimization problems. Recently, there has been significant interest in utilizing this information in the context of deep neural networks; however, relatively little is known about the quality of existing approximations in this context. Our work examines this question, identifies issues with existing approaches, and proposes a method called WoodFisher to compute a faithful and efficient estimate of the inverse Hessian. Our main application is to neural network compression, where we build on the classic Optimal Brain Damage/Surgeon framework. We demonstrate that WoodFisher significantly outperforms popular state-of-the-art methods for one-shot pruning. Further, even when iterative, gradual pruning is considered, our method results in a gain in test accuracy over the state-of-the-art approaches, for pruning popular neural networks (like ResNet-50, MobileNetV1) trained on standard image classification datasets such as ImageNet ILSVRC. We examine how our method can be extended to take into account first-order information, as well as illustrate its ability to automatically set layer-wise pruning thresholds and perform compression in the limited-data regime.