Miracle-dz/UniFAR-FTUs huggingface dataset:
Distilling Relation Embeddings from Pretrained Language Models

Pre-trained language models have been found to capture a surprisingly rich amount of lexical knowledge, ranging from commonsense properties of everyday concepts to detailed factual knowledge about named entities. Among others, this makes it possible to distill high-quality word vectors from pre-trained language models. However, it is currently unclear to what extent it is possible to distill relation embeddings, i.e. vectors that characterize the relationship between two words. Such relation embeddings are appealing because they can, in principle, encode relational knowledge in a more fine-grained way than is possible with knowledge graphs. To obtain relation embeddings from a pre-trained language model, we encode word pairs using a (manually or automatically generated) prompt, and we fine-tune the language model such that relationally similar word pairs yield similar output vectors. We find that the resulting relation embeddings are highly competitive on analogy (unsupervised) and relation classification (supervised) benchmarks, even without any task-specific fine-tuning. Source code to reproduce our experimental results and the model checkpoints are available in the following repository: https://github.com/asahi417/relbert
GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL

Out-of-distribution (OOD) detection is a well-studied topic in supervised learning. Extending the successes in supervised learning methods to the reinforcement learning (RL) setting, however, is difficult due to the data generating process - RL agents actively query their environment for data, and the data are a function of the policy followed by the agent. An agent could thus neglect a shift in the environment if its policy did not lead it to explore the aspect of the environment that shifted. Therefore, to achieve safe and robust generalization in RL, there exists an unmet need for OOD detection through active experimentation. Here, we attempt to bridge this lacuna by first defining a causal framework for OOD scenarios or environments encountered by RL agents in the wild. Then, we propose a novel task: that of Out-of-Task Distribution (OOTD) detection. We introduce an RL agent that actively experiments in a test environment and subsequently concludes whether it is OOTD or not. We name our method GalilAI, in honor of Galileo Galilei, as it discovers, among other causal processes, that gravitational acceleration is independent of the mass of a body. Finally, we propose a simple probabilistic neural network baseline for comparison, which extends extant Model-Based RL. We find that GalilAI outperforms the baseline significantly. See visualizations of our method https://galil-ai.github. io/
Two-sided fairness in rankings via Lorenz dominance

We consider the problem of generating rankings that are fair towards both users and item producers in recommender systems. We address both usual recommendation (e.g., of music or movies) and reciprocal recommendation (e.g., dating). Following concepts of distributive justice in welfare economics, our notion of fairness aims at increasing the utility of the worse-off individuals, which we formalize using the criterion of Lorenz efficiency. It guarantees that rankings are Pareto efficient, and that they maximally redistribute utility from better-off to worse-off, at a given level of overall utility. We propose to generate rankings by maximizing concave welfare functions, and develop an efficient inference procedure based on the Frank-Wolfe algorithm. We prove that unlike existing approaches based on fairness constraints, our approach always produces fair rankings. Our experiments also show that it increases the utility of the worse-off at lower costs in terms of overall utility.
Hyperparameter Tuning is All You Need for LISTA

Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unrolling an iterative algorithm and training it like a neural network. It has had great success on sparse recovery. In this paper, we show that adding momentum to intermediate variables in the LISTA network achieves a better convergence rate and, in particular, the network with instance-optimal parameters is superlinearly convergent. Moreover, our new theoretical results lead to a practical approach of automatically and adaptively calculating the parameters of a LISTA network layer based on its previous layers. Perhaps most surprisingly, such an adaptive-parameter procedure reduces the training of LISTA to tuning only three hyperparameters from data: a new record set in the context of the recent advances on trimming down LISTA complexity. We call this new ultra-light weight network HyperLISTA. Compared to state-of-the-art LISTA models, HyperLISTA achieves almost the same performance on seen data distributions and performs better when tested on unseen distributions (specifically, those with different sparsity levels and nonzero magnitudes). Code is available: https://github.com/VITA-Group/HyperLISTA.
Learning physics confers pose-sensitivity in structure-based virtual screening

In drug discovery, structure-based virtual high-throughput screening (vHTS) campaigns aim to identify bioactive ligands or"hits"for therapeutic protein targets from docked poses at specific binding sites. However, while generally successful at this task, many deep learning methods are known to be insensitive to protein-ligand interactions, decreasing the reliability of hit detection and hindering discovery at novel binding sites. Here, we overcome this limitation by introducing a class of models with two key features: 1) we condition bioactivity on pose quality score, and 2) we present poor poses of true binders to the model as negative examples. The conditioning forces the model to learn details of physical interactions. We evaluate these models on a new benchmark designed to detect pose-sensitivity.
Landscape analysis of an improved power method for tensor decomposition

In this work, we consider the optimization formulation for symmetric tensor decomposition recently introduced in the Subspace Power Method (SPM) of Kileel and Pereira. Unlike popular alternative functionals for tensor decomposition, the SPM objective function has the desirable properties that its maximal value is known in advance, and its global optima are exactly the rank-1 components of the tensor when the input is sufficiently low-rank. We analyze the non-convex optimization landscape associated with the SPM objective. Our analysis accounts for working with noisy tensors. We derive quantitative bounds such that any second-order critical point with SPM objective value exceeding the bound must equal a tensor component in the noiseless case, and must approximate a tensor component in the noisy case. For decomposing tensors of size $D^{\times m}$, we obtain a near-global guarantee up to rank $\widetilde{o}(D^{\lfloor m/2 \rfloor})$ under a random tensor model, and a global guarantee up to rank $\mathcal{O}(D)$ assuming deterministic frame conditions. This implies that SPM with suitable initialization is a provable, efficient, robust algorithm for low-rank symmetric tensor decomposition. We conclude with numerics that show a practical preferability for using the SPM functional over a more established counterpart.
Stochastic growth of a stable phase from a metastable phase

When a stable phase is adjacent to a metastable phase with a planar interface, the stable phase grows. We propose a stochastic model describing the phase growth accompanying latent heat. The model is based on an energy-conserving Potts model with a kinetic energy term defined on a sparsely random lattice only in one direction. For this model, we calculate the stable and metastable phases exactly using statistical mechanics. Performing numerical simulations, we find that the growth law is qualitatively different from that of the deterministic phase-field model.
Application of 2-D Convolutional Neural Networks for Damage Detection in Steel Frame Structures

Extracting damage-sensitive features that are insensitive to noise from time-series data was a significant challenge in traditional machine learning-based structural damage detection (SDD) methods. Moreover, this procedure slowed down the techniques and made the methods’ accuracy dependent on preprocessing. By growing deep learning (DL), several studies were conducted to solve this limit using deep architectures in the field of structural health monitoring (SHM). However, these methods mostly require substantial measurements for the training procedure and are expensive to process, making them unsuitable for real-time approaches. Thus, we present an application of 2-D convolutional neural networks (2-D CNNs) designed to perform both feature extraction and classification stages as a single organism to solve the highlighted problems. The method uses a network of lighted CNNs instead of deep and takes raw acceleration signals as input. Using lighted CNNs, in which every one of them is optimized for a specific element, increases the accuracy and makes the network faster to perform. Also, a new framework is proposed for decreasing the data required in the training phase. We verified our method on Qatar University Grandstand Simulator (QUGS) benchmark data provided by Structural Dynamics Team. The results showed improved accuracy over other methods, and running time was adequate for realtime applications.
Improving Fairness via Federated Learning

Recently, lots of algorithms have been proposed for learning a fair classifier from decentralized data. However, many theoretical and algorithmic questions remain open. First, is federated learning necessary, i.e., can we simply train locally fair classifiers and aggregate them? In this work, we first propose a new theoretical framework, with which we demonstrate that federated learning can strictly boost model fairness compared with such non-federated algorithms. We then theoretically and empirically show that the performance tradeoff of FedAvg-based fair learning algorithms is strictly worse than that of a fair classifier trained on centralized data. To bridge this gap, we propose FedFB, a private fair learning algorithm on decentralized data. The key idea is to modify the FedAvg protocol so that it can effectively mimic the centralized fair learning. Our experimental results show that FedFB significantly outperforms existing approaches, sometimes matching the performance of the centrally trained model.
2 5 O ct 2 02 1 Schubert calculus via Grassmann variables

In [5], Imanishi, Jinzenji and Kuwata provided a recipe for computing Euler number of Grassmann manifold G(k,N). In the process, they showed that the cohomology ring of G(k,N) is represented by Grassmann variables. The purpose of this paper is to compute some integral of Chern classes of the dual bundle of the tautological bundle on G(k,N) by using only Grassmann variables. In other words, we compute the intersection number of Schubert cycles by using only Grassmann variables.
AWSOM-LP: An Effective Log Parsing Technique Using Pattern Recognition and Frequency Analysis

Logs provide users with useful insights to help with a variety of development and operations tasks. The problem is that logs are often unstructured, making their analysis a complex task. This is mainly due to the lack of guidelines and best practices for logging, combined with a large number of logging libraries at the disposal of software developers. There exist studies that aim to parse automatically large logs. The main objective is to extract templates from samples of log data that are used to recognize future logs. In this paper, we propose AWSOM-LP, a powerful log parsing and abstraction tool, which is highly accurate, stable, and efficient. AWSOM-LP is built on the idea of applying pattern recognition and frequency analysis. First, log events are organized into patterns using a simple text processing method. Frequency analysis is then applied locally to instances of the same group to identify static and dynamic content of log events. When applied to 16 log datasets of the the LogPai project, AWSOM-LP achieves an average grouping accuracy of 93.5%, which outperforms the accuracy of five leading log parsing tools namely, Logram, Lenma, Drain, IPLoM and AEL. Additionally, AWSOM-LP can generate more than 80% of the final log templates from 10% to 50% of the entire log dataset and can parse up to a million log events in an average time of 5 minutes. AWSOM-LP is available online as an open source. It can be used by practitioners and researchers to parse effectively and efficiently large log files so as to support log analysis tasks.
Merge of X-ETL and XCube towards a Standard Hybrid Method for Designing Data Warehouses

There is no doubt that the hybrid approach is the best paradigm for designing effective multidimensional schemas. Its strength lies in its ability to combine the top-down and bottom-up approaches, thus exploiting the advantages of both approaches. In this paper, the authors try to identify and analyze the different hybrid methods developed for building data warehouses. The analysis revealed that the existing methods are too complicated and time consuming in the deployment phase. In order to solve this problem, the authors introduced a new hybrid method that is easy to use and saves a huge amount of deployment time. This new method consists of two main steps: the first data driven step allows an analysis of the source models by using the X-ETL method and gives rise to star models. The second requirements driven step performs a semantic analysis of the needs expressed in natural language by using the XCube Assist method. This analysis allows to improve the quality of star models generated by the X-ETL method without the intervention of a designer.
Does Momentum Help? A Sample Complexity Analysis

Heavy ball momentum is a popular acceleration idea in stochastic optimization. There have been several attempts to understand its perceived benefits, but the complete picture is still unclear. Specifically, the error expression in the presence of noise has two separate terms: the bias and the variance, but most existing works only focus on bias and show that momentum accelerates its decay. Such analyses overlook the interplay between bias and variance and, therefore, miss important implications. In this work, we analyze a sample complexity bound of stochastic approximation algorithms with heavy-ball momentum that accounts for both bias and variance. We find that for the same step size, which is small enough, the iterates with momentum have improved sample complexity compared to the ones without. However, by using a different step-size sequence, the non-momentum version can nullify this benefit. Subsequently, we show that our sample complexity bounds are indeed tight for a small enough neighborhood around the solution and large enough noise variance. Our analysis also sheds some light on the finite-time behavior of these algorithms. This explains the perceived benefit in the initial phase of momentum-based schemes.
Quality control, data cleaning, imputation

This chapter addresses important steps during the quality assurance and control of RWD, with particular emphasis on the identification and handling of missing values. A gentle introduction is provided on common statistical and machine learning methods for imputation. We discuss the main strengths and weaknesses of each method, and compare their performance in a literature review. We motivate why the imputation of RWD may require additional efforts to avoid bias, and highlight recent advances that account for informative missingness and repeated observations. Finally, we introduce alternative methods to address incomplete data without the need for imputation. Section
Hybrid Adversarial Imitation Learning

—Extrapolating beyond-demonstrator (BD) performance through the imitation learning (IL) algorithm aims to learn from and outperform the demonstrator. Most existing BDIL algorithms are performed in two stages by ﬁrst inferring a reward function before learning a policy via reinforcement learning (RL). However, such two-stage BDIL algorithms suffer from high computational complexity, weak robustness, and large performance variations. In particular, a poor reward function derived in the ﬁrst stage will inevitably incur severe performance loss in the second stage. In this work, we propose a hybrid adversarial imitation learning (HAIL) algorithm that is one-stage, model-free, generative-adversarial (GA) fashion and curiosity-driven. Thanks to the one-stage design, the HAIL can integrate both the reward function learning and the policy optimization into one procedure, which leads to many advantages such as low computational complexity, high robustness, and strong adaptability. More speciﬁcally, HAIL simultaneously imitates the demonstrator and explores BD performance by utilizing hybrid rewards. Extensive simulation results conﬁrm that HAIL can achieve higher performance as compared to other similar BDIL algorithms.
Association of Myocardial Enzyme Abnormality with Clinical Outcomes of Patients with COVID-19: A Retrospective Study

Background It has been observed that COVID-19 may cause myocardial damage, but there are few detailed reports on myocardial enzyme abnormalities. Methods In this retrospective study, we analyzed data from 157 consecutive laboratory-confirmed and hospitalized COVID-19 patients from Wuhan. We collected information on demographic and clinical characteristics, laboratory findings, and clinical outcomes. Logistic regression analysis was used to explore the risk factors associated with the severity of COVID-19. The association between myocardial enzyme abnormalities and the mortality was also investigated. Results The mortality in abnormal myocardial enzyme group was obviously higher than the normal group (P < 0.001). The majority of patients (n = 72, 97.3%) with normal cardiac enzyme group were of the common novel coronavirus pneumonia (NCP) type, whereas half of the patients with cardiac enzyme abnormalities (n = 40, 48.2%) developed critical and severe NCP type. The multivariable logistic regression analysis indicated that COVID-19 patients with increasing age (P = 0.035), higher levels of CRP (P = 0.038), and TNI (P = 0.036) were associated with increased death than other patients. Conclusions Myocardial enzyme abnormality and myocardial injury were associated with the severity and fatal outcomes of COVID-19. Clinicians should pay attention to the markers of myocardial injury in COVID-19 patients, especially those with older age, comorbidities, and inflammation.
Lateral Fully Organic P–N Diodes Created in a Single Donor–Acceptor Copolymer

P–N junctions exist in many solid‐state organic devices, such as light‐emitting diodes, solar cells, and thermoelectric devices. Creating P–N junctions by bulk chemical doping in a single organic material (like silicon doped by boron and phosphorus) may capitalize the vast scientific and technological groundwork established in the inorganic semiconducting field. However, high‐performance single‐organic‐material P–N junctions are seldom reported, because the diffusion of the dopant counterions often leads to transient rectification properties. Herein, a new type of lateral fully organic diodes created in single donor–acceptor (D–A) copolymer films with only one P‐type dopant is reported. The achieved lateral devices exhibit high current densities of ≈3.83 A cm−2 and a high rectification ratio of ≈2100, which are beyond the requirements for high‐frequency identification tags. The P‐ to N‐type polarity switching mechanism is proposed after spectroscopic and structural tests. Decent stability of the organic diode is obtained, which is due to the long channel length and low diffusion speed of the large size of dopants. This work opens the opportunities to create P–N junctions in ways of silicon‐based inorganic semiconductors and promises new opportunities for integrating organic materials for flexible and printable organic devices.
Response to Referee # 1 General Comments : Comment 1 :

Respiration and leaching are two main processes responsible for soil carbon loss. While the former has received considerable research attention, studies examining leaching processes are limited especially in semiarid grasslands due to low precipitation. Climate change may increase the extreme precipitation event (EPE) frequency in arid and semiarid regions, 15 potentially enhancing soil carbon loss through leaching and respiration. Here we incubated soil columns of three typical grassland soils from Inner Mongolia and Qinghai-Tibetan Plateau and examined the effect of simulated EPEs on soil carbon loss through respiration and leaching. EPEs induced transient increase of CO2 release through soil respiration, equivalent to 32% and 72% of the net ecosystem productivity (NEP) in the temperate grasslands (Xilinhot and Keqi) and 7% in the alpine grasslands (Gangcha). By comparison, leaching loss of soil carbon accounted for 290%, 120% and 15% of NEP at the 20 corresponding sites, respectively, with dissolved inorganic carbon (DIC) as the main form of carbon loss in the alkaline soils regardless of DIC sources. Moreover, DIC loss increased with re-occuring EPEs in the soil with the highest pH due to elevated contribution of dissolved CO2 from organic carbon degradation (indicated by DIC-δC). These results highlight that leaching loss of soil carbon (particularly in the form of DIC) is important in the regional carbon budget of arid and semiarid grasslands, and also imply that SOC mineralization measured by CO2 fluxes might be drastically under-estimated in 25 alkaline soils. With a projected increase of EPEs under climate change, soil carbon leaching processes and its influencing factors warrant better understanding and should be incorporated into soil carbon models when estimating carbon balance in grassland ecosystems.
Voice modulation: from origin and mechanism to social impact

Research on within-individual modulation of vocal cues is surprisingly scarce outside of human speech. Yet, voice modulation serves diverse functions in human and nonhuman nonverbal communication, from dynamically signalling motivation and emotion, to exaggerating physical traits such as body size and masculinity, to enabling song and musicality. The diversity of anatomical, neural, cognitive and behavioural adaptations necessary for the production and perception of voice modulation make it a critical target for research on the origins and functions of acoustic communication. This diversity also implicates voice modulation in numerous disciplines and technological applications. In this two-part theme issue comprising 21 articles from leading and emerging international researchers, we highlight the multidisciplinary nature of the voice sciences. Every article addresses at least two, if not several, critical topics: (i) development and mechanisms driving vocal control and modulation; (ii) cultural and other environmental factors affecting voice modulation; (iii) evolutionary origins and adaptive functions of vocal control including cross-species comparisons; (iv) social functions and real-world consequences of voice modulation; and (v) state-of-the-art in multidisciplinary methodologies and technologies in voice modulation research. With this collection of works, we aim to facilitate cross-talk across disciplines to further stimulate the burgeoning field of voice modulation. This article is part of the theme issue ‘Voice modulation: from origin and mechanism to social impact (Part I)’.