Miracle-dz/UniFAR-FTUs huggingface dataset:
Adversarial Training for Machine Reading Comprehension with Virtual Embeddings

Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks. Though there are successful applications of AT on some NLP tasks, the distinguishing characteristics of NLP tasks have not been exploited. In this paper, we aim to apply AT on machine reading comprehension (MRC) tasks. Furthermore, we adapt AT for MRC tasks by proposing a novel adversarial training method called PQAT that perturbs the embedding matrix instead of word vectors. To differentiate the roles of passages and questions, PQAT uses additional virtual P/Q-embedding matrices to gather the global perturbations of words from passages and questions separately. We test the method on a wide range of MRC tasks, including span-based extractive RC and multiple-choice RC. The results show that adversarial training is effective universally, and PQAT further improves the performance.
WISDOM Project – IX. Giant molecular clouds in the lenticular galaxy NGC 4429: effects of shear and tidal forces on clouds

We present high spatial resolution (≈ 12 pc) Atacama Large Millimeter/sub-millimeter Array 12CO(J = 3 − 2) observations of the nearby lenticular galaxy NGC4429. We identify 217 giant molecular clouds within the 450 pc radius molecular gas disc. The clouds generally have smaller sizes and masses but higher surface densities and observed linewidths than those of Milky Way disc clouds. An unusually steep size – line width relation (σ ∝ R0.8 c ) and large cloud internal velocity gradients (0.05 – 0.91 km s−1 pc−1) and observed Virial parameters (〈αobs,vir〉 ≈ 4.0) are found, that appear due to internal rotation driven by the background galactic gravitational potential. Removing this rotation, an internal Virial equilibrium appears to be established between the self-gravitational (Usg) and turbulent kinetic (Eturb) energies of each cloud, i.e. 〈αsg,vir ≡ 2Eturb |Usg | 〉 ≈ 1.3. However, to properly account for both self and external gravity (shear and tidal forces), we formulate a modified Virial theorem and define an effective Virial parameter αeff,vir ≡ αsg,vir + Eext |Usg | (and associated effective velocity dispersion). The NGC4429 clouds then appear to be in a critical state in which the self-gravitational energy and the contribution of external gravity to the cloud’s energy budget (Eext) are approximately equal, i.e. Eext |Usg | ≈ 1. As such, 〈αeff,vir〉 ≈ 2.2 and most clouds are not virialised but remain marginally gravitationally bound. We show this is consistent with the clouds having sizes similar to their tidal radii and being generally radially elongated. External gravity is thus as important as self-gravity to regulate the clouds of NGC4429.
Towards Practical Credit Assignment for Deep Reinforcement Learning

Credit assignment is a fundamental problem in reinforcement learning, the problem of measuring an action's influence on future rewards. Explicit credit assignment methods have the potential to boost the performance of RL algorithms on many tasks, but thus far remain impractical for general use. Recently, a family of methods called Hindsight Credit Assignment (HCA) was proposed, which explicitly assign credit to actions in hindsight based on the probability of the action having led to an observed outcome. This approach has appealing properties, but remains a largely theoretical idea applicable to a limited set of tabular RL tasks. Moreover, it is unclear how to extend HCA to deep RL environments. In this work, we explore the use of HCA-style credit in a deep RL context. We first describe the limitations of existing HCA algorithms in deep RL that lead to their poor performance or complete lack of training, then propose several theoretically-justified modifications to overcome them. We explore the quantitative and qualitative effects of the resulting algorithm on the Arcade Learning Environment (ALE) benchmark, and observe that it improves performance over Advantage Actor-Critic (A2C) on many games where non-trivial credit assignment is necessary to achieve high scores and where hindsight probabilities can be accurately estimated.
XIRL: Cross-embodiment Inverse Reinforcement Learning

We investigate the visual cross-embodiment imitation setting, in which agents learn policies from videos of other agents (such as humans) demonstrating the same task, but with stark differences in their embodiments -- shape, actions, end-effector dynamics, etc. In this work, we demonstrate that it is possible to automatically discover and learn vision-based reward functions from cross-embodiment demonstration videos that are robust to these differences. Specifically, we present a self-supervised method for Cross-embodiment Inverse Reinforcement Learning (XIRL) that leverages temporal cycle-consistency constraints to learn deep visual embeddings that capture task progression from offline videos of demonstrations across multiple expert agents, each performing the same task differently due to embodiment differences. Prior to our work, producing rewards from self-supervised embeddings typically required alignment with a reference trajectory, which may be difficult to acquire under stark embodiment differences. We show empirically that if the embeddings are aware of task progress, simply taking the negative distance between the current state and goal state in the learned embedding space is useful as a reward for training policies with reinforcement learning. We find our learned reward function not only works for embodiments seen during training, but also generalizes to entirely new embodiments. Additionally, when transferring real-world human demonstrations to a simulated robot, we find that XIRL is more sample efficient than current best methods. Qualitative results, code, and datasets are available at https://x-irl.github. io
Государственный научный центр вирусологии и биотехнологии «Вектор» (Новосибирская область, р.п. Кольцово) Новосибирский национальный исследовательский государственный университет (Новосибирск) * Институт систематики и экологии животных СО РАН (Новосибирск)

Россия охватывает большую часть Евразии и представляет особый интерес для экологии вирусов гриппа и их эволюции. Важность этого региона была подтверждена вспышками птичьего гриппа H5N1, зафиксированными среди диких птиц с 2005 по 2010 годы. В статье содержится подробный сравнительный анализ вирусологических, молекулярно-биологических и патогенетических характеристик вирусов H5N1-субтипа, изолированных в России. Это исследование свидетельствует о необходимости постоянного наблюдения для обнаружения новых вариантов вирусов гриппа, чтобы способствовать предотвращению новых вспышек.
On Generating Lagrangian Cuts for Two-Stage Stochastic Integer Programs

We investigate new methods for generating Lagrangian cuts to solve two-stage stochastic integer programs. Lagrangian cuts can be added to a Benders reformulation and are derived from solving single scenario integer programming subproblems identical to those used in the nonanticipative Lagrangian dual of a stochastic integer program. Although Lagrangian cuts have the potential to significantly strengthen the Benders relaxation, generating Lagrangian cuts can be computationally demanding. We investigate new techniques for generating Lagrangian cuts with the goal of obtaining methods that provide significant improvements to the Benders relaxation quickly. Computational results demonstrate that our proposed method improves the Benders relaxation significantly faster than previous methods for generating Lagrangian cuts and, when used within a branch-and-cut algorithm, significantly reduces the size of the search tree for three classes of test problems.
Emergent fractal phase in energy stratified random models

We study the effects of partial correlations in kinetic hopping terms of long-range disordered random matrix models on their localization properties. We consider a set of models interpolating between fully-localized Richardson’s model and the celebrated Rosenzweig-Porter model (with implemented translation-invariant symmetry). In order to do this, we propose the energy-stratified spectral structure of the hopping term allowing one to decrease the range of correlations gradually. We show both analytically and numerically that any deviation from the completely correlated case leads to the emergent non-ergodic delocalization in the system unlike the predictions of localization of cooperative shielding. In order to describe the models with correlated kinetic terms, we develop the generalization of the Dyson Brownian motion and cavity approaches basing on stochastic matrix process with independent rank-one matrix increments and examine its applicability to the above set of models.
Cyberbullying Detection Using Deep Neural Network from Social Media Comments in Bangla Language

Cyberbullying or Online harassment detection on social media for various major languages is currently being given a good amount of focus by researchers worldwide. Being the seventh most speaking language in the world and increasing usage of online platform among the Bengali speaking people urge to find effective detection technique to handle the online harassment. In this paper, we have proposed binary and multiclass classification model using hybrid neural network for bully expression detection in Bengali language. We have used 44,001 users comments from popular public Facebook pages, which fall into five classes - Non-bully, Sexual, Threat, Troll and Religious. We have examined the performance of our proposed models from different perspective. Our binary classification model gives 87.91% accuracy, whereas introducing ensemble technique after neural network for multiclass classification, we got 85% accuracy.
Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models

We consider the problem of learning a tree-structured Ising model from data, such that subsequent predictions computed using the model are accurate. Con-cretely, we aim to learn a model such that posteriors $p$ (Xi| X s) for small sets of variables $S$ are accurate. Since its introduction more than 50 years ago, the Chow-Liu algorithm, which efficiently computes the maximum likelihood tree, has been the benchmark algorithm for learning tree-structured graphical models. A bound on the sample complexity of the Chow-Liu algorithm with respect to the prediction-centric local total variation loss was shown in [7]. While those results demonstrated that it is possible to learn a useful model even when recovering the true underlying graph is impossible, their bound depends on the maximum strength of interactions and thus does not achieve the information-theoretic optimum. In this paper, we introduce a new algorithm that carefully combines elements of the Chow-Liu algorithm with tree metric reconstruction methods to efficiently and optimally learn tree Ising models under a prediction-centric loss. Our algorithm is robust to model misspecification and adver-sarial corruptions. In contrast, we show that the celebrated Chow- Liu algorithm can be arbitrarily suboptimal.
$\ell_0$-based Sparse Canonical Correlation Analysis

Canonical Correlation Analysis (CCA) models are powerful for studying the associations between two sets of variables. The canonically correlated representations, termed \textit{canonical variates} are widely used in unsupervised learning to analyze unlabeled multi-modal registered datasets. Despite their success, CCA models may break (or overfit) if the number of variables in either of the modalities exceeds the number of samples. Moreover, often a significant fraction of the variables measures modality-specific information, and thus removing them is beneficial for identifying the \textit{canonically correlated variates}. Here, we propose $\ell_0$-CCA, a method for learning correlated representations based on sparse subsets of variables from two observed modalities. Sparsity is obtained by multiplying the input variables by stochastic gates, whose parameters are learned together with the CCA weights via an $\ell_0$-regularized correlation loss. We further propose $\ell_0$-Deep CCA for solving the problem of non-linear sparse CCA by modeling the correlated representations using deep nets. We demonstrate the efficacy of the method using several synthetic and real examples. Most notably, by gating nuisance input variables, our approach improves the extracted representations compared to other linear, non-linear and sparse CCA-based models.
Speedy Performance Estimation for Neural Architecture Search

Reliable yet efficient evaluation of generalisation performance of a proposed architecture is crucial to the success of neural architecture search (NAS). Traditional approaches face a variety of limitations: training each architecture to completion is prohibitively expensive, early stopped validation accuracy may correlate poorly with fully trained performance, and model-based estimators require large training sets. We instead propose to estimate the final test performance based on a simple measure of training speed. Our estimator is theoretically motivated by the connection between generalisation and training speed, and is also inspired by the reformulation of a PAC-Bayes bound under the Bayesian setting. Our model-free estimator is simple, efficient, and cheap to implement, and does not require hyperparameter-tuning or surrogate training before deployment. We demonstrate on various NAS search spaces that our estimator consistently outperforms other alternatives in achieving better correlation with the true test performance rankings. We further show that our estimator can be easily incorporated into both query-based and one-shot NAS methods to improve the speed or quality of the search.
Explainable AI and Adoption of Financial Algorithmic Advisors: An Experimental Study

We study whether receiving advice from either a human or algorithmic advisor, accompanied by five types of Local and Global explanation labelings, has an effect on the readiness to adopt, willingness to pay, and trust in a financial AI consultant. We compare the differences over time and in various key situations using a unique experimental framework where participants play a web-based game with real monetary consequences. We observed that accuracy-based explanations of the model in initial phases leads to higher adoption rates. When the performance of the model is immaculate, there is less importance associated with the kind of explanation for adoption. Using more elaborate feature-based or accuracy-based explanations helps substantially in reducing the adoption drop upon model failure. Furthermore, using an autopilot increases adoption significantly. Participants assigned to the AI-labeled advice with explanations were willing to pay more for the advice than the AI-labeled advice with "No-explanation" alternative. These results add to the literature on the importance of XAI for algorithmic adoption and trust.
Hyperbolic Temporal Knowledge Graph Embeddings with Relational and Time Curvatures

Knowledge Graph (KG) completion has been excessively studied with a massive number of models proposed for the Link Prediction (LP) task. The main limitation of such models is their insensitivity to time. Indeed, the temporal aspect of stored facts is often ignored. To this end, more and more works consider time as a parameter to complete KGs. In this paper, we first demonstrate that, by simply increasing the number of negative samples, the recent AttH model can achieve competitive or even better performance than the state-of-the-art on Temporal KGs (TKGs), albeit its nontemporality. We further propose Hercules, a time-aware extension of AttH model, which defines the curvature of a Riemannian manifold as the product of both relation and time. Our experiments show that both Hercules and AttH achieve competitive or new state-of-the-art performances on ICEWS04 and ICEWS05-15 datasets. Therefore, one should raise awareness when learning TKGs representations to identify whether time truly boosts performances.
HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation

User interest modeling is critical for personalized news recommendation. Existing news recommendation methods usually learn a single user embedding for each user from their previous behaviors to represent their overall interest. However, user interest is usually diverse and multi-grained, which is difficult to be accurately modeled by a single user embedding. In this paper, we propose a news recommendation method with hierarchical user interest modeling, named HieRec. Instead of a single user embedding, in our method each user is represented in a hierarchical interest tree to better capture their diverse and multi-grained interest in news. We use a three-level hierarchy to represent 1) overall user interest; 2) user interest in coarse-grained topics like sports; and 3) user interest in fine-grained topics like football. Moreover, we propose a hierarchical user interest matching framework to match candidate news with different levels of user interest for more accurate user interest targeting. Extensive experiments on two real-world datasets validate our method can effectively improve the performance of user modeling for personalized news recommendation.
Personalized PercepNet: Real-Time, Low-Complexity Target Voice Separation and Enhancement

The presence of multiple talkers in the surrounding environment poses a difficult challenge for real-time speech communication systems considering the constraints on network size and complexity. In this paper, we present Personalized PercepNet, a real-time speech enhancement model that separates a target speaker from a noisy multi-talker mixture without compromising on complexity of the recently proposed PercepNet. To enable speaker-dependent speech enhancement, we first show how we can train a perceptually motivated speaker embedder network to produce a representative embedding vector for the given speaker. Personalized PercepNet uses the target speaker embedding as additional information to pick out and enhance only the target speaker while suppressing all other competing sounds. Our experiments show that the proposed model significantly outperforms PercepNet and other baselines, both in terms of objective speech enhancement metrics and human opinion scores.
Joint Inter-Path and Intra-Path Multiplexing for Terahertz Widely-Spaced Multi-Subarray Hybrid Beamforming Systems

Terahertz (THz) communications with multi-GHz bandwidth are envisioned as a key technology for 6G systems. Ultra-massive (UM) MIMO with hybrid beamforming architectures are widely investigated to provide a high array gain to overcome the huge propagation loss at THz band. However, most of the existing hybrid beamforming architectures can only utilize the multiplexing offered by the multipath components, i.e., inter-path multiplexing, which is very limited due to the spatially sparse THz channel. In this paper, a widely-spaced multi-subarray (WSMS) hybrid beamforming architecture is proposed, which improves the multiplexing gain by exploiting a new type of intra-path multiplexing provided by the spherical-wave propagation among $k$ widely-spaced subarrays, in addition to the inter-path multiplexing. The resulting multiplexing gain of WSMS architecture is $k$ times of the existing architectures. For WSMS architecture, a novel design problem is formulated by optimizing the number of subarrays, subarray spacing, and hybrid beamforming matrices to maximize the spectral efficiency, which is decomposed into two subproblems. An optimal closed-form solution is derived for the first hybrid beamforming subproblem, while a dominant-line-of-sight-relaxation algorithm is proposed for the second array configuration subproblem. Extensive simulation results demonstrate that the WSMS architecture and proposed algorithms enhance the spectral efficiency substantially.
Acoustic Power for Swarms of Microscopic Robots

Ultrasound can power implanted medical devices. This paper evaluates its feasibility for microscopic robots in tissue that mechanically harvest energy using pistons. At these sizes, viscous drag dominates the piston motion and acoustic attenuation limits how far power can reach. Combining these factors shows that frequencies around 100kHz can deliver hundreds of picowatts to well-separated micron-size robots in low-attenuation tissues within about 10cm of the skin. However, applications of microscopic robots could involve large numbers, in which case the robots themselves significantly increase acoustic attenuation. Robots can mitigate this attenuation using cooperative swarm behaviors, with trade-offs among individual power, group performance and the complexity of the robot controllers. With such mitigating behaviors, acoustic power can be useful for swarms of a few hundred billion robots in the body, that each use tens of picowatts, on average, and can tolerate significant variability in available power, e.g, as robots in the bloodstream move from near the skin to deep within the body, or from lowto high-attenuation tissue such as the lungs.
Incentive Mechanism for Privacy-Preserving Federated Learning

Federated learning (FL) is an emerging paradigm for machine learning, in which data owners can collaboratively train a model by sharing gradients instead of their raw data. Two fundamental research problems in FL are incentive mechanism and privacy protection . The former focuses on how to incentivize data owners to participate in FL. The latter studies how to protect data owners’ privacy while maintaining high utility of trained models. However, incentive mechanism and privacy protection in FL have been studied separately and no work solves both problems at the same time. In this work, we address the two problems simultaneously by an FL-Market that incentivizes data owners’ participation by providing appropriate payments and privacy protection. FL-Market enables data owners to obtain compensation according to their privacy loss quantified by local differential privacy (LDP). Our insight is that, by meeting data owners’ personalized privacy preferences and providing appropriate payments, we can (1) incentivize privacy risk-tolerant data owners to set larger privacy parameters (i.e., gradients with less noise) and (2) provide preferred privacy protection for privacy risk-averse data owners. To achieve this, we design a personalized LDP-based FL framework with a deep learning-empowered auction mechanism for incentivizing trading gradients with less noise and optimal aggregation mechanisms for model updates. Our experiments verify the effectiveness of the proposed framework and mechanisms.
Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering

Disfluencies is an under-studied topic in NLP, even though it is ubiquitous in human conversation. This is largely due to the lack of datasets containing disfluencies. In this paper, we present a new challenge question answering dataset, Disfl-QA, a derivative of SQuAD, where humans introduce contextual disfluencies in previously fluent questions. Disfl-QA contains a variety of challenging disfluencies that require a more comprehensive understanding of the text than what was necessary in prior datasets. Experiments show that the performance of existing state-of-the-art question answering models degrades significantly when tested on Disfl-QA in a zero-shot setting.We show data augmentation methods partially recover the loss in performance and also demonstrate the efficacy of using gold data for fine-tuning. We argue that we need large-scale disfluency datasets in order for NLP models to be robust to them. The dataset is publicly available at: https://github.com/google-research-datasets/disfl-qa.