Miracle-dz/UniFAR-FTUs huggingface dataset:
Suboptimality of Constrained Least Squares and Improvements via Non-Linear Predictors

We study the problem of predicting as well as the best linear predictor in a bounded Euclidean ball with respect to the squared loss. When only boundedness of the data generating distribution is assumed, we establish that the least squares estimator constrained to a bounded Euclidean ball does not attain the classical $O(d/n)$ excess risk rate, where $d$ is the dimension of the covariates and $n$ is the number of samples. In particular, we construct a bounded distribution such that the constrained least squares estimator incurs an excess risk of order $\Omega(d^{3/2}/n)$ hence refuting a recent conjecture of Ohad Shamir [JMLR 2015]. In contrast, we observe that non-linear predictors can achieve the optimal rate $O(d/n)$ with no assumptions on the distribution of the covariates. We discuss additional distributional assumptions sufficient to guarantee an $O(d/n)$ excess risk rate for the least squares estimator. Among them are certain moment equivalence assumptions often used in the robust statistics literature. While such assumptions are central in the analysis of unbounded and heavy-tailed settings, our work indicates that in some cases, they also rule out unfavorable bounded distributions.
DeepActsNet: Spatial and Motion features from Face, Hands, and Body Combined with Convolutional and Graph Networks for Improved Action Recognition

Existing action recognition methods mainly focus on joint and bone information in human body skeleton data due to its robustness to complex backgrounds and dynamic characteristics of the environments. In this paper, we combine body skeleton data with spatial and motion information from face and two hands, and present Deep Action Stamps (DeepActs), a novel data representation to encode actions from video sequences. We also present DeepActsNet, a deep learning based model with modality-specific Convolutional and Graph sub-networks for highly accurate action recognition based on Deep Action Stamps. Experiments on three challenging action recognition datasets (NTU60, NTU120, and SYSU) show that DeepActs produce considerable improvements in the recognition performance of standard convolutional and graph networks. Experiments also show that the fusion of modality-specific convolutional and structural features learnt by our DeepActsNet yields consistent improvements in action recognition accuracy over the state-of-the-art on the target datasets.
Assessing the Severity of Health States based on Social Media Posts

The unprecedented growth of Internet users has resulted in an abundance of unstructured information on social media including health forums, where patients request health-related information or opinions from other users. Previous studies have shown that online peer support has limited effectiveness without expert intervention. Therefore, a system capable of assessing the severity of health state from the patients' social media posts can help health professionals (HP) in prioritizing the user's post. In this study, we inspect the efficacy of different aspects of Natural Language Understanding (NLU) to identify the severity of the user's health state in relation to two perspectives(tasks) (a) Medical Condition (i.e., Recover, Exist, Deteriorate, Other) and (b) Medication (i.e., Effective, Ineffective, Serious Adverse Effect, Other) in online health communities. We propose a multiview learning framework that models both the textual content as well as contextual-information to assess the severity of the user's health state. Specifically, our model utilizes the NLU views such as sentiment, emotions, personality, and use of figurative language to extract the contextual information. The diverse NLU views demonstrate its effectiveness on both the tasks and as well as on the individual disease to assess a user's health.
VirtualFlow: Decoupling Deep Learning Model Execution from Underlying Hardware

State-of-the-art deep learning systems tightly couple model execution with the underlying hardware. This coupling poses important challenges in a world where the scale of deep learning workloads is growing rapidly: workloads with high resource requirements are inaccessible to most users, experimentation on smaller test beds is impossible, and results are difficult to reproduce across different hardware. We propose VirtualFlow, a novel system approach leveraging virtual node processing to decouple model execution from the hardware. In each execution step, the batch is divided and processed with data parallelism on many virtual nodes instead of physical devices (GPUs, TPUs), and the gradients are aggregated and applied to the model after all virtual nodes finish processing their data. With multiple virtual nodes mapped to each device, the system allows users to run models at much larger batch sizes that would otherwise exceed the memory limits of the underlying physical resources. VirtualFlow significantly improves model training reproducibility across different hardware, and enables models to run on shared clusters with dynamically changing resources for better efficiency. Our implementation of VirtualFlow enables virtual node processing with elasticity for TensorFlow. Evaluation with representative deep learning models (ResNet, BERT, Transformer) demonstrates strong convergence guarantees on different hardware with out-of-the-box hyperparameters, and up to 48% lower job completion times with resource elasticity.
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the embeddings. However, an implicit and untested assumption of their method is that the bias sub-space is actually linear. In this work, we generalize their method to a kernelized, non-linear version. We take inspiration from kernel principal component analysis and derive a non-linear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word embeddings and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).
Domain-Embeddings Based DGA Detection with Incremental Training Method

DGA-based botnet, which uses Domain Generation Algorithms (DGAs) to evade supervision, has become a part of the most destructive threats to network security. Over the past decades, a wealth of defense mechanisms focusing on domain features have emerged to address the problem. Nonetheless, DGA detection remains a daunting and challenging task due to the big data nature of Internet traffic and the potential fact that the linguistic features extracted only from the domain names are insufficient and the enemies could easily forge them to disturb detection. In this paper, we propose a novel DGA detection system which employs an incremental word-embeddings method to capture the interactions between end hosts and domains, characterize time-series patterns of DNS queries for each IP address and therefore explore temporal similarities between domains. We carefully modify the Word2Vec algorithm and leverage it to automatically learn dynamic and discriminative feature representations for over 1.9 million domains, and develop an simple classifier for distinguishing malicious domains from the benign. Given the ability to identify temporal patterns of domains and update models incrementally, the proposed scheme makes the progress towards adapting to the changing and evolving strategies of DGA domains. Our system is evaluated and compared with the state-of-art system FANCI and two deep-learning methods CNN and LSTM, with data from a large university’s network named TUNET. The results suggest that our system outperforms the strong competitors by a large margin on multiple metrics and meanwhile achieves a remarkable speed-up on model updating.
A Survey of Resource Management for Processing-in-Memory and Near-Memory Processing Architectures

Due to the amount of data involved in emerging deep learning and big data applications, operations related to data movement have quickly become a bottleneck. Data-centric computing (DCC), as enabled by processing-in-memory (PIM) and near-memory processing (NMP) paradigms, aims to accelerate these types of applications by moving the computation closer to the data. Over the past few years, researchers have proposed various memory architectures that enable DCC systems, such as logic layers in 3D-stacked memories or charge-sharing-based bitwise operations in dynamic random-access memory (DRAM). However, application-specific memory access patterns, power and thermal concerns, memory technology limitations, and inconsistent performance gains complicate the offloading of computation in DCC systems. Therefore, designing intelligent resource management techniques for computation offloading is vital for leveraging the potential offered by this new paradigm. In this article, we survey the major trends in managing PIM and NMP-based DCC systems and provide a review of the landscape of resource management techniques employed by system designers for such systems. Additionally, we discuss the future challenges and opportunities in DCC management.
Learning-Based Massive Beamforming

Developing resource allocation algorithms with strong real-time and high efficiency has been an imperative topic in wireless networks. Conventional optimization-based iterative resource allocation algorithms often suffer from slow convergence, especially for massive multiple-input-multiple-output (MIMO) beamforming problems. This paper studies learningbased efficient massive beamforming methods for multi-user MIMO networks. The considered massive beamforming problem is challenging in two aspects. First, the beamforming matrix to be learned is quite high-dimensional in case with a massive number of antennas. Second, the objective is often time-varying and the solution space is not fixed due to some communication requirements. All these challenges make learning representation for massive beamforming an extremely difficult task. In this paper, by exploiting the structure of the most popular WMMSE beamforming solution, we propose convolutional massive beamforming neural networks (CMBNN) using both supervised and unsupervised learning schemes with particular design of network structure and input/output. Numerical results demonstrate the efficacy of the proposed CMBNN in terms of running time and system throughput.
Multiple-trait Adaptive Fisher's Method for Genome-wide Association Studies

In genome-wide association studies (GWASs), there is an increasing need for detecting the associations between a genetic variant and multiple traits. In studies of complex diseases, it is common to measure several potentially correlated traits in a single GWAS. Despite the multivariate nature of the studies, single-trait-based methods remain the most widely-adopted analysis procedure, owing to their simplicity for studies with multiple traits as their outcome. However, the association between a genetic variant and a single trait sometimes can be weak, and ignoring the actual correlation among traits may lose power. On the contrary, multiple-trait analysis, a method analyzes a group of traits simultaneously, has been proven to be more powerful by incorporating information from the correlated traits. Although existing methods have been developed for multiple traits, several drawbacks limit their wide application in GWASs. In this paper, we propose a multiple-trait adaptive Fisher's (MTAF) method to test associations between a genetic variant and multiple traits at once, by adaptively aggregating evidence from each trait. The proposed method can accommodate both continuous and binary traits and it has reliable performance under various scenarios. Using a simulation study, we compared our proposed method with several existing methods and demonstrated its competitiveness in terms of type I error control and statistical power. By applying the method to the Study of Addiction: Genetics and Environment (SAGE) dataset, we successfully identified several genes associated with substance dependence.
$\Lambda$-coalescents arising in populations with dormancy

Consider a population evolving from year to year through three seasons: spring, summer and winter. Every spring starts with $N$ dormant individuals waking up independently of each other according to a given distribution. Once an individual is awake, it starts reproducing at a constant rate. By the end of spring, all individuals are awake and continue reproducing independently as Yule processes during the whole summer. In the winter, $N$ individuals chosen uniformly at random go to sleep until the next spring, and the other individuals die. We show that because an individual that wakes up unusually early can have a large number of surviving descendants, for some choices of model parameters the genealogy of the population will be described by a $\Lambda$-coalescent. In particular, the beta coalescent can describe the genealogy when the rate at which individuals wake up increases exponentially over time. We also characterize the set of all $\Lambda$-coalescents that can arise in this framework.
Evaluation of Local Explanation Methods for Multivariate Time Series Forecasting

Being able to interpret a machine learning model is a crucial task in many applications of machine learning. Specifically, local interpretability is important in determining why a model makes particular predictions. Despite the recent focus on AI interpretability, there has been a lack of research in local interpretability methods for time series forecasting while the few interpretable methods that exist mainly focus on time series classification tasks. In this study, we propose two novel evaluation metrics for time series forecasting: Area Over the Perturbation Curve for Regression and Ablation Percentage Threshold. These two metrics can measure the local fidelity of local explanation models. We extend the theoretical foundation to collect experimental results on two popular datasets, \textit{Rossmann sales} and \textit{electricity}. Both metrics enable a comprehensive comparison of numerous local explanation models and find which metrics are more sensitive. Lastly, we provide heuristical reasoning for this analysis.
Quantum Cuntz-Krieger algebras

Motivated by the theory of Cuntz-Krieger algebras we define and study C ∗ C^\ast -algebras associated to directed quantum graphs. For classical graphs the C ∗ C^\ast -algebras obtained this way can be viewed as free analogues of Cuntz-Krieger algebras, and need not be nuclear. We study two particular classes of quantum graphs in detail, namely the trivial and the complete quantum graphs. For the trivial quantum graph on a single matrix block, we show that the associated quantum Cuntz-Krieger algebra is neither unital, nuclear nor simple, and does not depend on the size of the matrix block up to K K KK -equivalence. In the case of the complete quantum graphs we use quantum symmetries to show that, in certain cases, the corresponding quantum Cuntz-Krieger algebras are isomorphic to Cuntz algebras. These isomorphisms, which seem far from obvious from the definitions, imply in particular that these C ∗ C^\ast -algebras are all pairwise non-isomorphic for complete quantum graphs of different dimensions, even on the level of K K KK -theory. We explain how the notion of unitary error basis from quantum information theory can help to elucidate the situation. We also discuss quantum symmetries of quantum Cuntz-Krieger algebras in general.
Intimate Partner Violence and Injury Prediction From Radiology Reports

Intimate partner violence (IPV) is an urgent, prevalent, and under-detected public health issue. We present machine learning models to assess patients for IPV and injury. We train the predictive algorithms on radiology reports with 1) IPV labels based on entry to a violence prevention program and 2) injury labels provided by emergency radiology fellowship-trained physicians. Our dataset includes 34,642 radiology reports and 1479 patients of IPV victims and control patients. Our best model predicts IPV a median of 3.08 years before violence prevention program entry with a sensitivity of 64% and a specificity of 95%. We conduct error analysis to determine for which patients our model has especially high or low performance and discuss next steps for a deployed clinical risk model.
OpenAttack: An Open-source Textual Adversarial Attack Toolkit

Textual adversarial attacking has received wide and increasing attention in recent years. Various attack models have been proposed, which are enormously distinct and implemented with different programming frameworks and settings. These facts hinder quick utilization and fair comparison of attack models. In this paper, we present an open-source textual adversarial attack toolkit named OpenAttack to solve these issues. Compared with existing other textual adversarial attack toolkits, OpenAttack has its unique strengths in support for all attack types, multilinguality, and parallel processing. Currently, OpenAttack includes 15 typical attack models that cover all attack types. Its highly inclusive modular design not only supports quick utilization of existing attack models, but also enables great flexibility and extensibility. OpenAttack has broad uses including comparing and evaluating attack models, measuring robustness of a model, assisting in developing new attack models, and adversarial training. Source code and documentation can be obtained at https://github.com/thunlp/OpenAttack.
A Relationship Between SIR Model and Generalized Logistic Distribution with Applications to SARS and COVID-19

This paper shows that the generalized logistic distribution model is derived from the well-known compartment model, consisting of susceptible, infected and recovered compartments, abbreviated as the SIR model, under certain conditions. In the SIR model, there are uncertainties in predicting the final values for the number of infected population and the infectious parameter. However, by utilizing the information obtained from the generalized logistic distribution model, we can perform the SIR numerical computation more stably and more accurately. Applications to severe acute respiratory syndrome (SARS) and Coronavirus disease 2019 (COVID-19) using this combined method are also introduced.
Multimode optically adiabatic silica glass submicron taper

Silica nanofibers fabricated by tapering optical fibers have attracted considerable interest as an ultimate platform for high-efficiency light-matter interactions. While previously demonstrated applications relied exclusively on the low-loss transmission of only the fundamental mode, the implementation of multimode tapers that adiabatically transmit several modes has remained very challenging, hindering their use in various emerging applications in multimode nonlinear optics and quantum optics. Here, we report the first realization of multimode submicron tapers that permit the simultaneous adiabatic transmission of multiple higher-order modes including the LP02 mode, through introducing deep wet-etching of conventional fiber before fiber tapering. Furthermore, as a critical application, we demonstrate "fundamental-to-fundamental" all-fiber third-harmonic generation with high conversion efficiencies. Our work paves the way for ultrahigh-efficiency multimode nonlinear and quantum optics, facilitating nonclassical light generation in the multimode regime, multimode soliton interactions and photonic quantum gates, and manipulation of the evanescent-field-induced optical trapping potentials of atoms and nanoparticles.
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition

Despite the recent achievements made in the multi-modal emotion recognition task, two problems still exist and have not been well investigated: 1) the relationship between different emotion categories are not utilized, which leads to sub-optimal performance; and 2) current models fail to cope well with low-resource emotions, especially for unseen emotions. In this paper, we propose a modality-transferable model with emotion embeddings to tackle the aforementioned issues. We use pre-trained word embeddings to represent emotion categories for textual data. Then, two mapping functions are learned to transfer these embeddings into visual and acoustic spaces. For each modality, the model calculates the representation distance between the input sequence and target emotions and makes predictions based on the distances. By doing so, our model can directly adapt to the unseen emotions in any modality since we have their pre-trained embeddings and modality mapping functions. Experiments show that our model achieves state-of-the-art performance on most of the emotion categories. Besides, our model also outperforms existing baselines in the zero-shot and few-shot scenarios for unseen emotions.
Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Drifts

Bayesian deep learning offers a principled way to address many issues concerning safety of artificial intelligence (AI), such as model uncertainty,model interpretability, and prediction bias. However, due to the lack of efficient Monte Carlo algorithms for sampling from the posterior of deep neural networks (DNNs), Bayesian deep learning has not yet powered our AI system. We propose a class of adaptive stochastic gradient Markov chain Monte Carlo (SGMCMC) algorithms, where the drift function is biased to enhance escape from saddle points and the bias is adaptively adjusted according to the gradient of past samples. We establish the convergence of the proposed algorithms under mild conditions, and demonstrate via numerical examples that the proposed algorithms can significantly outperform the existing SGMCMC algorithms, such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian Monte Carlo (SGHMC) and preconditioned SGLD, in both simulation and optimization tasks.
The Solar Wind Angular Momentum Flux as Observed by Parker Solar Probe

The long-term evolution of the Sun’s rotation period cannot be directly observed, and is instead inferred from trends in the measured rotation periods of other Sun-like stars. Assuming the Sun spins down as it ages, following rotation rate ∝ age−1/2, requires the current solar angular momentum (AM) loss rate to be around 6 × 1030 erg. Magnetohydrodynamic models, and previous observations of the solar wind (from the Helios and Wind spacecraft), generally predict a values closer to 1 × 1030 erg or 3 × 1030 erg, respectively. Recently, the Parker Solar Probe (PSP) observed tangential solar wind speeds as high as ∼50 km s−1 in a localized region of the inner heliosphere. If such rotational flows were prevalent throughout the corona, it would imply that the solar wind AM-loss rate is an order of magnitude larger than all of those previous estimations. In this Letter, we evaluate the AM flux in the solar wind, using data from the first two orbits of PSP. The solar wind is observed to contain both large positive (as seen during perihelion), and negative AM fluxes. We analyze two solar wind streams that were repeatedly traversed by PSP; the first is a slow wind stream whose average AM flux fluctuates between positive and negative values, and the second is an intermediate speed stream that contains a positive AM flux (more consistent with a constant flow of AM). When the data from PSP are evaluated holistically, the average equatorial AM flux implies a global AM-loss rate of around (2.6–4.2) × 1030 erg (which is more consistent with observations from previous spacecraft).