Miracle-dz/UniFAR-FTUs huggingface dataset:
Causal Bandits with Unknown Graph Structure

In causal bandit problems, the action set consists of interventions on variables of a causal graph. Several researchers have recently studied such bandit problems and pointed out their practical applications. However, all existing works rely on a restrictive and impractical assumption that the learner is given full knowledge of the causal graph structure upfront. In this paper, we develop novel causal bandit algorithms without knowing the causal graph. Our algorithms work well for causal trees, causal forests and a general class of causal graphs. The regret guarantees of our algorithms greatly improve upon those of standard multi-armed bandit (MAB) algorithms under mild conditions. Lastly, we prove our mild conditions are necessary: without them one cannot do better than standard MAB algorithms.
Bias Mitigation of Face Recognition Models Through Calibration

Face recognition models suffer from bias: for example, the probability of a false positive (incorrect face match) strongly depends on sensitive attributes like ethnicity. As a result, these models may disproportionately and negatively impact minority groups when used in law enforcement. In this work, we introduce the Bias Mitigation Calibration (BMC) method, which (i) increases model accuracy (improving the state-of-the-art), (ii) produces fairly-calibrated probabilities, (iii) signiﬁcantly reduces the gap in the false positive rates, and (iv) does not require knowledge of the sensitive attribute
Dispersive estimates for linearized water wave type equations in $\mathbb R^d$

We derive a $L^1_x (\mathbb R^d)-L^{\infty}_x ( \mathbb R^d)$ decay estimate of order $\mathcal O \left( t^{-d/2}\right)$ for the linear propagators $$\exp \left( {\pm it \sqrt{ |D|\left(1+ \beta |D|^2\right) \tanh |D | } }\right), \qquad \beta \in \{0, 1\}. \quad D = -i\nabla,$$ with a loss of $3d/4$ or $d/4$-derivatives in the case $\beta=0$ or $\beta=1$, respectively. These linear propagators are known to be associated with the linearized water wave equations, where the parameter $\beta$ measures surface tension effects. As an application we prove low regularity well-posedness for a Whitham-Boussinesq type system in $\mathbb R^d$, $d\ge 2$. This generalizes a recent result by Dinvay, Selberg and the third author where they proved low regularity well-posedness in $\mathbb R$ and $\mathbb R^2$.
Encouraging Neural Machine Translation to Satisfy Terminology Constraints

We present a new approach to encourage neural machine translation to satisfy lexical constraints. Our method acts at the training step and thereby avoiding the introduction of any extra computational overhead at inference step. The proposed method combines three main ingredients. The first one consists in augmenting the training data to specify the constraints. Intuitively, this encourages the model to learn a copy behavior when it encounters constraint terms. Compared to previous work, we use a simplified augmentation strategy without source factors. The second ingredient is constraint token masking, which makes it even easier for the model to learn the copy behavior and generalize better. The third one, is a modification of the standard cross entropy loss to bias the model towards assigning high probabilities to constraint words. Empirical results show that our method improves upon related baselines in terms of both BLEU score and the percentage of generated constraint terms.
How to Decompose a Tensor with Group Structure

In this work we study the orbit recovery problem, which is a natural abstraction for the problem of recovering a planted signal from noisy measurements under unknown group actions. Many important inverse problems in statistics, engineering and the sciences fit into this framework. Prior work has studied cases when the group is discrete and/or abelian. However fundamentally new techniques are needed in order to handle more complex group actions. Our main result is a quasi-polynomial time algorithm to solve orbit recovery over SO(3) – i.e. the cryoelectron tomography problem which asks to recover the three-dimensional structure of a molecule from noisy measurements of randomly rotated copies of it. We analyze a variant of the frequency marching heuristic in the framework of smoothed analysis. Our approach exploits the layered structure of the invariant polynomials, and simultaneously yields a new class of tensor decomposition algorithms that work in settings when the tensor is not low-rank but rather where the factors are algebraically related to each other by a group action. Email: cliu568@mit.edu. This work was supported in part by Ankur Moitra’s NSF CAREER Award CCF-1453261 and NSF Large CCF1565235. Email: moitra@mit.edu. This work was supported in part by a Microsoft Trustworthy AI Grant, NSF CAREER Award CCF-1453261, NSF Large CCF1565235, a David and Lucile Packard Fellowship and an ONR Young Investigator Award.
Robust Implicit Networks via Non-Euclidean Contractions

Implicit neural networks, a.k. a. , deep equilibrium networks, are a class of implicit-depth learning models where function evaluation is performed by solving a fixed point equation. They generalize classic feedforward models and are equivalent to infinite-depth weight-tied feedforward networks. While implicit models show improved accuracy and significant reduction in memory consumption, they can suffer from ill-posedness and convergence instability. This paper provides a new framework, which we call Non-Euclidean Monotone Operator Network (NEMON), to design well-posed and robust implicit neural networks based upon contraction theory for the non-Euclidean norm $\ell_{\infty}$. Our framework includes (i) a novel condition for well-posedness based on one-sided Lipschitz constants, (ii) an average iteration for computing fixed-points, and (iii) explicit estimates on input-output Lipschitz constants. Additionally, we design a training problem with the well-posedness condition and the average iteration as constraints and, to achieve robust models, with the input-output Lipschitz constant as a regularizer. Our $\ell_{\infty}$ well-posedness condition leads to a larger polytopic training search space than existing conditions and our average iteration enjoys accelerated convergence. Finally, we evaluate our framework in image classification through the MNIST and the CIFAR-10 datasets. Our numerical results demonstrate improved accuracy and robustness of the implicit models with smaller input-output Lipschitz bounds. Code is available at https://github.com/davydovalexander/Non-Euclidean_Mon_Op_Net.
Lightweight Dual-channel Target Speaker Separation for Mobile Voice Communication

Nowadays, there is a strong need to deploy the target speaker separation (TSS) model on mobile devices with a limitation of the model size and computational complexity. To better perform TSS for mobile voice communication, we first make a dual-channel dataset based on a specific scenario, LibriPhone. Specifically, to better mimic the real-case scenario, instead of simulating from the single-channel dataset, LibriPhone is made by simultaneously replaying pairs of utterances from LibriSpeech by two professional artificial heads and recording by two built-in microphones of the mobile. Then, we propose a lightweight time-frequency domain separation model, LSTM-Former, which is based on the LSTM framework with source-to-noise ratio (SI-SNR) loss. For the experiments on Libri-Phone, we explore the dual-channel LSTMFormer model and a single-channel version by a random single channel of Libri-Phone. Experimental result shows that the dual-channel LSTM-Former outperforms the single-channel LSTMFormer with relative 25% improvement. This work provides a feasible solution for the TSS task on mobile devices, playing back and recording multiple data sources in real application scenarios for getting dual-channel real data can assist the lightweight model to achieve higher performance.
Quantum Measurement Adversary

Multi-source extractors are functions that extract uniform randomness from multiple (weak) sources of randomness. Quantum multi-source extractors were considered by Kasher and Kempe (2010) (for the quantum independent adversary and the quantum bounded storage adversary), Chung et al. (2014) (for the general entangled adversary) and Arnon-Friedman et al. (2016) (for the quantum Markov adversary). One of the main objectives of this work is to unify all the existing quantum multi-source adversary models. We propose two new models of adversaries: 1) the quantum measurement adversary (<inline-formula> <tex-math notation="LaTeX">$\mathsf {qma}$ </tex-math></inline-formula>), which generates side information using entanglement and on post-measurement; and 2) the quantum communication adversary (<inline-formula> <tex-math notation="LaTeX">$\mathsf {qca}$ </tex-math></inline-formula>), which generates side information using entanglement and communication between multiple sources. We show that: 1) <inline-formula> <tex-math notation="LaTeX">$\mathsf {qma}$ </tex-math></inline-formula> is the strongest adversary among all the known adversaries, in the sense that the side information of all other adversaries can be generated by <inline-formula> <tex-math notation="LaTeX">$\mathsf {qma}$ </tex-math></inline-formula>; 2) The (generalized) inner-product function (in fact a general class of two-wise independent functions) continues to work as a good extractor with matching parameters as that of Chor and Goldreich (1985) against classical adversaries; 3) A non-malleable extractor proposed by Li (2012) (against classical adversaries) continues to be secure against quantum side information. This result implies a non-malleable extractor result of Aggarwal et al. (2019) with uniform seed. We strengthen their result via a completely different proof to make the non-malleable extractor of Li secure against quantum side information even when the seed is not uniform; 4) A modification (working with weak local randomness instead of uniform local randomness) of the Dodis and Wichs (2009) protocol for privacy-amplification is secure against active quantum adversaries (those who arbitrarily modify the messages exchanged in the protocol). This strengthens on a recent result due to Aggarwal et al. (2019) which uses uniform local randomness; 5) A tight efficiency lower bound for the (generalized) inner-product function (in fact a general class of two-wise independent functions).
A Primer on Multi-Neuron Relaxation-based Adversarial Robustness Certification

The existence of adversarial examples poses a real danger when deep neural networks are deployed in the real world. The go-to strategy to quantify this vulnerability is to evaluate the model against specific attack algorithms. This approach is however inherently limited, as it says little about the robustness of the model against more powerful attacks not included in the evaluation. We develop a unified mathematical framework to describe relaxation-based robustness certification methods, which go beyond adversary-specific robustness evaluation and instead provide provable robustness guarantees against attacks by any adversary. We discuss the fundamental limitations posed by single-neuron relaxations and show how the recent ``k-ReLU'' multi-neuron relaxation framework of Singh et al. (2019) obtains tighter correlation-aware activation bounds by leveraging additional relational constraints among groups of neurons. Specifically, we show how additional pre-activation bounds can be mapped to corresponding post-activation bounds and how they can in turn be used to obtain tighter robustness certificates. We also present an intuitive way to visualize different relaxation-based certification methods. By approximating multiple non-linearities jointly instead of separately, the k-ReLU method is able to bypass the convex barrier imposed by single neuron relaxations.
Trajectory Optimization of Chance-Constrained Nonlinear Stochastic Systems for Motion Planning and Control

—We present gPC-SCP: G eneralized P olynomial C haos-based S equential C onvex P rogramming method to compute a sub-optimal solution for a continuous-time chance-constrained stochastic nonlinear optimal control problem (SNOC) problem. The approach enables motion planning and control of robotic systems under uncertainty. The proposed method involves two steps. The ﬁrst step is to derive a deterministic nonlinear optimal control problem (DNOC) with convex constraints that are surrogate to the SNOC by using gPC expansion and the distributionally-robust convex subset of the chance constraints. The second step is to solve the DNOC problem using sequential convex programming (SCP) for trajectory generation and control. We prove that in the unconstrained case, the optimal value of the DNOC converges to that of SNOC asymptotically and that any feasible solution of the constrained DNOC is a feasible solution of the chance-constrained SNOC. We derive a stable stochastic model predictive controller using the gPC-SCP for tracking a trajectory in the presence of uncertainty. We empirically demonstrate the efﬁcacy of the gPC-SCP method for the following three test cases: 1) collision checking under uncertainty in actuation, 2) collision checking with stochastic obstacle model, and 3) safe trajectory tracking under uncertainty in the dynamics and obstacle location by using a receding horizon control approach. We validate the effectiveness of the gPC-SCP method on the robotic spacecraft testbed.
Simultaneously Transmitting And Reflecting (STAR) RIS Assisted NOMA Systems

In this paper, a novel simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted non-orthogonal multiple access (NOMA) system is proposed, where the STAR-RIS can simultaneously transmit and reflect the incident signals. Our objective is to maximize the achievable sum rate by jointly optimizing the decoding order, power allocation coefficients, active beamforming, transmission and reflection beamformings. However, the formulated problem is non-convex with intricately coupled variables. To tackle this challenge, a suboptimal two-layer iterative algorithm is proposed. Specifically, in the inner-layer iteration, for a given decoding order, the power allocation coefficients, active beamforming, transmission and reflection beamformings are optimized alternatively. For the outer-layer iteration, the decoding order of NOMA users in each cluster is updated with the solutions obtained from the inner-layer iteration. Simulation results are provided to demonstrate that the proposed STAR-RSI-NOMA system outperforms conventional RIS assisted systems.
Weakly-Supervised Methods for Suicide Risk Assessment: Role of Related Domains

Social media has become a valuable resource for the study of suicidal ideation and the assessment of suicide risk. Among social media platforms, Reddit has emerged as the most promising one due to its anonymity and its focus on topic-based communities (subreddits) that can be indicative of someone’s state of mind or interest regarding mental health disorders such as r/SuicideWatch, r/Anxiety, r/depression. A challenge for previous work on suicide risk assessment has been the small amount of labeled data. We propose an empirical investigation into several classes of weakly-supervised approaches, and show that using pseudo-labeling based on related issues around mental health (e.g., anxiety, depression) helps improve model performance for suicide risk assessment.
On the Power of Shallow Learning

A deluge of recent work has explored equivalences between wide neural networks and kernel methods. A central theme is that one can analytically ﬁnd the kernel corresponding to a given wide network architecture, but despite major implications for architecture design, no work to date has asked the converse question: given a kernel, can one ﬁnd a network that realizes it? We afﬁrmatively answer this question for fully-connected architectures, completely characterizing the space of achievable kernels. Furthermore, we give a surprising constructive proof that any kernel of any wide, deep, fully-connected net can also be achieved with a network with just one hidden layer and a specially-designed pointwise activation function. We experimentally verify our construction and demonstrate that, by just choosing the activation function, we can design a wide shallow network that mimics the generalization performance of any wide, deep, fully-connected network.
Exploring the Disproportion Between Scientific Productivity and Knowledge Amount

The pursuit of knowledge is the permanent goal of human beings. Scientific literature, as the major medium that carries knowledge between scientists, exhibits explosive growth during the last century. Despite the frequent use of many tangible measures, such as citation, impact factor and g-index, to quantify the influence of papers from different perspectives based on scientific productivity, it has not yet been well understood how the relationship between scientific productivity and knowledge amount turns out to be, i.e., how the knowledge value of papers and knowledge amount vary with development of the discipline. This raises the question of whether high scientific productivity equals large knowledge amount. Here, building on rich literature on academic conferences and journals, we collect 185 million articles covering 19 disciplines published during 1970 to 2020, and establish citation network research area to represent the knowledge flow from the authors of the article being cited to the authors of the articles that cite it under each specific area. As a result, the structure formed during the evolution of each scientific area can implicitly tells how the knowledge flows between nodes and how it behaves as the number of literature (productivity) increases. By leveraging Structural entropy in structured high-dimensional space and Shannon entropy in unstructured probability space, we propose the Quantitative Index of Knowledge (KQI), which is taken as the subtraction between the two types of entropy, to reflect the extent of disorder difference (knowledge amount) caused by structure (order). With the aid of KQI, we find that, although the published literature shows an explosive growth, the amount of knowledge (KQI) contained in it obviously slows down, and there is a threshold after which the growth of knowledge accelerates.
On the Design of Strategic Task Recommendations for Sustainable Crowdsourcing-Based Content Moderation

Crowdsourcing-based content moderation is a platform that hosts content moderation tasks for crowd workers to review user submissions (e.g. text, images and videos) and make decisions regarding the admissibility of the posted content, along with a gamut of other tasks such as image labeling and speech-to-text conversion. In an attempt to reduce cognitive overload at the workers and improve system efficiency, these platforms offer personalized task recommendations according to the worker's preferences. However, the current state-of-the-art recommendation systems disregard the effects on worker's mental health, especially when they are repeatedly exposed to content moderation tasks with extreme content (e.g. violent images, hate-speech). In this paper, we propose a novel, strategic recommendation system for the crowdsourcing platform that recommends jobs based on worker's mental status. Specifically, this paper models interaction between the crowdsourcing platform's recommendation system (leader) and the worker (follower) as a Bayesian Stackelberg game where the type of the follower corresponds to the worker's cognitive atrophy rate and task preferences. We discuss how rewards and costs should be designed to steer the game towards desired outcomes in terms of maximizing the platform's productivity, while simultaneously improving the working conditions of crowd workers.
Heavy flavor pentaquarks with four heavy quarks

Hong-Tao An1,2,∗ Kan Chen1,2,3,4,† Zhan-Wei Liu1,2,5,‡ and Xiang Liu1,2,5§ School of Physical Science and Technology, Lanzhou University, Lanzhou 730000, China Research Center for Hadron and CSR Physics, Lanzhou University and Institute of Modern Physics of CAS, Lanzhou 730000, China Center of High Energy Physics, Peking University, Beijing 100871, China School of Physics and State Key Laboratory of Nuclear Physics and Technology, Peking University, Beijing 100871, China Lanzhou Center for Theoretical Physics, Key Laboratory of Theoretical Physics of Gansu Province, and Frontiers Science Center for Rare Isotopes, Lanzhou University, Lanzhou 730000, China (Dated: June 8, 2021)
Multi-armed Bandit Algorithms on System-on-Chip: Go Frequentist or Bayesian?

Multi-armed Bandit (MAB) algorithms identify the best arm among multiple arms via exploration-exploitation trade-off without prior knowledge of arm statistics. Their usefulness in wireless radio, IoT, and robotics demand deployment on edge devices, and hence, a mapping on system-on-chip (SoC) is desired. Theoretically, the Bayesian approach-based Thompson Sampling (TS) algorithm offers better performance than the frequentist approach-based Upper Confidence Bound (UCB) algorithm. However, TS is not synthesizable due to \textit{Beta} function. We address this problem by approximating it via a pseudo-random number generator-based approach and efficiently realize the TS algorithm on Zynq SoC. In practice, the type of arms distribution (e.g., Bernoulli, Gaussian, etc.) is unknown and hence, a single algorithm may not be optimal. We propose a reconfigurable and intelligent MAB (RI-MAB) framework. Here, intelligence enables the identification of appropriate MAB algorithms for a given environment, and reconfigurability allows on-the-fly switching between algorithms on the SoC. This eliminates the need for parallel implementation of algorithms resulting in huge savings in resources and power consumption. We analyze the functional correctness, area, power, and execution time of the proposed and existing architectures for various arm distributions, word-length, and hardware-software co-design approaches. We demonstrate the superiority of the RI-MAB over TS and UCB only architectures.
Graph Belief Propagation Networks

With the wide-spread availability of complex relational data, semi-supervised node classification in graphs has become a central machine learning problem. Graph neural networks are a recent class of easy-to-train and accurate methods for this problem that map the features in the neighborhood of a node to its label, but they ignore label correlation during inference and their predictions are difficult to interpret. On the other hand, collective classification is a traditional approach based on interpretable graphical models that explicitly model label correlations. Here, we introduce a model that combines the advantages of these two approaches, where we compute the marginal probabilities in a conditional random field, similar to collective classification, and the potentials in the random field are learned through end-to-end training, akin to graph neural networks. In our model, potentials on each node only depend on that node's features, and edge potentials are learned via a coupling matrix. This structure enables simple training with interpretable parameters, scales to large networks, naturally incorporates training labels at inference, and is often more accurate than related approaches. Our approach can be viewed as either an interpretable message-passing graph neural network or a collective classification method with higher capacity and modernized training.