pwc-archive/papers-with-abstracts huggingface dataset:
Non-autoregressive Translation with Disentangled Context Transformer
Authors: Jungo Kasai, James Cross, Marjan Ghazvininejad, Jiatao Gu
Date: 2020-01-01 00:00:00
Proceeding: ICML 2020 1
Tasks: Machine Translation, Sentence, Translation

State-of-the-art neural machine translation models generate a translation from left to right and every step is conditioned on the previously generated tokens. The sequential nature of this generation process causes fundamental latency in inference since we cannot generate multiple tokens in each sentence in parallel. We propose an attention-masking based model, called Disentangled Context (DisCo) transformer, that simultaneously generates all tokens given different contexts. The DisCo transformer is trained to predict every output token given an arbitrary subset of the other reference tokens. We also develop the parallel easy-first inference algorithm, which iteratively refines every token in parallel and reduces the number of required iterations. Our extensive experiments on 7 translation directions with varying data sizes demonstrate that our model achieves competitive, if not better, performance compared to the state of the art in non-autoregressive machine translation while significantly reducing decoding time on average.
On the (In)tractability of Computing Normalizing Constants for the Product of Determinantal Point Processes
Authors: Naoto Ohsaka, Tatsuya Matsuoka
Date: 2020-01-01 00:00:00
Proceeding: ICML 2020 1
Tasks: Open-Ended Question Answering, Point Processes

We consider the product of determinantal point processes (DPPs), a point process whose probability mass is proportional to the product of principal minors of multiple matrices as a natural, promising generalization of DPPs.
We study the computational complexity of computing its normalizing constant, which is among the most essential probabilistic inference tasks.
Our complexity-theoretic results (almost) rule out the existence of efficient algorithms for this task, unless input matrices are forced to have favorable structures.
In particular, we prove the following:
(1) Computing $\sum_{S} \det(\mat{A}_{S,S})^p$ exactly for every (fixed) positive even integer $p$ is UP-hard and Mod3P-hard, which gives a negative answer to an open question posed by Kulesza and Taskar (2012).
(2) $\sum_{S} \det(\mat{A}_{S,S}) \det(\mat{B}_{S,S}) \det(\mat{C}_{S,S})$ is NP-hard to approximate within a factor of $ 2^{O(|I|^{1-\epsilon})} $ for any $\epsilon > 0$, where $|I|$ is the input size. This result is stronger than #P-hardness for the case of two matrices by Gillenwater (2014).
(3) There exists a $ k^{O(k)} |I|^{O(1)} $-time algorithm for computing $\sum_{S} \det(\mat{A}_{S,S}) \det(\mat{B}_{S,S})$, where $k$ is ``the maximum rank of $\mat{A}$ and $\mat{B}$'' or ``the treewidth of the graph formed by non-zero entries of $\mat{A}$ and $\mat{B}$.'' Such parameterized algorithms are said to be fixed-parameter tractable.
On the Power of Compressed Sensing with Generative Models
Authors: Akshay Kamath, Eric Price, Sushrut Karmalkar
Date: 2020-01-01 00:00:00
Proceeding: ICML 2020 1
Tasks: compressed sensing

The goal of compressed sensing is to learn a structured signal $x$ from a limited number of noisy linear measurements $y \approx Ax$.  In traditional compressed sensing, ``structure'' is represented by sparsity in some known basis.  Inspired by the success of deep learning in modeling images, recent work starting with Bora et.al has instead considered structure to come from a generative model $G: \R^k \to \R^n$. In this paper, we prove results that (i)establish the difficulty of this task and show that existing bounds are tight and (ii) demonstrate that the latter task is a generalization of the former.

First, we provide a lower bound matching the upper bound of Bora et.al. for compressed sensing from $L$-Lipschitz generative models $G$.  In particular, there exists such a function that requires roughly $\Omega(k \log L)$ linear measurements for sparse recovery to be possible. This holds even for the more relaxed goal of \emph{nonuniform} recovery.

Second, we show that generative models generalize sparsity as a representation of structure. In particular, we construct a ReLU-based neural network $G: \R^{k} \to \R^n$ with $O(1)$ layers and $O(n)$ activations per layer, such that the range of $G$ contains all $k$-sparse vectors.
Projection-free Distributed Online Convex Optimization with $O(\sqrt{T})$ Communication Complexity
Authors: Yuanyu Wan, Wei-Wei Tu, Lijun Zhang
Date: 2020-01-01 00:00:00
Proceeding: ICML 2020 1

To deal with complicated constraints via locally light computation in distributed online learning, recent study has presented a projection-free algorithm called distributed online conditional gradient (D-OCG), and achieved an $O(T^{3/4})$ regret bound, where $T$ is the number of prediction rounds. However, in each round, the local learners of D-OCG need to communicate with their neighbors to share the local gradients, which results in a high communication complexity of $O(T)$. In this paper, we first propose an improved variant of D-OCG, namely D-BOCG, which enjoys an $O(T^{3/4})$ regret bound with only $O(\sqrt{T})$ communication complexity. The key idea is to divide the total prediction rounds into $\sqrt{T}$ equally-sized blocks, and only update the local learners in the beginning of each block by performing iterative linear optimization steps. Furthermore, to handle the more challenging bandit setting, in which only the loss value is available, we incorporate the classical one-point gradient estimator into D-BOCG, and obtain similar theoretical guarantees.
Sparse Subspace Clustering with Entropy-Norm
Authors: Liang Bai, Jiye Liang
Date: 2020-01-01 00:00:00
Proceeding: ICML 2020 1
Tasks: Clustering

Sparse subspace clustering (SSC) and spectral clustering (SC) are both state-of-the-art methods to identify complex clusters in high-dimensional input space. However, there are few researches to discuss the relation between them. Therefore, in this paper, we provide an explicit theoretical connection between them from the respective of learning a data similarity matrix. We show that spectral clustering with Gaussian kernel can be viewed as sparse subspace clustering with entropy-norm (SSC+E). Compared to existing SSC algorithms, the SSC+E algorithm can obtain a sparse, analytical, symmetrical and nonnegative similarity matrix. Besides, it makes use of Gaussian kernel to compute the sparse similarity matrix of objects, which can avoid the complex computation of the sparse optimization program of SSC. Finally, we provide the experimental analysis to compare the efficiency and effectiveness of sparse subspace clustering and spectral clustering on ten benchmark data sets. The theoretical and experimental analysis can well help users for the selection of high-dimensional data clustering algorithms.
ETHAN at SemEval-2020 Task 5: Modelling Causal Reasoning inLanguage using neuro-symbolic cloud computing
Authors: Len Yabloko
Date: 2020-08-28 00:00:00
Proceeding: COLING 2020 8
Tasks: Cloud Computing, Counterfactual Detection, Language Modeling, Language Modelling

Experimental Testing of Hybrid AI Node implemented entirely on free cloud computing infrastructure. The ultimate goal of this research is to create modular reusable hybrid neuro-symbolic architecture for Artificial Intelligence. As a test case I model natural language comprehension of causal relations from open domain text corpus that combines semi-supervised language model (Huggingface Transformers) with constituency and dependency parsers (Allen Institute for Artificial Intelligence). The experimental results presented in this paper show potential for SOTA-level performance and demonstrate hybrid Artificial Intelligence in action. On SemEval-2020 Task 5 Subtask 1 ETHAN achieves F1 0.878 which is the second best result at the time of my writing. On Subtask 2 ETHAN currently scores 0.673 which is within the top ten results. All code used for experiments is available as open source and can be executed directly from the web browser without any installation or configuration. I briefly discuss some theoretical approaches that led to the proposed solutions and future directions of this research.
Efficient Waveform Covariance Matrix Design and Antenna Selection for MIMO Radar
Authors: Arindam Bose, Shahin Khobahi, Mojtaba Soltanalian
Date: 2020-02-13 00:00:00

Controlling the radar beam-pattern by optimizing the transmit covariance matrix is a well-established approach for performance enhancement in multiple-input-multiple-output (MIMO) radars. In this paper, we investigate the joint optimization of the waveform covariance matrix and the antenna position vector for a MIMO radar system to approximate a given transmit beam-pattern, as well as to minimize the cross-correlation between the probing signals at a number of given target locations. We formulate this design task as a non-convex optimization problem and then propose a cyclic optimization approach to efficiently approximate its solution. We further propose a local binary search algorithm in order to efficiently design the corresponding antenna positions. We show that the proposed method can be extended to the more general case of approximating the given beam-pattern using a minimal number of antennas as well as optimizing their positions. Our numerical investigations demonstrate a great performance both in terms of accuracy and computational complexity, making the proposed framework a good candidate for usage in real-time radar waveform processing applications such as MIMO radar transmit beamforming for aerial drones that are in motion.
Generate High Resolution Images With Generative Variational Autoencoder
Authors: Abhinav Sagar
Date: 2020-08-12 00:00:00
Proceeding: arXiv 2020 8
Tasks: Decoder, SSIM, Vocal Bursts Intensity Prediction

In this work, we present a novel neural network to generate high resolution images. We replace the decoder of VAE with a discriminator while using the encoder as it is. The encoder is fed data from a normal distribution while the generator is fed from a gaussian distribution. The combination from both is given to a discriminator which tells whether the generated image is correct or not. We evaluate our network on 3 different datasets: MNIST, LSUN and CelebA dataset. Our network beats the previous state of the art using MMD, SSIM, log likelihood, reconstruction error, ELBO and KL divergence as the evaluation metrics while generating much sharper images. This work is potentially very exciting as we are able to combine the advantages of generative models and inference models in a principled bayesian manner.
Joint Design of RF and gradient waveforms via auto-differentiation for 3D tailored excitation in MRI
Authors: Tianrui Luo, Douglas C. Noll, Jeffrey A. Fessler, Jon-Fredrik Nielsen
Date: 2020-08-24 00:00:00
Tasks: Computational Efficiency

This paper proposes a new method for joint design of radiofrequency (RF) and gradient waveforms in Magnetic Resonance Imaging (MRI), and applies it to the design of 3D spatially tailored saturation and inversion pulses. The joint design of both waveforms is characterized by the ODE Bloch equations, to which there is no known direct solution. Existing approaches therefore typically rely on simplified problem formulations based on, e.g., the small-tip approximation or constraining the gradient waveforms to particular shapes, and often apply only to specific objective functions for a narrow set of design goals (e.g., ignoring hardware constraints). This paper develops and exploits an auto-differentiable Bloch simulator to directly compute Jacobians of the (Bloch-simulated) excitation pattern with respect to RF and gradient waveforms. This approach is compatible with \emph{arbitrary} sub-differentiable loss functions, and optimizes the RF and gradients directly without restricting the waveform shapes. For computational efficiency, we derive and implement explicit Bloch simulator Jacobians (approximately halving computation time and memory usage). To enforce hardware limits (peak RF, gradient, and slew rate), we use a change of variables that makes the 3D pulse design problem effectively unconstrained; we then optimize the resulting problem directly using the proposed auto-differentiation framework. We demonstrate our approach with two kinds of 3D excitation pulses that cannot be easily designed with conventional approaches: Outer-volume saturation (90{\deg} flip angle), and inner-volume inversion.
On a Class of Bias-Amplifying Variables that Endanger Effect Estimates
Authors: Judea Pearl
Date: 2012-03-15 00:00:00

This note deals with a class of variables that, if conditioned on, tends to amplify confounding bias in the analysis of causal effects. This class, independently discovered by Bhattacharya and Vogt (2007) and Wooldridge (2009), includes instrumental variables and variables that have greater influence on treatment selection than on the outcome. We offer a simple derivation and an intuitive explanation of this phenomenon and then extend the analysis to non linear models. We show that: 1. the bias-amplifying potential of instrumental variables extends over to non-linear models, though not as sweepingly as in linear models; 2. in non-linear models, conditioning on instrumental variables may introduce new bias where none existed before; 3. in both linear and non-linear models, instrumental variables have no effect on selection-induced bias.
Privacy-preserving Voice Analysis via Disentangled Representations
Authors: Ranya Aloufi, Hamed Haddadi, David Boyle
Date: 2020-07-29 00:00:00
Tasks: Attribute, Privacy Preserving, Representation Learning, speech-recognition, Speech Recognition, User Identification

Voice User Interfaces (VUIs) are increasingly popular and built into smartphones, home assistants, and Internet of Things (IoT) devices. Despite offering an always-on convenient user experience, VUIs raise new security and privacy concerns for their users. In this paper, we focus on attribute inference attacks in the speech domain, demonstrating the potential for an attacker to accurately infer a target user's sensitive and private attributes (e.g. their emotion, sex, or health status) from deep acoustic models. To defend against this class of attacks, we design, implement, and evaluate a user-configurable, privacy-aware framework for optimizing speech-related data sharing mechanisms. Our objective is to enable primary tasks such as speech recognition and user identification, while removing sensitive attributes in the raw speech data before sharing it with a cloud service provider. We leverage disentangled representation learning to explicitly learn independent factors in the raw data. Based on a user's preferences, a supervision signal informs the filtering out of invariant factors while retaining the factors reflected in the selected preference. Our experimental evaluation over five datasets shows that the proposed framework can effectively defend against attribute inference attacks by reducing their success rates to approximately that of guessing at random, while maintaining accuracy in excess of 99% for the tasks of interest. We conclude that negotiable privacy settings enabled by disentangled representations can bring new opportunities for privacy-preserving applications.
Transmit Antenna Selection in MIMO Wiretap Channels: A Machine Learning Approach
Authors: Dongxuan He, Chenxi Liu
Date: 2018-07-04 00:00:00
Proceeding: 7 2018 7
Tasks: BIG-bench Machine Learning

In this letter, we exploit the potential benefits
of machine learning in enhancing physical layer security in multi-input multi-output multi-antenna-eavesdropper wiretap channels. To this end, we focus on the scenario where the source adopts transmit antenna selection (TAS) as the transmission strategy. We assume that the channel state information (CSI) of the legitimate receiver is available to the source, while the CSI of the eavesdropper can be either known or not known at the source. By modeling the problem of TAS as a multiclass classification problem, we propose two machine learning-based schemes, namely, the support vector machine-based scheme and the naive-Bayesbased scheme, to select the optimal antenna that maximizes the secrecy performance of the considered system. Compared to the conventional TAS scheme, we show that our proposed schemes can achieve almost the same secrecy performance with relatively small feedback overhead. The work presented here provides insights into the design of new machine learning-based secure transmission schemes
Secure Transmission via Lens Antenna Array in Millimeter Wave MISO Systems
Authors: Hyojin Lee, Wanjik Kim, and Joonhyuk Kang
Date: 2019-04-28 00:00:00
Proceeding: 2019 2019 4

Millimeter wave (mmWave) communication is considered as a promising technology for future wireless communication systems. Since a radio-frequency (RF) chain has high cost and high power consumption, it is impractical to implement fully digital beamforming in the mmWave communication systems. In this paper, we propose a secure transmission scheme with limited RF chains by implementing a lens antenna array in a mmWave multiple-input single-output multiple-eavesdropper (MISOME) wiretap channel. Due to the energy-focusing property of a lens antenna array, the proposed scheme requires a few RF chains for secure communication, not just the number of all antennas. We adopt artificial noise (AN) to degrade the wiretap channel in both cases of available channel state information of the eavesdropper (CSIE) at the transmitter and unavailable that. For the CSIE available case, a simple antennas selection scheme is proposed. We propose an algorithm for selecting antennas and designing analog phase shifters in the CSIE unavailable case. Numerical results demonstrate a remarkable secrecy performance enhancement compared with other hybrid beamforming strategies exploiting a conventional uniform planar array (UPA)
dynamic Long Short-Term Memory Neural-Network-Based Indict Remaining-Useful-Life Prognosis for Satellite Lithium-ion Battery
Authors: Cunsong Wang 1, Ningyun Lu 1, *, Senlin Wang 2, Yuehua Cheng 3 and Bin Jiang 1
Date: 2018-11-28 00:00:00
Proceeding: https://xueshu.baidu.com/usercenter/paper/show?paperid=1t110tb0bg620c70t36u00d0hc679631&site=xueshu_se 2018 11
Tasks: Incremental Learning, L2 Regularization, Prognosis

On-line remaining-useful-life (RUL) prognosis is still a problem for satellite Lithium-ion
(Li-ion) batteries. Meanwhile, capacity, widely used as a health indicator of a battery (HI),
is inconvenient or even impossible to measure. Aiming at practical and precise prediction of the
RUL of satellite Li-ion batteries, a dynamic long short-term memory (DLSTM) neural-network-based
indirect RUL prognosis is proposed in this paper. Firstly, an indirect HI based on the Spearman
correlation analysis method is extracted from the battery discharge voltages, and the relationship
between the indirect HI indices and battery capacity is established using a polynomial fitting
method. Then, by integrating the Adam method, L2 regularization method, and incremental learning,
a DLSTM method is proposed and applied for Li-ion battery RUL prognosis. Finally, verification of
the results on NASA #5 battery data sets demonstrates that the proposed method has better dynamic
performance and higher accuracy than the three other popular methods.
Efficiently Solving MDPs with Stochastic Mirror Descent
Authors: Yujia Jin, Aaron Sidford
Date: 2020-08-28 00:00:00
Proceeding: ICML 2020 1

We present a unified framework based on primal-dual stochastic mirror descent for approximately solving infinite-horizon Markov decision processes (MDPs) given a generative model. When applied to an average-reward MDP with $A_{tot}$ total state-action pairs and mixing time bound $t_{mix}$ our method computes an $\epsilon$-optimal policy with an expected $\widetilde{O}(t_{mix}^2 A_{tot} \epsilon^{-2})$ samples from the state-transition matrix, removing the ergodicity dependence of prior art. When applied to a $\gamma$-discounted MDP with $A_{tot}$ total state-action pairs our method computes an $\epsilon$-optimal policy with an expected $\widetilde{O}((1-\gamma)^{-4} A_{tot} \epsilon^{-2})$ samples, matching the previous state-of-the-art up to a $(1-\gamma)^{-1}$ factor. Both methods are model-free, update state values and policies simultaneously, and run in time linear in the number of samples taken. We achieve these results through a more general stochastic mirror descent framework for solving bilinear saddle-point problems with simplex and box domains and we demonstrate the flexibility of this framework by providing further applications to constrained MDPs.
On the model-based stochastic value gradient for continuous reinforcement learning
Authors: Brandon Amos, Samuel Stanton, Denis Yarats, Andrew Gordon Wilson
Date: 2020-08-28 00:00:00
Tasks: Continuous Control, Humanoid Control, Model-based Reinforcement Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

For over a decade, model-based reinforcement learning has been seen as a way to leverage control-based domain knowledge to improve the sample-efficiency of reinforcement learning agents. While model-based agents are conceptually appealing, their policies tend to lag behind those of model-free agents in terms of final reward, especially in non-trivial environments. In response, researchers have proposed model-based agents with increasingly complex components, from ensembles of probabilistic dynamics models, to heuristics for mitigating model error. In a reversal of this trend, we show that simple model-based agents can be derived from existing ideas that not only match, but outperform state-of-the-art model-free agents in terms of both sample-efficiency and final reward. We find that a model-free soft value estimate for policy evaluation and a model-based stochastic value gradient for policy improvement is an effective combination, achieving state-of-the-art results on a high-dimensional humanoid control task, which most model-based agents are unable to solve. Our findings suggest that model-based policy evaluation deserves closer attention.
Dynamic Graph Neural Network for Traffic Forecasting in Wide Area Networks
Authors: Tanwi Mallick, Mariam Kiran, Bashir Mohammed, Prasanna Balaprakash
Date: 2020-08-28 00:00:00
Tasks: Graph Neural Network, Management

Wide area networking infrastructures (WANs), particularly science and research WANs, are the backbone for moving large volumes of scientific data between experimental facilities and data centers. With demands growing at exponential rates, these networks are struggling to cope with large data volumes, real-time responses, and overall network performance. Network operators are increasingly looking for innovative ways to manage the limited underlying network resources. Forecasting network traffic is a critical capability for proactive resource management, congestion mitigation, and dedicated transfer provisioning. To this end, we propose a nonautoregressive graph-based neural network for multistep network traffic forecasting. Specifically, we develop a dynamic variant of diffusion convolutional recurrent neural networks to forecast traffic in research WANs. We evaluate the efficacy of our approach on real traffic from ESnet, the U.S. Department of Energy's dedicated science network. Our results show that compared to classical forecasting methods, our approach explicitly learns the dynamic nature of spatiotemporal traffic patterns, showing significant improvements in forecasting accuracy. Our technique can surpass existing statistical and deep learning approaches by achieving approximately 20% mean absolute percentage error for multiple hours of forecasts despite dynamic network traffic settings.