pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Shrinkage Estimation of the High-Dimensional Minimum-Variance Portfolio
Authors: Taras Bodnar, Nestor Parolya, Erik Thorsen
Date: 2021-06-03 00:00:00
Tasks: Vocal Bursts Intensity Prediction

In this paper, new results in random matrix theory are derived which allow us to construct a shrinkage estimator of the global minimum variance (GMV) portfolio when the shrinkage target is a random object. More specifically, the shrinkage target is determined as the holding portfolio estimated from previous data. The theoretical findings are applied to develop theory for dynamic estimation of the GMV portfolio, where the new estimator of its weights is shrunk to the holding portfolio at each time of reconstruction. Both cases with and without overlapping samples are considered in the paper. The non-overlapping samples corresponds to the case when different data of the asset returns are used to construct the traditional estimator of the GMV portfolio weights and to determine the target portfolio, while the overlapping case allows intersections between the samples. The theoretical results are derived under weak assumptions imposed on the data-generating process. No specific distribution is assumed for the asset returns except from the assumption of finite $4+\varepsilon$, $\varepsilon>0$, moments. Also, the population covariance matrix with unbounded spectrum can be considered. The performance of new trading strategies is investigated via an extensive simulation. Finally, the theoretical findings are implemented in an empirical illustration based on the returns on stocks included in the S\&P 500 index.
Explicit no arbitrage domain for sub-SVIs via reparametrization
Authors: Claude Martini, Arianna Mingone
Date: 2021-06-04 00:00:00

The no Butterfly arbitrage domain of Gatheral SVI 5-parameters formula for the volatility smile has been recently described. It requires in general a numerical minimization of 2 functions altogether with a few root finding procedures. We study here the case of some sub-SVIs (all with 3 parameters): the Symmetric SVI, the Vanishing Upward/Downward SVI, and SSVI, for which we provide an explicit domain, with no numerical procedure required.
Modeling premiums of non-life insurance companies in India
Authors: Kartik Sethi, Siddhartha P. Chakrabarty
Date: 2021-06-04 00:00:00

We undertake an empirical analysis for the premium data of non-life insurance companies operating in India, in the paradigm of fitting the data for the parametric distribution of Lognormal and the extreme value based distributions of Generalized Extreme Value and Generalized Pareto. The best fit to the data for ten companies considered, is obtained for the Generalized Extreme Value distribution.
Time-dependent relations between gaps and returns in a Bitcoin order book
Authors: Roberto Mota Navarro, Paulino Monroy Castillero, Francois Leyvraz
Date: 2021-06-04 00:00:00

Several studies have shown that large changes in the returns of an asset are associated with the sized of the gaps present in the order book In general, these associations have been studied without explicitly considering the dynamics of either gaps or returns. Here we present a study of these relationships. Our results suggest that the causal relationship between gaps and returns is limited to instantaneous causation.
Knowledge-Driven Stock Trend Prediction and Explanation via Temporal Convolutional Network
Authors: Shumin Deng；Ningyu Zhang；Wen Zhang；Jiaoyan Chen；Jeff Z. Pan
Date: 2019-05-13 00:00:00
Tasks: Prediction, Stock Trend Prediction

Deep neural networks have achieved promising results in stock
trend prediction. However, most of these models have two common
drawbacks, including (i) current methods are not sensitive enough
to abrupt changes of stock trend, and (ii) forecasting results are
not interpretable for humans. To address these two problems, we
propose a novel Knowledge-Driven Temporal Convolutional Network (KDTCN) for stock trend prediction and explanation. Firstly,
we extract structured events from financial news, and utilize external knowledge from knowledge graph to obtain event embeddings.
Then, we combine event embeddings and price values together to
forecast stock trend. We evaluate the prediction accuracy to show
how knowledge-driven events work on abrupt changes. We also
visualize the effect of events and linkage among events based on
knowledge graph, to explain why knowledge-driven events are
common sources of abrupt changes. Experiments demonstrate that
KDTCN can (i) react to abrupt changes much faster and outperform
state-of-the-art methods on stock datasets, as well as (ii) facilitate
the explanation of prediction particularly with abrupt changes.
Joint Transceiver Optimization for Wireless Communication PHY with Convolutional Neural Network
Authors: Banghua Zhu, Jintao Wang, Longzhuang He, Jian Song
Date: 2018-08-09 00:00:00

Deep Learning has a wide application in the area of natural language processing and image processing due to its strong ability of generalization. In this paper, we propose a novel neural network structure for jointly optimizing the transmitter and receiver in communication physical layer under fading channels. We build up a convolutional autoencoder to simultaneously conduct the role of modulation, equalization and demodulation. The proposed system is able to design different mapping scheme from input bit sequences of arbitrary length to constellation symbols according to different channel environments. The simulation results show that the performance of neural network based system is superior to traditional modulation and equalization methods in terms of time complexity and bit error rate (BER) under fading channels. The proposed system can also be combined with other coding techniques to further improve the performance. Furthermore, the proposed system network is more robust to channel variation than traditional communication methods.
Learning Physical-Layer Communication with Quantized Feedback
Authors: Jinxiang Song, Bile Peng, Christian Häger, Henk Wymeersch, Anant Sahai
Date: 2019-04-19 00:00:00
Tasks: Quantization

Data-driven optimization of transmitters and receivers can reveal new modulation and detection schemes and enable physical-layer communication over unknown channels. Previous work has shown that practical implementations of this approach require a feedback signal from the receiver to the transmitter. In this paper, we study the impact of quantized feedback in data-driven learning of physical-layer communication. A novel quantization method is proposed, which exploits the specific properties of the feedback signal and is suitable for non-stationary signal distributions. The method is evaluated for linear and nonlinear channels. Simulation results show that feedback quantization does not appreciably affect the learning process and can lead to excellent performance, even with $1$-bit quantization. In addition, it is shown that learning is surprisingly robust to noisy feedback where random bit flips are applied to the quantization bits.
Blind interactive learning of modulation schemes: Multi-agent cooperation without co-design
Authors: Anant Sahai, Joshua Sanz, Vignesh Subramanian, Caryn Tran, Kailas Vodrahalli
Date: 2019-10-21 00:00:00

We examine the problem of learning to cooperate in the context of wireless communication. In our setting, two agents must learn modulation schemes that enable them to communicate across a power-constrained additive white Gaussian noise channel. We investigate whether learning is possible under different levels of information sharing between distributed agents which are not necessarily co-designed. We employ the "Echo" protocol, a "blind" interactive learning protocol where an agent hears, understands, and repeats (echoes) back the message received from another agent, simultaneously training itself to communicate. To capture the idea of cooperation between "not necessarily co-designed" agents we use two different populations of function approximators - neural networks and polynomials. We also include interactions between learning agents and non-learning agents with fixed modulation protocols such as QPSK and 16QAM. We verify the universality of the Echo learning approach, showing it succeeds independent of the inner workings of the agents. In addition to matching the communication expectations of others, we show that two learning agents can collaboratively invent a successful communication approach from independent random initializations. We complement our simulations with an implementation of the Echo protocol in software-defined radios. To explore the continuum of co-design, we study how learning is impacted by different levels of information sharing between agents, including sharing training symbols, losses, and full gradients. We find that co-design (increased information sharing) accelerates learning. Learning higher order modulation schemes is a more difficult task, and the beneficial effect of co-design becomes more pronounced as the task becomes harder.
Extending the RISC-V ISA for Efficient RNN-based 5G Radio Resource Management
Authors: Renzo Andri, Tomas Henriksson, Luca Benini
Date: 2020-02-27 00:00:00
Tasks: Management

Radio Resource Management (RRM) in 5G mobile communication is a challenging problem for which Recurrent Neural Networks (RNN) have shown promising results. Accelerating the compute-intensive RNN inference is therefore of utmost importance. Programmable solutions are desirable for effective 5G-RRM top cope with the rapidly evolving landscape of RNN variations. In this paper, we investigate RNN inference acceleration by tuning both the instruction set and micro-architecture of a micro-controller-class open-source RISC-V core. We couple HW extensions with software optimizations to achieve an overall improvement in throughput and energy efficiency of 15$\times$ and 10$\times$ w.r.t. the baseline core on a wide range of RNNs used in various RRM tasks.
Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPs
Authors: Tao Liu, Ruida Zhou, Dileep Kalathil, P. R. Kumar, Chao Tian
Date: 2021-06-04 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Safe Exploration

We address the issue of safety in reinforcement learning. We pose the problem in an episodic framework of a constrained Markov decision process. Existing results have shown that it is possible to achieve a reward regret of $\tilde{\mathcal{O}}(\sqrt{K})$ while allowing an $\tilde{\mathcal{O}}(\sqrt{K})$ constraint violation in $K$ episodes. A critical question that arises is whether it is possible to keep the constraint violation even smaller. We show that when a strictly safe policy is known, then one can confine the system to zero constraint violation with arbitrarily high probability while keeping the reward regret of order $\tilde{\mathcal{O}}(\sqrt{K})$. The algorithm which does so employs the principle of optimistic pessimism in the face of uncertainty to achieve safe exploration. When no strictly safe policy is known, though one is known to exist, then it is possible to restrict the system to bounded constraint violation with arbitrarily high probability. This is shown to be realized by a primal-dual algorithm with an optimistic primal estimate and a pessimistic dual update.
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation
Authors: Evgenii Nikishin, Romina Abachi, Rishabh Agarwal, Pierre-Luc Bacon
Date: 2021-06-06 00:00:00
Proceeding: ICML Workshop URL 2021 7
Tasks: Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The shortcomings of maximum likelihood estimation in the context of model-based reinforcement learning have been highlighted by an increasing number of papers. When the model class is misspecified or has a limited representational capacity, model parameters with high likelihood might not necessarily result in high performance of the agent on a downstream control task. To alleviate this problem, we propose an end-to-end approach for model learning which directly optimizes the expected returns using implicit differentiation. We treat a value function that satisfies the Bellman optimality operator induced by the model as an implicit function of model parameters and show how to differentiate the function. We provide theoretical and empirical evidence highlighting the benefits of our approach in the model misspecification regime compared to likelihood-based methods.
Multi-modal Entity Alignment in Hyperbolic Space
Authors: Hao Guo, Jiuyang Tang, Weixin Zeng, Xiang Zhao, Li Liu
Date: 2021-06-07 00:00:00
Tasks: Entity Alignment, Knowledge Graphs, Multi-modal Entity Alignment

Many AI-related tasks involve the interactions of data in multiple modalities. It has been a new trend to merge multi-modal information into knowledge graph(KG), resulting in multi-modal knowledge graphs (MMKG). However, MMKGs usually suffer from low coverage and incompleteness. To mitigate this problem, a viable approach is to integrate complementary knowledge from other MMKGs. To this end, although existing entity alignment approaches could be adopted, they operate in the Euclidean space, and the resulting Euclidean entity representations can lead to large distortion of KG's hierarchical structure. Besides, the visual information has yet not been well exploited. In response to these issues, in this work, we propose a novel multi-modal entity alignment approach, Hyperbolic multi-modal entity alignment(HMEA), which extends the Euclidean representation to hyperboloid manifold. We first adopt the Hyperbolic Graph Convolutional Networks (HGCNs) to learn structural representations of entities. Regarding the visual information, we generate image embeddings using the densenet model, which are also projected into the hyperbolic space using HGCNs. Finally, we combine the structure and visual representations in the hyperbolic space and use the aggregated embeddings to predict potential alignment results. Extensive experiments and ablation studies demonstrate the effectiveness of our proposed model and its components.
Learning proofs for the classification of nilpotent semigroups
Authors: Carlos Simpson
Date: 2021-06-06 00:00:00
Tasks: BIG-bench Machine Learning, Classification

Machine learning is applied to find proofs, with smaller or smallest numbers of nodes, for the classification of 4-nilpotent semigroups.
Deep Matching Prior: Test-Time Optimization for Dense Correspondence
Authors: Sunghwan Hong, Seungryong Kim
Date: 2021-06-06 00:00:00
Proceeding: ICCV 2021 10
Tasks: Dense Pixel Correspondence Estimation, Geometric Matching

Conventional techniques to establish dense correspondences across visually or semantically similar images focused on designing a task-specific matching prior, which is difficult to model. To overcome this, recent learning-based methods have attempted to learn a good matching prior within a model itself on large training data. The performance improvement was apparent, but the need for sufficient training data and intensive learning hinders their applicability. Moreover, using the fixed model at test time does not account for the fact that a pair of images may require their own prior, thus providing limited performance and poor generalization to unseen images. In this paper, we show that an image pair-specific prior can be captured by solely optimizing the untrained matching networks on an input pair of images. Tailored for such test-time optimization for dense correspondence, we present a residual matching network and a confidence-aware contrastive loss to guarantee a meaningful convergence. Experiments demonstrate that our framework, dubbed Deep Matching Prior (DMP), is competitive, or even outperforms, against the latest learning-based methods on several benchmarks for geometric matching and semantic matching, even though it requires neither large training data nor intensive learning. With the networks pre-trained, DMP attains state-of-the-art performance on all benchmarks.
Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasets
Authors: Lan Zhang, Victor Prokhorov, Ehsan Shareghi
Date: 2021-06-07 00:00:00
Proceeding: ACL (RepL4NLP) 2021 8
Tasks: Decoder, Disentanglement, Inductive Bias

To highlight the challenges of achieving representation disentanglement for text domain in an unsupervised setting, in this paper we select a representative set of successfully applied models from the image domain. We evaluate these models on 6 disentanglement metrics, as well as on downstream classification tasks and homotopy. To facilitate the evaluation, we propose two synthetic datasets with known generative factors. Our experiments highlight the existing gap in the text domain and illustrate that certain elements such as representation sparsity (as an inductive bias), or representation coupling with the decoder could impact disentanglement. To the best of our knowledge, our work is the first attempt on the intersection of unsupervised representation disentanglement and text, and provides the experimental framework and datasets for examining future developments in this direction.
ImGAGN:Imbalanced Network Embedding via Generative Adversarial Graph Networks
Authors: Liang Qu, Huaisheng Zhu, Ruiqi Zheng, Yuhui Shi, Hongzhi Yin
Date: 2021-06-05 00:00:00
Tasks: Attribute, Classification, imbalanced classification, Network Embedding, Node Classification

Imbalanced classification on graphs is ubiquitous yet challenging in many real-world applications, such as fraudulent node detection. Recently, graph neural networks (GNNs) have shown promising performance on many network analysis tasks. However, most existing GNNs have almost exclusively focused on the balanced networks, and would get unappealing performance on the imbalanced networks. To bridge this gap, in this paper, we present a generative adversarial graph network model, called ImGAGN to address the imbalanced classification problem on graphs. It introduces a novel generator for graph structure data, named GraphGenerator, which can simulate both the minority class nodes' attribute distribution and network topological structure distribution by generating a set of synthetic minority nodes such that the number of nodes in different classes can be balanced. Then a graph convolutional network (GCN) discriminator is trained to discriminate between real nodes and fake (i.e., generated) nodes, and also between minority nodes and majority nodes on the synthetic balanced network. To validate the effectiveness of the proposed method, extensive experiments are conducted on four real-world imbalanced network datasets. Experimental results demonstrate that the proposed method ImGAGN outperforms state-of-the-art algorithms for semi-supervised imbalanced node classification task.
Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization
Authors: Siwei Yang, Shaozuo Yu, Bingchen Zhao, Yin Wang
Date: 2021-06-06 00:00:00

Visual pattern recognition over agricultural areas is an important application of aerial image processing. In this paper, we consider the multi-modality nature of agricultural aerial images and show that naively combining different modalities together without taking the feature divergence into account can lead to sub-optimal results. Thus, we apply a Switchable Normalization block to our DeepLabV3 segmentation model to alleviate the feature divergence. Using the popular symmetric Kullback Leibler divergence measure, we show that our model can greatly reduce the divergence between RGB and near-infrared channels. Together with a hybrid loss function, our model achieves nearly 10\% improvements in mean IoU over previously published baseline.
Antipodes of Label Differential Privacy: PATE and ALIBI
Authors: Mani Malek, Ilya Mironov, Karthik Prasad, Igor Shilov, Florian Tramèr
Date: 2021-06-07 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Bayesian Inference, Memorization, Privacy Preserving, Privacy Preserving Deep Learning

We consider the privacy-preserving machine learning (ML) setting where the trained model must satisfy differential privacy (DP) with respect to the labels of the training examples. We propose two novel approaches based on, respectively, the Laplace mechanism and the PATE framework, and demonstrate their effectiveness on standard benchmarks. While recent work by Ghazi et al. proposed Label DP schemes based on a randomized response mechanism, we argue that additive Laplace noise coupled with Bayesian inference (ALIBI) is a better fit for typical ML tasks. Moreover, we show how to achieve very strong privacy levels in some regimes, with our adaptation of the PATE framework that builds on recent advances in semi-supervised learning. We complement theoretical analysis of our algorithms' privacy guarantees with empirical evaluation of their memorization properties. Our evaluation suggests that comparing different algorithms according to their provable DP guarantees can be misleading and favor a less private algorithm with a tighter analysis. Code for implementation of algorithms and memorization attacks is available from https://github.com/facebookresearch/label_dp_antipodes.
ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias
Authors: Yufei Xu, Qiming Zhang, Jing Zhang, DaCheng Tao
Date: 2021-06-07 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Image Classification, Inductive Bias, Object Detection, Video Object Segmentation

Transformers have shown great potential in various computer vision tasks owing to their strong capability in modeling long-range dependency using the self-attention mechanism. Nevertheless, vision transformers treat an image as 1D sequence of visual tokens, lacking an intrinsic inductive bias (IB) in modeling local visual structures and dealing with scale variance. Alternatively, they require large-scale training data and longer training schedules to learn the IB implicitly. In this paper, we propose a novel Vision Transformer Advanced by Exploring intrinsic IB from convolutions, ie, ViTAE. Technically, ViTAE has several spatial pyramid reduction modules to downsample and embed the input image into tokens with rich multi-scale context by using multiple convolutions with different dilation rates. In this way, it acquires an intrinsic scale invariance IB and is able to learn robust feature representation for objects at various scales. Moreover, in each transformer layer, ViTAE has a convolution block in parallel to the multi-head self-attention module, whose features are fused and fed into the feed-forward network. Consequently, it has the intrinsic locality IB and is able to learn local features and global dependencies collaboratively. Experiments on ImageNet as well as downstream tasks prove the superiority of ViTAE over the baseline transformer and concurrent works. Source code and pretrained models will be available at GitHub.