pwc-archive/papers-with-abstracts huggingface dataset:
Portfolio Optimization based on Neural Networks Sensitivities from Assets Dynamics respect Common Drivers
Authors: Alejandro Rodriguez Dominguez
Date: 2022-02-17 00:00:00
Tasks: Clustering, Portfolio Optimization, Sensitivity

We present a framework for modeling asset and portfolio dynamics, incorporating this information into portfolio optimization. For this framework, we introduce the Commonality Principle, providing a solution for the optimal selection of portfolio drivers as the common drivers. Portfolio constituent dynamics are modeled by Partial Differential Equations, and solutions approximated with neural networks. Sensitivities with respect to the common drivers are obtained via Automatic Adjoint Differentiation. Information on asset dynamics is incorporated via sensitivities into portfolio optimization. Portfolio constituents are embedded into the space of sensitivities with respect to their common drivers, and a distance matrix in this space called the Sensitivity matrix is used to solve the convex optimization for diversification. The sensitivity matrix measures the similarity of the projections of portfolio constituents on a vector space formed by common drivers' returns and is used to optimize for diversification on both idiosyncratic and systematic risks while adding directionality and future behavior information via returns dynamics. For portfolio optimization, we perform hierarchical clustering on the sensitivity matrix. To the best of the author's knowledge, this is the first time that sensitivities' dynamics approximated with neural networks have been used for portfolio optimization. Secondly, that hierarchical clustering on a matrix of sensitivities is used to solve the convex optimization problem and incorporate the hierarchical information of these sensitivities. Thirdly, public and listed variables can be used to obtain maximum idiosyncratic and systematic diversification by means of the sensitivity space with respect to optimal portfolio drivers. We reach over-performance in many experiments with respect to all other out-of-sample methods for different markets and real datasets.
Beyond Trading Data: The Hidden Influence of Public Awareness and Interest on Cryptocurrency Volatility
Authors: Zeyd Boukhers, Azeddine Bouabdallah, Cong Yang, Jan Jürjens
Date: 2022-02-12 00:00:00
Tasks: Decision Making

Since Bitcoin first appeared on the scene in 2009, cryptocurrencies have become a worldwide phenomenon as important decentralized financial assets. Their decentralized nature, however, leads to notable volatility against traditional fiat currencies, making the task of accurately forecasting the crypto-fiat exchange rate complex. This study examines the various independent factors that affect the volatility of the Bitcoin-Dollar exchange rate. To this end, we propose CoMForE, a multimodal AdaBoost-LSTM ensemble model, which not only utilizes historical trading data but also incorporates public sentiments from related tweets, public interest demonstrated by search volumes, and blockchain hash-rate data. Our developed model goes a step further by predicting fluctuations in the overall cryptocurrency value distribution, thus increasing its value for investment decision-making. We have subjected this method to extensive testing via comprehensive experiments, thereby validating the importance of multimodal combination over exclusive reliance on trading data. Further experiments show that our method significantly surpasses existing forecasting tools and methodologies, demonstrating a 19.29% improvement. This result underscores the influence of external independent factors on cryptocurrency volatility.
Machine Learning Models in Stock Market Prediction
Authors: Gurjeet Singh
Date: 2022-02-06 00:00:00
Tasks: BIG-bench Machine Learning, Prediction, regression, Stock Market Prediction, Time Series Analysis

The paper focuses on predicting the Nifty 50 Index by using 8 Supervised Machine Learning Models. The techniques used for empirical study are Adaptive Boost (AdaBoost), k-Nearest Neighbors (kNN), Linear Regression (LR), Artificial Neural Network (ANN), Random Forest (RF), Stochastic Gradient Descent (SGD), Support Vector Machine (SVM) and Decision Trees (DT). Experiments are based on historical data of Nifty 50 Index of Indian Stock Market from 22nd April, 1996 to 16th April, 2021, which is time series data of around 25 years. During the period there were 6220 trading days excluding all the non trading days. The entire trading dataset was divided into 4 subsets of different size-25% of entire data, 50% of entire data, 75% of entire data and entire data. Each subset was further divided into 2 parts-training data and testing data. After applying 3 tests- Test on Training Data, Test on Testing Data and Cross Validation Test on each subset, the prediction performance of the used models were compared and after comparison, very interesting results were found. The evaluation results indicate that Adaptive Boost, k- Nearest Neighbors, Random Forest and Decision Trees under performed with increase in the size of data set. Linear Regression and Artificial Neural Network shown almost similar prediction results among all the models but Artificial Neural Network took more time in training and validating the model. Thereafter Support Vector Machine performed better among rest of the models but with increase in the size of data set, Stochastic Gradient Descent performed better than Support Vector Machine.
EmotionFlow: Capture the Dialogue Level Emotion Transitions
Authors: Xiaohui Song, Liangjun Zang, Rong Zhang, Songlin Hu, Longtao Huang
Date: 2022-05-07 00:00:00
Proceeding: ICASSP 2022 5
Tasks: Emotion Recognition, Emotion Recognition in Conversation

Emotion recognition in conversations (ERC) has attracted increasing interests in recent years, due to its wide range of applications, such as customer service analysis, health-care consultation, etc. One key challenge of ERC is that users' emotions would change due to the impact of others' emotions. That is, the emotions within the conversation can spread among the communication participants. However, the spread impact of emotions in a conversation is rarely addressed in existing researches. To this end, we propose \textbf{EmotionFlow} for ERC with the consideration of the spread of participants' emotions during a conversation. EmotionFlow first encodes users' utterance by concatenating the context with an auxiliary question, which helps to learn user-specific features. Then, conditional random field is applied to capture the sequential information at emotional level. We conduct extensive experiments on a public dataset Multimodal EmotionLines Dataset (MELD), and the results demonstrate the effectiveness of our proposed model.
Storage placement policy for minimizing frequency deviation: A combinatorial optimization approach
Authors: Ram Machlev, Nilanjan Roy Chowdhury, Juri Belikov, Yoash Levron
Date: 2020-11-01 00:00:00
Tasks: Combinatorial Optimization

As the share of renewable sources is increasing the need for multiple storage units appropriately sized and located is essential to achieve better inertial response. This work focuses on the question of "how to distribute constant number of storage units in the gird under transient events such that the inertial response of the maximum frequency deviation is minimized?". To answer this question, we provide a comprehensive modeling framework for energy storage units placement and size for frequency stability under spatial effects. The distributed storage units are modeled as grid supporting inverters and the total storage capacity in the grid is bounded based on the allowed steady-state frequency deviation after disturbances. The problem of finding the optimal distributions can be considered as combinatorial problem which consists of high dimensional solutions. In this light, we develop two numeric approaches based on Brute-force search and adaptation of the Cross-entropy method for finding the best distribution and examined it on a case study of the future Israeli grid. The results on the case study provide a new insight---the storage units should be placed around the area of the disturbances, including in sites with high inertia in accordance to the network topology.
Vision-Language Pre-Training with Triple Contrastive Learning
Authors: Jinyu Yang, Jiali Duan, Son Tran, Yi Xu, Sampath Chanda, Liqun Chen, Belinda Zeng, Trishul Chilimbi, Junzhou Huang
Date: 2022-02-21 00:00:00
Proceeding: CVPR 2022 1
Tasks: Contrastive Learning, cross-modal alignment, Cross-Modal Retrieval, Image-text Retrieval, Question Answering, Representation Learning, Retrieval, Text Retrieval, Visual Question Answering, Visual Question Answering (VQA), Zero-Shot Cross-Modal Retrieval

Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attributed to its capability in maximizing the mutual information (MI) between an image and its matched text. However, simply performing cross-modal alignment (CMA) ignores data potential within each modality, which may result in degraded representations. For instance, although CMA-based models are able to map image-text pairs close together in the embedding space, they fail to ensure that similar inputs from the same modality stay close by. This problem can get even worse when the pre-training data is noisy. In this paper, we propose triple contrastive learning (TCL) for vision-language pre-training by leveraging both cross-modal and intra-modal self-supervision. Besides CMA, TCL introduces an intra-modal contrastive objective to provide complementary benefits in representation learning. To take advantage of localized and structural information from image and text input, TCL further maximizes the average MI between local regions of image/text and their global summary. To the best of our knowledge, ours is the first work that takes into account local structure information for multi-modality representation learning. Experimental evaluations show that our approach is competitive and achieves the new state of the art on various common down-stream vision-language tasks such as image-text retrieval and visual question answering.
Pseudo Numerical Methods for Diffusion Models on Manifolds
Authors: Luping Liu, Yi Ren, Zhijie Lin, Zhou Zhao
Date: 2022-02-20 00:00:00
Proceeding: ICLR 2022 4
Tasks: Denoising, Image Generation

Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples. However, DDPMs require hundreds to thousands of iterations to produce final samples. Several prior works have successfully accelerated DDPMs through adjusting the variance schedule (e.g., Improved Denoising Diffusion Probabilistic Models) or the denoising equation (e.g., Denoising Diffusion Implicit Models (DDIMs)). However, these acceleration methods cannot maintain the quality of samples and even introduce new noise at a high speedup rate, which limit their practicability. To accelerate the inference process while keeping the sample quality, we provide a fresh perspective that DDPMs should be treated as solving differential equations on manifolds. Under such a perspective, we propose pseudo numerical methods for diffusion models (PNDMs). Specifically, we figure out how to solve differential equations on manifolds and show that DDIMs are simple cases of pseudo numerical methods. We change several classical numerical methods to corresponding pseudo numerical methods and find that the pseudo linear multi-step method is the best in most situations. According to our experiments, by directly using pre-trained models on Cifar10, CelebA and LSUN, PNDMs can generate higher quality synthetic images with only 50 steps compared with 1000-step DDIMs (20x speedup), significantly outperform DDIMs with 250 steps (by around 0.4 in FID) and have good generalization on different variance schedules. Our implementation is available at https://github.com/luping-liu/PNDM.
ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond
Authors: Qiming Zhang, Yufei Xu, Jing Zhang, DaCheng Tao
Date: 2022-02-21 00:00:00
Tasks: Image Classification, Inductive Bias

Vision transformers have shown great potential in various computer vision tasks owing to their strong capability to model long-range dependency using the self-attention mechanism. Nevertheless, they treat an image as a 1D sequence of visual tokens, lacking an intrinsic inductive bias (IB) in modeling local visual structures and dealing with scale variance, which is instead learned implicitly from large-scale training data with longer training schedules. In this paper, we propose a Vision Transformer Advanced by Exploring intrinsic IB from convolutions, i.e., ViTAE. Technically, ViTAE has several spatial pyramid reduction modules to downsample and embed the input image into tokens with rich multi-scale context using multiple convolutions with different dilation rates. In this way, it acquires an intrinsic scale invariance IB and can learn robust feature representation for objects at various scales. Moreover, in each transformer layer, ViTAE has a convolution block parallel to the multi-head self-attention module, whose features are fused and fed into the feed-forward network. Consequently, it has the intrinsic locality IB and is able to learn local features and global dependencies collaboratively. The proposed two kinds of cells are stacked in both isotropic and multi-stage manners to formulate two families of ViTAE models, i.e., the vanilla ViTAE and ViTAEv2. Experiments on the ImageNet dataset as well as downstream tasks on the MS COCO, ADE20K, and AP10K datasets validate the superiority of our models over the baseline transformer models and concurrent works. Besides, we scale up our ViTAE model to 644M parameters and obtain the state-of-the-art classification performance, i.e., 88.5% Top-1 classification accuracy on ImageNet validation set and the best 91.2% Top-1 accuracy on ImageNet real validation set, without using extra private data.
Reward Modeling for Mitigating Toxicity in Transformer-based Language Models
Authors: Farshid Faal, Ketra Schmitt, Jia Yuan Yu
Date: 2022-02-19 00:00:00
Tasks: Language Modeling, Language Modelling, Text Generation

Transformer-based language models are able to generate fluent text and be efficiently adapted across various natural language generation tasks. However, language models that are pretrained on large unlabeled web text corpora have been shown to suffer from degenerating toxic content and social bias behaviors, consequently hindering their safe deployment. Various detoxification methods were proposed to mitigate the language model's toxicity; however, these methods struggled to detoxify language models when conditioned on prompts that contain specific social identities related to gender, race, or religion. In this study, we propose Reinforce-Detoxify; A reinforcement learning-based method for mitigating toxicity in language models. We address the challenge of safety in language models and propose a new reward model that is able to detect toxic content and mitigate unintended bias towards social identities in toxicity prediction. The experiments demonstrate that the Reinforce-Detoxify method for language model detoxification outperforms existing detoxification approaches in automatic evaluation metrics, indicating the ability of our approach in language model detoxification and less prone to unintended bias toward social identities in generated content.
Resolution Strategies for Networking the IoT at the Edge via Named Functions
Authors: Christopher Scherb
Date: 2019-01-12 00:00:00
Proceeding: 15th IEEE Annual Consumer Communications & Networking Conference (CCNC) 2019 1
Tasks: Cloud Computing, Edge-computing

Named Function Networking (NFN) is an extension for Information Centric Networking (ICN) to execute computation inside the Network. Thereby, NFN consists of two contributions: A workflow definition and a resolution strategy. The ICN communication model enables NFN to reuse already computed results by using the network s content store. To resolve a computation, NFN first tries to find a cached result and only if no result was found, the computation is executed so-called FoX (Find or Execute) resolution strategy. Initially, NFN was optimized for cloud computing and data center computing next to the big data objects. However, new trends like IoT and edge computing describe other requirements to network computations (e.g. dynamic computations of automotive services at the edge). In this paper, we explore how the resolution strategy can be modified to fit to IoT scenarios without changing the workflow definition. Based on two exemplary IoT use  cases (home automation and automotive IOT), new resolution strategies will be presented.
Missing Data Infill with Automunge
Authors: Nicholas J. Teague
Date: 2022-02-19 00:00:00
Tasks: Imputation

Missing data is a fundamental obstacle in the practice of data science. This paper surveys a few conventions for imputation as available in the Automunge open source python library platform for tabular data preprocessing, including "ML infill" in which auto ML models are trained for target features from partitioned extracts of a training set. A series of validation experiments were performed to benchmark imputation scenarios towards downstream model performance, in which it was found for the given benchmark sets that in many cases ML infill outperformed for both numeric and categoric target features, and was otherwise at minimum within noise distributions of the other imputation scenarios. Evidence also suggested supplementing ML infill with the addition of support columns with boolean integer markers signaling presence of infill was usually beneficial to downstream model performance. We consider these results sufficient to recommend defaulting to ML infill for tabular learning, and further recommend supplementing imputations with support columns signaling presence of infill, each as can be prepared with push-button operation in the Automunge library. Our contributions include an auto ML derived missing data imputation library for tabular learning in the python ecosystem, fully integrated into a preprocessing platform with an extensive library of feature transformations, with a novel production friendly implementation that bases imputation models on a designated train set for consistent basis towards additional data.
PointSCNet: Point Cloud Structure and Correlation Learning Based on Space Filling Curve-Guided Sampling
Authors: Xingye Chen, Yiqi Wu, Wenjie Xu, Jin Li, Huaiyi Dong, Yilin Chen
Date: 2022-02-21 00:00:00
Tasks: 3D Point Cloud Classification, Semantic Segmentation

Geometrical structures and the internal local region relationship, such as symmetry, regular array, junction, etc., are essential for understanding a 3D shape. This paper proposes a point cloud feature extraction network named PointSCNet, to capture the geometrical structure information and local region correlation information of a point cloud. The PointSCNet consists of three main modules: the space-filling curve-guided sampling module, the information fusion module, and the channel-spatial attention module. The space-filling curve-guided sampling module uses Z-order curve coding to sample points that contain geometrical correlation. The information fusion module uses a correlation tensor and a set of skip connections to fuse the structure and correlation information. The channel-spatial attention module enhances the representation of key points and crucial feature channels to refine the network. The proposed PointSCNet is evaluated on shape classification and part segmentation tasks. The experimental results demonstrate that the PointSCNet outperforms or is on par with state-of-the-art methods by learning the structure and correlation of point clouds effectively.
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube
Authors: Aravind Sivakumar, Kenneth Shaw, Deepak Pathak
Date: 2022-02-21 00:00:00

We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired human-robot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots in learning to act autonomously in the real world. Videos at https://robotic-telekinesis.github.io/
Transformer Quality in Linear Time
Authors: Weizhe Hua, Zihang Dai, Hanxiao Liu, Quoc V. Le
Date: 2022-02-21 00:00:00
Tasks: 8k, Language Modeling, Language Modelling, Masked Language Modeling

We revisit the design choices in Transformers, and propose methods to address their weaknesses in handling long sequences. First, we propose a simple layer named gated attention unit, which allows the use of a weaker single-head attention with minimal quality loss. We then propose a linear approximation method complementary to this new layer, which is accelerator-friendly and highly competitive in quality. The resulting model, named FLASH, matches the perplexity of improved Transformers over both short (512) and long (8K) context lengths, achieving training speedups of up to 4.9$\times$ on Wiki-40B and 12.1$\times$ on PG-19 for auto-regressive language modeling, and 4.8$\times$ on C4 for masked language modeling.
Survey on Large Scale Neural Network Training
Authors: Julia Gusak, Daria Cherniuk, Alena Shilova, Alexander Katrutsa, Daniel Bershatsky, Xunyi Zhao, Lionel Eyraud-Dubois, Oleg Shlyazhko, Denis Dimitrov, Ivan Oseledets, Olivier Beaumont
Date: 2022-02-21 00:00:00
Tasks: GPU, Survey

Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training. Hence, many models do not fit one GPU device or can be trained using only a small per-GPU batch size. This survey provides a systematic overview of the approaches that enable more efficient DNNs training. We analyze techniques that save memory and make good use of computation and communication resources on architectures with a single or several GPUs. We summarize the main categories of strategies and compare strategies within and across categories. Along with approaches proposed in the literature, we discuss available implementations.