pwc-archive/papers-with-abstracts huggingface dataset:
SPot: A tool for identifying operating segments in financial tables
Authors: Zhiqiang Ma, Steven Pomerville, Mingyang Di, Armineh Nourbakhsh
Date: 2020-05-17 00:00:00
Tasks: Benchmarking

In this paper we present SPot, an automated tool for detecting operating segments and their related performance indicators from earnings reports. Due to their company-specific nature, operating segments cannot be detected using taxonomy-based approaches. Instead, we train a Bidirectional RNN classifier that can distinguish between common metrics such as "revenue" and company-specific metrics that are likely to be operating segments, such as "iPhone" or "cloud services". SPot surfaces the results in an interactive web interface that allows users to trace and adjust performance metrics for each operating segment. This facilitates credit monitoring, enables them to perform competitive benchmarking more effectively, and can be used for trend analysis at company and sector levels.
Learning to rank music tracks using triplet loss
Authors: Laure Prétet, Gaël Richard, Geoffroy Peeters
Date: 2020-05-18 00:00:00
Tasks: Learning-To-Rank, Triplet

Most music streaming services rely on automatic recommendation algorithms to exploit their large music catalogs. These algorithms aim at retrieving a ranked list of music tracks based on their similarity with a target music track. In this work, we propose a method for direct recommendation based on the audio content without explicitly tagging the music tracks. To that aim, we propose several strategies to perform triplet mining from ranked lists. We train a Convolutional Neural Network to learn the similarity via triplet loss. These different strategies are compared and validated on a large-scale experiment against an auto-tagging based approach. The results obtained highlight the efficiency of our system, especially when associated with an Auto-pooling layer.
Utilizing FastText for Venue Recommendation
Authors: Makbule Gulcin Ozsoy
Date: 2020-05-14 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Venue recommendation systems model the past interactions (i.e., check-ins) of the users and recommend venues. Traditional recommendation systems employ collaborative filtering, content-based filtering or matrix factorization. Recently, vector space embedding and deep learning algorithms are also used for recommendation. In this work, I propose a method for recommending top-k venues by utilizing the sequentiality feature of check-ins and a recent vector space embedding method, namely the FastText. Our proposed method; forms groups of check-ins, learns the vector space representations of the venues and utilizes the learned embeddings to make venue recommendations. I measure the performance of the proposed method using a Foursquare check-in dataset.The results show that the proposed method performs better than the state-of-the-art methods.
Evaluating Information Retrieval Systems for Kids
Authors: Ashlee Milton, Maria Soledad Pera
Date: 2020-05-21 00:00:00
Tasks: Information Retrieval, Position, Retrieval

Evaluation of information retrieval systems (IRS) is a prominent topic among information retrieval researchers--mainly directed at a general population. Children require unique IRS and by extension different ways to evaluate these systems, but as a large population that use IRS have largely been ignored on the evaluation front. In this position paper, we explore many perspectives that must be considered when evaluating IRS; we specially discuss problems faced by researchers who work with children IRS, including lack of evaluation frameworks, limitations of data, and lack of user judgment understanding.
Deep learning method to remove chemical, kinetic and electric artifacts on ISEs
Authors: Byunghyun Ban
Date: 2020-05-20 00:00:00

We suggest a deep learning based sensor signal processing method to remove chemical, kinetic and electrical artifacts from ion selective electrodes' measured values. An ISE is used to investigate the concentration of a specific ion from aqueous solution, by measuring the Nernst potential along the glass membrane. However, application of ISE on a mixture of multiple ion has some problem. First problem is a chemical artifact which is called ion interference effect. Electrically charged particles interact with each other and flows through the glass membrane of different ISEs. Second problem is the kinetic artifact caused by the movement of the liquid. Water molecules collide with the glass membrane causing abnormal peak values of voltage. The last artifact is the interference of ISEs. When multiple ISEs are dipped into same solution, one electrode's signal emission interference voltage measurement of other electrodes. Therefore, an ISE is recommended to be applied on single-ion solution, without any other sensors applied at the same time. Deep learning approach can remove both 3 artifacts at the same time. The proposed method used 5 layers of artificial neural networks to regress correct signal to remove complex artifacts with one-shot calculation. Its MAPE was less than 1.8% and R2 of regression was 0.997. A randomly chosen value of AI-processed data has MAPE less than 5% (p-value 0.016).
Q-NAV: NAV Setting Method based on Reinforcement Learning in Underwater Wireless Networks
Authors: Seok-Hyeon Park, Ohyun Jo
Date: 2020-05-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

The demand on the underwater communications is extremely increasing in searching for underwater resources, marine expedition, or environmental researches, yet there are many problems with the wireless communications because of the characteristics of the underwater environments. Especially, with the underwater wireless networks, there happen inevitable delay time and spacial inequality due to the distances between the nodes. To solve these problems, this paper suggests a new solution based on ALOHA-Q. The suggested method use random NAV value. and Environments take reward through communications success or fail. After then, The environments setting NAV value from reward. This model minimizes usage of energy and computing resources under the underwater wireless networks, and learns and setting NAV values through intense learning. The results of the simulations show that NAV values can be environmentally adopted and select best value to the circumstances, so the problems which are unnecessary delay times and spacial inequality can be solved. Result of simulations, NAV time decreasing 17.5% compared with original NAV.
Color Point Cloud Registration Based on Supervoxel Correspondence
Authors: YANG YANG, WEILE CHEN, Muyi Wang, DEXING ZHONG, Shaoyi Du
Date: 2020-01-06 00:00:00
Proceeding: IEEE Access 2020 1
Tasks: Point Cloud Registration

With the development of RGBD sensors, the high-quality color point cloud can be obtained
expediently. In this paper, we propose a novel registration method for 3D color point clouds from different
views, which is a critical issue in many applications. Different from traditional feature-based methods,
we design a hybrid feature representation with color moments of the point, which could be applied
naturally for any color point cloud. And these features are extracted from point clouds based on the
supervoxel segmentation. By jointly conducting these features for similarity measure, a weight parameter
is dynamically adapted between the color and the spatial information. The registration algorithm is under
a classic iterative framework for building the correspondence and estimating transformation parameters.
In addition, we provide a mutual correspondence matching condition with hybrid features to build some
more robust relationships for estimating transformation parameters. Experimental results demonstrate that
our method can effectively reduce the number of point data for registration and achieve good matching results
even in a poor initial condition.
Malliavin-Mancino estimators implemented with non-uniform fast Fourier transforms
Authors: Patrick Chang, Etienne Pienaar, Tim Gebbie
Date: 2020-03-05 00:00:00
Tasks: Benchmarking

We implement and test kernel averaging Non-Uniform Fast Fourier Transform (NUFFT) methods to enhance the performance of correlation and covariance estimation on asynchronously sampled event-data using the Malliavin-Mancino Fourier estimator. The methods are benchmarked for Dirichlet and Fej\'{e}r Fourier basis kernels. We consider test cases formed from Geometric Brownian motions to replicate synchronous and asynchronous data for benchmarking purposes. We consider three standard averaging kernels to convolve the event-data for synchronisation via over-sampling for use with the Fast Fourier Transform (FFT): the Gaussian kernel, the Kaiser-Bessel kernel, and the exponential of semi-circle kernel. First, this allows us to demonstrate the performance of the estimator with different combinations of basis kernels and averaging kernels. Second, we investigate and compare the impact of the averaging scales explicit in each averaging kernel and its relationship between the time-scale averaging implicit in the Malliavin-Mancino estimator. Third, we demonstrate the relationship between time-scale averaging based on the number of Fourier coefficients used in the estimator to a theoretical model of the Epps effect. We briefly demonstrate the methods on Trade-and-Quote (TAQ) data from the Johannesburg Stock Exchange to make an initial visualisation of the correlation dynamics for various time-scales under market microstructure.
Parameter Continuation Methods for the Optimization of Deep Neural Networks
Authors: Harsh Nilesh Pathank, Randy Clinton Paffenroth
Date: 2019-12-16 00:00:00
Proceeding: 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA) 2019 12

There are many extant methods for approximating the solutions of non-convex optimization problems arising in deep neural networks, including stochastic gradient descent, RMSProp, AdaGrad, and ADAM. In this paper, we propose a novel training strategy for deep neural networks based on ideas from numerical parameter continuation methods. Parameter continuation methods have a long history in many application domains such as bifurcation analysis and the study of systems of differential equations. However, as far as we are aware, such powerful methods have seen relatively limited use in the optimization of deep neural networks. Specifically, herein we derive a homotopy formulation of standard activation functions. Such a derivation allows one to decompose the optimization of deep neural networks into a sequence of optimization problems, each of which is armed with a good initial guess based upon the solution of the previous problem. The whole process is initiated by using a closed-form solution for the first of these problems provided by the homotopy formulation. Intuitively, there is a deep connection between our homotopy techniques and many ideas used in transfer and curriculum learning. However, our proposed methods leverage decades of theoretical and computational work in parameter continuation methods and can be viewed as an initial bridge between those techniques and deep neural networks. In particular, we propose a method that we call Natural Parameter Adaption Continuation with Secant approximation (NPACS). This method provides an effective optimization technique that uses standard algorithms such as ADAM in a novel way to achieve faster and more stable convergence. We demonstrate the effectiveness of our method on standard benchmark problems, and we compute local minima more quickly and with lower train and test loss values than current state-of-the-art techniques in a majority of cases.
Spatial Resolution of Local Field Potential Signals in Macaque V4
Authors: Armin Najarpour Foroushani, Sujaya Neupane, Pablo De Heredia Pastor, Christopher C. Pack, Mohamad Sawan
Date: 2019-11-18 00:00:00

A main challenge for the development of cortical visual prostheses is to spatially localize individual spots of light, called phosphenes, by assigning appropriate stimulating parameters to implanted electrodes. Imitating the natural responses to phosphene-like stimuli at different positions can help in designing a systematic procedure to determine these parameters. The key characteristic of such a system is the ability to discriminate between responses to different positions in the visual field. While most previous prosthetic devices have targeted the primary visual cortex, the extrastriate cortex has the advantage of covering a large part of the visual field with a smaller amount of cortical tissue, providing the possibility of a more compact implant. Here, we studied how well ensembles of Multiunit activity (MUA) and Local Field Potentials (LFPs) responses from extrastriate cortical visual area V4 of a behaving macaque monkey can discriminate between two-dimensional spatial positions. We found that despite the large receptive field sizes in V4, the combined responses from multiple sites, whether MUA or LFP, has the capability for fine and coarse discrimination of positions. We identified a selection procedure that could significantly increase the discrimination performance while reducing the required number of electrodes. Analysis of noise correlation in MUA and LFP responses showed that noise correlations in LFP responses carry more information about the spatial positions. Overall, these findings suggest that spatial positions could be localized with patterned stimulation in extrastriate area V4.
Detecting Anomalies in Image Classification by Means of Semantic Relationships
Authors: Andrea Pasini, Elena Baralis
Date: 2019-06-01 00:00:00
Proceeding: IEEE AIKE 2019 6
Tasks: Anomaly Detection, General Classification, image-classification, Image Classification, Semantic Segmentation

This paper presents a semantic anomaly detection method (SAD) to detect anomalies in the predictions of any
pixelwise semantic segmentation algorithm. This semantic information (e.g., relative positions and sizes of all the object pairs in an image), learned from the training set and stored in a knowledge base as configuration rules, allows the detection of potential misclassifications in the baseline model predictions.
Our approach highlights the objects which are not consistent with the contextual information in the knowledge base. It also provides an interpretable motivation for the detected anomaly, based on the semantic information provided by the configuration rules.
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
Authors: Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, Ligeng Zhu, Chuang Gan, Song Han
Date: 2020-05-28 00:00:00
Proceeding: ACL 2020 6
Tasks: CPU, Decoder, GPU, Machine Translation, Neural Architecture Search, Raspberry Pi 4, Translation

Transformers are ubiquitous in Natural Language Processing (NLP) tasks, but they are difficult to be deployed on hardware due to the intensive computation. To enable low-latency inference on resource-constrained hardware platforms, we propose to design Hardware-Aware Transformers (HAT) with neural architecture search. We first construct a large design space with $\textit{arbitrary encoder-decoder attention}$ and $\textit{heterogeneous layers}$. Then we train a $\textit{SuperTransformer}$ that covers all candidates in the design space, and efficiently produces many $\textit{SubTransformers}$ with weight sharing. Finally, we perform an evolutionary search with a hardware latency constraint to find a specialized $\textit{SubTransformer}$ dedicated to run fast on the target hardware. Extensive experiments on four machine translation tasks demonstrate that HAT can discover efficient models for different hardware (CPU, GPU, IoT device). When running WMT'14 translation task on Raspberry Pi-4, HAT can achieve $\textbf{3}\times$ speedup, $\textbf{3.7}\times$ smaller size over baseline Transformer; $\textbf{2.7}\times$ speedup, $\textbf{3.6}\times$ smaller size over Evolved Transformer with $\textbf{12,041}\times$ less search cost and no performance loss. HAT code is https://github.com/mit-han-lab/hardware-aware-transformers.git
Bayesian Restoration of Audio Degraded by Low-Frequency Pulses Modeled via Gaussian Process
Authors: Hugo Tremonte de Carvalho, Flávio Rainho Ávila, Luiz Wagner Pereira Biscainho
Date: 2020-05-28 00:00:00

A common defect found when reproducing old vinyl and gramophone recordings with mechanical devices are the long pulses with significant low-frequency content caused by the interaction of the arm-needle system with deep scratches or even breakages on the media surface. Previous approaches to their suppression on digital counterparts of the recordings depend on a prior estimation of the pulse location, usually performed via heuristic methods. This paper proposes a novel Bayesian approach capable of jointly estimating the pulse location; interpolating the almost annihilated signal underlying the strong discontinuity that initiates the pulse; and also estimating the long pulse tail by a simple Gaussian Process, allowing its suppression from the corrupted signal. The posterior distribution for the model parameters as well for the pulse is explored via Markov-Chain Monte Carlo (MCMC) algorithms. Controlled experiments indicate that the proposed method, while requiring significantly less user intervention, achieves perceptual results similar to those of previous approaches and performs well when dealing with naturally degraded signals.
User Behavior Retrieval for Click-Through Rate Prediction
Authors: Jiarui Qin, Wei-Nan Zhang, Xin Wu, Jiarui Jin, Yuchen Fang, Yong Yu
Date: 2020-05-28 00:00:00
Tasks: Click-Through Rate Prediction, Prediction, Retrieval

Click-through rate (CTR) prediction plays a key role in modern online personalization services. In practice, it is necessary to capture user's drifting interests by modeling sequential user behaviors to build an accurate CTR prediction model. However, as the users accumulate more and more behavioral data on the platforms, it becomes non-trivial for the sequential models to make use of the whole behavior history of each user. First, directly feeding the long behavior sequence will make online inference time and system load infeasible. Second, there is much noise in such long histories to fail the sequential model learning. The current industrial solutions mainly truncate the sequences and just feed recent behaviors to the prediction model, which leads to a problem that sequential patterns such as periodicity or long-term dependency are not embedded in the recent several behaviors but in far back history. To tackle these issues, in this paper we consider it from the data perspective instead of just designing more sophisticated yet complicated models and propose User Behavior Retrieval for CTR prediction (UBR4CTR) framework. In UBR4CTR, the most relevant and appropriate user behaviors will be firstly retrieved from the entire user history sequence using a learnable search method. These retrieved behaviors are then fed into a deep model to make the final prediction instead of simply using the most recent ones. It is highly feasible to deploy UBR4CTR into industrial model pipeline with low cost. Experiments on three real-world large-scale datasets demonstrate the superiority and efficacy of our proposed framework and models.
Self-supervised Modal and View Invariant Feature Learning
Authors: Longlong Jing, Yu-cheng Chen, Ling Zhang, Mingyi He, YingLi Tian
Date: 2020-05-28 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval

Most of the existing self-supervised feature learning methods for 3D data either learn 3D features from point cloud data or from multi-view images. By exploring the inherent multi-modality attributes of 3D objects, in this paper, we propose to jointly learn modal-invariant and view-invariant features from different modalities including image, point cloud, and mesh with heterogeneous networks for 3D data. In order to learn modal- and view-invariant features, we propose two types of constraints: cross-modal invariance constraint and cross-view invariant constraint. Cross-modal invariance constraint forces the network to maximum the agreement of features from different modalities for same objects, while the cross-view invariance constraint forces the network to maximum agreement of features from different views of images for same objects. The quality of learned features has been tested on different downstream tasks with three modalities of data including point cloud, multi-view images, and mesh. Furthermore, the invariance cross different modalities and views are evaluated with the cross-modal retrieval task. Extensive evaluation results demonstrate that the learned features are robust and have strong generalizability across different tasks.
Language Models are Few-Shot Learners
Authors: Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei
Date: 2020-05-28 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: answerability prediction, Articles, Common Sense Reasoning, Coreference Resolution, Domain Adaptation, Few-Shot Learning, Language Modeling, Language Modelling, Multi-task Language Understanding, Multi-Task Learning, Natural Language Inference, Question Answering, Reading Comprehension, Sentence, Sentence Completion, Unsupervised Machine Translation, Word Sense Disambiguation, Zero-Shot Learning

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general.