pwc-archive/papers-with-abstracts huggingface dataset:
Multimodal Meta-Learning for Time Series Regression
Authors: Sebastian Pineda Arango, Felix Heinrich, Kiran Madhusudhanan, Lars Schmidt-Thieme
Date: 2021-08-05 00:00:00
Tasks: Meta-Learning, regression, Time Series, Time Series Analysis, Time Series Regression

Recent work has shown the efficiency of deep learning models such as Fully Convolutional Networks (FCN) or Recurrent Neural Networks (RNN) to deal with Time Series Regression (TSR) problems. These models sometimes need a lot of data to be able to generalize, yet the time series are sometimes not long enough to be able to learn patterns. Therefore, it is important to make use of information across time series to improve learning. In this paper, we will explore the idea of using meta-learning for quickly adapting model parameters to new short-history time series by modifying the original idea of Model Agnostic Meta-Learning (MAML) \cite{finn2017model}. Moreover, based on prior work on multimodal MAML \cite{vuorio2019multimodal}, we propose a method for conditioning parameters of the model through an auxiliary network that encodes global information of the time series to extract meta-features. Finally, we apply the data to time series of different domains, such as pollution measurements, heart-rate sensors, and electrical battery data. We show empirically that our proposed meta-learning method learns TSR with few data fast and outperforms the baselines in 9 of 12 experiments.
Two-Stage Sector Rotation Methodology Using Machine Learning and Deep Learning Techniques
Authors: Tugce Karatas, Ali Hirsa
Date: 2021-08-05 00:00:00
Tasks: BIG-bench Machine Learning

Market indicators such as CPI and GDP have been widely used over decades to identify the stage of business cycles and also investment attractiveness of sectors given market conditions. In this paper, we propose a two-stage methodology that consists of predicting ETF prices for each sector using market indicators and ranking sectors based on their predicted rate of returns. We initially start with choosing sector specific macroeconomic indicators and implement Recursive Feature Elimination algorithm to select the most important features for each sector. Using our prediction tool, we implement different Recurrent Neural Networks models to predict the future ETF prices for each sector. We then rank the sectors based on their predicted rate of returns. We select the best performing model by evaluating the annualized return, annualized Sharpe ratio, and Calmar ratio of the portfolios that includes the top four ranked sectors chosen by the model. We also test the robustness of the model performance with respect to lookback windows and look ahead windows. Our empirical results show that our methodology beats the equally weighted portfolio performance even in the long run. We also find that Echo State Networks exhibits an outstanding performance compared to other models yet it is faster to implement compared to other RNN models.
Lossless Multi-Scale Constitutive Elastic Relations with Artificial Intelligence
Authors: Jaber Rezaei Mianroodi, Shahed Rezaei, Nima H. Siboni, Bai-Xiang Xu, Dierk Raabe
Date: 2021-08-05 00:00:00

The elastic properties of materials derive from their electronic and atomic nature. However, simulating bulk materials fully at these scales is not feasible, so that typically homogenized continuum descriptions are used instead. A seamless and lossless transition of the constitutive description of the elastic response of materials between these two scales has been so far elusive. Here we show how this problem can be overcome by using Artificial Intelligence (AI). A Convolutional Neural Network (CNN) model is trained, by taking the structure image of a nanoporous material as input and the corresponding elasticity tensor, calculated from Molecular Statics (MS), as output. Trained with the atomistic data, the CNN model captures the size- and pore-dependency of the material's elastic properties which, on the physics side, can stem from surfaces and non-local effects. Such effects are often ignored in upscaling from atomistic to classical continuum theory. To demonstrate the accuracy and the efficiency of the trained CNN model, a Finite Element Method (FEM) based result of an elastically deformed nanoporous beam equipped with the CNN as constitutive law is compared with that by a full atomistic simulation. The good agreement between the atomistic simulations and the FEM-AI combination for a system with size and surface effects establishes a new lossless scale bridging approach to such problems. The trained CNN model deviates from the atomistic result by 9.6\% for porosity scenarios of up to 90\% but it is about 230 times faster than the MS calculation and does not require to change simulation methods between different scales. The efficiency of the CNN evaluation together with the preservation of important atomistic effects makes the trained model an effective atomistically-informed constitutive model for macroscopic simulations of nanoporous materials and solving of inverse problems.
Efficient recurrent neural network methods for anomalously diffusing single particle short and noisy trajectories
Authors: Òscar Garibo i Orts, Miguel A. Garcia-March, J. Alberto Conejero
Date: 2021-08-05 00:00:00

Anomalous diffusion occurs at very different scales in nature, from atomic systems to motions in cell organelles, biological tissues or ecology, and also in artificial materials, such as cement. Being able to accurately measure the anomalous exponent associated with a given particle trajectory, thus determining whether the particle subdiffuses, superdiffuses or performs normal diffusion is of key importance to understand the diffusion process. Also, it is often important to trustingly identify the model behind the trajectory, as this gives a large amount of information on the system dynamics. Both aspects are particularly difficult when the input data are short and noisy trajectories. It is even more difficult if one cannot guarantee that the trajectories output in experiments is homogeneous, hindering the statistical methods based on ensembles of trajectories. We present a data-driven method able to infer the anomalous exponent and to identify the type of anomalous diffusion process behind single, noisy and short trajectories, with good accuracy. This model was used in our participation in the Anomalous Diffusion (AnDi) Challenge. A combination of convolutional and recurrent neural networks were used to achieve state-of-the-art results when compared to methods participating in the AnDi Challenge, ranking top 4 in both classification and diffusion exponent regression.
Differentially Private n-gram Extraction
Authors: Kunho Kim, Sivakanth Gopi, Janardhan Kulkarni, Sergey Yekhanin
Date: 2021-08-05 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Response Generation, Sentence, Sentence Completion

We revisit the problem of $n$-gram extraction in the differential privacy setting. In this problem, given a corpus of private text data, the goal is to release as many $n$-grams as possible while preserving user level privacy. Extracting $n$-grams is a fundamental subroutine in many NLP applications such as sentence completion, response generation for emails etc. The problem also arises in other applications such as sequence mining, and is a generalization of recently studied differentially private set union (DPSU). In this paper, we develop a new differentially private algorithm for this problem which, in our experiments, significantly outperforms the state-of-the-art. Our improvements stem from combining recent advances in DPSU, privacy accounting, and new heuristics for pruning in the tree-based approach initiated by Chen et al. (2012).
An Elementary Proof that Q-learning Converges Almost Surely
Authors: Matthew T. Regehr, Alex Ayoub
Date: 2021-08-05 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Watkins' and Dayan's Q-learning is a model-free reinforcement learning algorithm that iteratively refines an estimate for the optimal action-value function of an MDP by stochastically "visiting" many state-ation pairs [Watkins and Dayan, 1992]. Variants of the algorithm lie at the heart of numerous recent state-of-the-art achievements in reinforcement learning, including the superhuman Atari-playing deep Q-network [Mnih et al., 2015]. The goal of this paper is to reproduce a precise and (nearly) self-contained proof that Q-learning converges. Much of the available literature leverages powerful theory to obtain highly generalizable results in this vein. However, this approach requires the reader to be familiar with and make many deep connections to different research areas. A student seeking to deepen their understand of Q-learning risks becoming caught in a vicious cycle of "RL-learning Hell". For this reason, we give a complete proof from start to finish using only one external result from the field of stochastic approximation, despite the fact that this minimal dependence on other results comes at the expense of some "shininess".
THALIS: Human-Machine Analysis of Longitudinal Symptoms in Cancer Therapy
Authors: Carla Floricel, Nafiul Nipu, Mikayla Biggs, Andrew Wentzel, Guadalupe Canahuate, Lisanne van Dijk, Abdallah Mohamed, C. David Fuller, G. Elisabeta Marai
Date: 2021-08-05 00:00:00
Tasks: Diagnostic

Although cancer patients survive years after oncologic therapy, they are plagued with long-lasting or permanent residual symptoms, whose severity, rate of development, and resolution after treatment vary largely between survivors. The analysis and interpretation of symptoms is complicated by their partial co-occurrence, variability across populations and across time, and, in the case of cancers that use radiotherapy, by further symptom dependency on the tumor location and prescribed treatment. We describe THALIS, an environment for visual analysis and knowledge discovery from cancer therapy symptom data, developed in close collaboration with oncology experts. Our approach leverages unsupervised machine learning methodology over cohorts of patients, and, in conjunction with custom visual encodings and interactions, provides context for new patients based on patients with similar diagnostic features and symptom evolution. We evaluate this approach on data collected from a cohort of head and neck cancer patients. Feedback from our clinician collaborators indicates that THALIS supports knowledge discovery beyond the limits of machines or humans alone, and that it serves as a valuable tool in both the clinic and symptom research.
Potential Applications of Artificial Intelligence and Machine Learning in Radiochemistry and Radiochemical Engineering
Authors: E. William Webb, Peter J. H. Scott
Date: 2021-08-05 00:00:00
Tasks: BIG-bench Machine Learning

Artificial intelligence and machine learning are poised to disrupt PET imaging from bench to clinic. In this perspective we offer insights into how the technology could be applied to improve the design and synthesis of new radiopharmaceuticals for PET imaging, including identification of an optimal labeling approach as well as strategies for radiolabeling reaction optimization.
Quantum Topological Data Analysis with Linear Depth and Exponential Speedup
Authors: Shashanka Ubaru, Ismail Yunus Akhalwaya, Mark S. Squillante, Kenneth L. Clarkson, Lior Horesh
Date: 2021-08-05 00:00:00
Tasks: Quantum Machine Learning, Topological Data Analysis

Quantum computing offers the potential of exponential speedups for certain classical computations. Over the last decade, many quantum machine learning (QML) algorithms have been proposed as candidates for such exponential improvements. However, two issues unravel the hope of exponential speedup for some of these QML algorithms: the data-loading problem and, more recently, the stunning dequantization results of Tang et al. A third issue, namely the fault-tolerance requirements of most QML algorithms, has further hindered their practical realization. The quantum topological data analysis (QTDA) algorithm of Lloyd, Garnerone and Zanardi was one of the first QML algorithms that convincingly offered an expected exponential speedup. From the outset, it did not suffer from the data-loading problem. A recent result has also shown that the generalized problem solved by this algorithm is likely classically intractable, and would therefore be immune to any dequantization efforts. However, the QTDA algorithm of Lloyd et~al. has a time complexity of $O(n^4/(\epsilon^2 \delta))$ (where $n$ is the number of data points, $\epsilon$ is the error tolerance, and $\delta$ is the smallest nonzero eigenvalue of the restricted Laplacian) and requires fault-tolerant quantum computing, which has not yet been achieved. In this paper, we completely overhaul the QTDA algorithm to achieve an improved exponential speedup and depth complexity of $O(n\log(1/(\delta\epsilon)))$. Our approach includes three key innovations: (a) an efficient realization of the combinatorial Laplacian as a sum of Pauli operators; (b) a quantum rejection sampling approach to restrict the superposition to the simplices in the complex; and (c) a stochastic rank estimation method to estimate the Betti numbers. We present a theoretical error analysis, and the circuit and computational time and depth complexities for Betti number estimation.
Using Machine Learning to Predict Game Outcomes Based on Player-Champion Experience in League of Legends
Authors: Tiffany D. Do, Seong Ioi Wang, Dylan S. Yu, Matthew G. McMillian, Ryan P. McMahan
Date: 2021-08-05 00:00:00

League of Legends (LoL) is the most widely played multiplayer online battle arena (MOBA) game in the world. An important aspect of LoL is competitive ranked play, which utilizes a skill-based matchmaking system to form fair teams. However, players' skill levels vary widely depending on which champion, or hero, that they choose to play as. In this paper, we propose a method for predicting game outcomes in ranked LoL games based on players' experience with their selected champion. Using a deep neural network, we found that game outcomes can be predicted with 75.1% accuracy after all players have selected champions, which occurs before gameplay begins. Our results have important implications for playing LoL and matchmaking. Firstly, individual champion skill plays a significant role in the outcome of a match, regardless of team composition. Secondly, even after the skill-based matchmaking, there is still a wide variance in team skill before gameplay begins. Finally, players should only play champions that they have mastered, if they want to win games.
Quantum Continual Learning Overcoming Catastrophic Forgetting
Authors: Wenjie Jiang, Zhide Lu, Dong-Ling Deng
Date: 2021-08-05 00:00:00
Tasks: BIG-bench Machine Learning, Continual Learning, Incremental Learning, Quantum Machine Learning

Catastrophic forgetting describes the fact that machine learning models will likely forget the knowledge of previously learned tasks after the learning process of a new one. It is a vital problem in the continual learning scenario and recently has attracted tremendous concern across different communities. In this paper, we explore the catastrophic forgetting phenomena in the context of quantum machine learning. We find that, similar to those classical learning models based on neural networks, quantum learning systems likewise suffer from such forgetting problem in classification tasks emerging from various application scenes. We show that based on the local geometrical information in the loss function landscape of the trained model, a uniform strategy can be adapted to overcome the forgetting problem in the incremental learning setting. Our results uncover the catastrophic forgetting phenomena in quantum machine learning and offer a practical method to overcome this problem, which opens a new avenue for exploring potential quantum advantages towards continual learning.
Cross-lingual Capsule Network for Hate Speech Detection in Social Media
Authors: Aiqi Jiang, Arkaitz Zubiaga
Date: 2021-08-06 00:00:00
Tasks: Hate Speech Detection

Most hate speech detection research focuses on a single language, generally English, which limits their generalisability to other languages. In this paper we investigate the cross-lingual hate speech detection task, tackling the problem by adapting the hate speech resources from one language to another. We propose a cross-lingual capsule network learning model coupled with extra domain-specific lexical semantics for hate speech (CCNL-Ex). Our model achieves state-of-the-art performance on benchmark datasets from AMI@Evalita2018 and AMI@Ibereval2018 involving three languages: English, Spanish and Italian, outperforming state-of-the-art baselines on all six language pairs.
SWSR: A Chinese Dataset and Lexicon for Online Sexism Detection
Authors: Aiqi Jiang, Xiaohan Yang, Yang Liu, Arkaitz Zubiaga
Date: 2021-08-06 00:00:00
Tasks: Abusive Language

Online sexism has become an increasing concern in social media platforms as it has affected the healthy development of the Internet and can have negative effects in society. While research in the sexism detection domain is growing, most of this research focuses on English as the language and on Twitter as the platform. Our objective here is to broaden the scope of this research by considering the Chinese language on Sina Weibo. We propose the first Chinese sexism dataset -- Sina Weibo Sexism Review (SWSR) dataset --, as well as a large Chinese lexicon SexHateLex made of abusive and gender-related terms. We introduce our data collection and annotation process, and provide an exploratory analysis of the dataset characteristics to validate its quality and to show how sexism is manifested in Chinese. The SWSR dataset provides labels at different levels of granularity including (i) sexism or non-sexism, (ii) sexism category and (iii) target type, which can be exploited, among others, for building computational methods to identify and investigate finer-grained gender-related abusive language. We conduct experiments for the three sexism classification tasks making use of state-of-the-art machine learning models. Our results show competitive performance, providing a benchmark for sexism detection in the Chinese language, as well as an error analysis highlighting open challenges needing more research in Chinese NLP. The SWSR dataset and SexHateLex lexicon are publicly available.
Tell me a story about yourself: The words of shopping experience and self-satisfaction
Authors: L Petruzzellis, A Fronzetti Colladon, M Visentin, J. -C. Chebat
Date: 2021-08-06 00:00:00

In this paper we investigate the verbal expression of shopping experience obtained by a sample of customers asked to freely verbalize how they felt when entering a store. Using novel tools of Text Mining and Social Network Analysis, we analyzed the interviews to understand the connection between the emotions aroused during the shopping experience, satisfaction and the way participants link these concepts to self-satisfaction and self-identity. The results show a prominent role of emotions in the discourse about the shopping experience before purchasing and an inward-looking connection to the self. Our results also suggest that modern retail environment should enhance the hedonic shopping experience in terms of fun, fantasy, moods, and emotions.
Sentence Semantic Regression for Text Generation
Authors: Wei Wang, Piji Li, Hai-Tao Zheng
Date: 2021-08-06 00:00:00
Tasks: Decoder, Dialogue Generation, Language Modeling, Language Modelling, regression, Sentence, Text Generation

Recall the classical text generation works, the generation framework can be briefly divided into two phases: \textbf{idea reasoning} and \textbf{surface realization}. The target of idea reasoning is to figure out the main idea which will be presented in the following talking/writing periods. Surface realization aims to arrange the most appropriate sentence to depict and convey the information distilled from the main idea. However, the current popular token-by-token text generation methods ignore this crucial process and suffer from many serious issues, such as idea/topic drift. To tackle the problems and realize this two-phase paradigm, we propose a new framework named Sentence Semantic Regression (\textbf{SSR}) based on sentence-level language modeling. For idea reasoning, two architectures \textbf{SSR-AR} and \textbf{SSR-NonAR} are designed to conduct sentence semantic regression autoregressively (like GPT2/3) and bidirectionally (like BERT). In the phase of surface realization, a mixed-granularity sentence decoder is designed to generate text with better consistency by jointly incorporating the predicted sentence-level main idea as well as the preceding contextual token-level information. We conduct experiments on four tasks of story ending prediction, story ending generation, dialogue generation, and sentence infilling. The results show that SSR can obtain better performance in terms of automatic metrics and human evaluation.
LadRa-Net: Locally-Aware Dynamic Re-read Attention Net for Sentence Semantic Matching
Authors: Kun Zhang, Guangyi Lv, Le Wu, Enhong Chen, Qi Liu, Meng Wang
Date: 2021-08-06 00:00:00
Tasks: Language Modelling, Natural Language Inference, Paraphrase Identification, Sentence

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, which is widely used in various natural language tasks, such as Natural Language Inference (NLI), Paraphrase Identification (PI), and so on. Much recent progress has been made in this area, especially attention-based methods and pre-trained language model based methods. However, most of these methods focus on all the important parts in sentences in a static way and only emphasize how important the words are to the query, inhibiting the ability of attention mechanism. In order to overcome this problem and boost the performance of attention mechanism, we propose a novel dynamic re-read attention, which can pay close attention to one small region of sentences at each step and re-read the important parts for better sentence representations. Based on this attention variation, we develop a novel Dynamic Re-read Network (DRr-Net) for sentence semantic matching. Moreover, selecting one small region in dynamic re-read attention seems insufficient for sentence semantics, and employing pre-trained language models as input encoders will introduce incomplete and fragile representation problems. To this end, we extend DRrNet to Locally-Aware Dynamic Re-read Attention Net (LadRa-Net), in which local structure of sentences is employed to alleviate the shortcoming of Byte-Pair Encoding (BPE) in pre-trained language models and boost the performance of dynamic reread attention. Extensive experiments on two popular sentence semantic matching tasks demonstrate that DRr-Net can significantly improve the performance of sentence semantic matching. Meanwhile, LadRa-Net is able to achieve better performance by considering the local structures of sentences. In addition, it is exceedingly interesting that some discoveries in our experiments are consistent with some findings of psychological research.