pwc-archive/papers-with-abstracts huggingface dataset:
Analysis of three dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis
Authors: Hongwei Guo, Xiaoying Zhuang, Pengwan Chen, Naif Alajlan, Timon Rabczuk
Date: 2020-10-03 00:00:00
Tasks: Transfer Learning

In this work, we present a deep collocation method for three dimensional potential problems in nonhomogeneous media. This approach utilizes a physics informed neural network with material transfer learning reducing the solution of the nonhomogeneous partial differential equations to an optimization problem. We tested different cofigurations of the physics informed neural network including smooth activation functions, sampling methods for collocation points generation and combined optimizers. A material transfer learning technique is utilised for nonhomogeneous media with different material gradations and parameters, which enhance the generality and robustness of the proposed method. In order to identify the most influential parameters of the network configuration, we carried out a global sensitivity analysis. Finally, we provide a convergence proof of our DCM. The approach is validated through several benchmark problems, also testing different material variations.
Explaining Neural Network Predictions for Functional Data Using Principal Component Analysis and Feature Importance
Authors: Katherine Goode, Daniel Ries, Joshua Zollweg
Date: 2020-10-15 00:00:00
Tasks: BIG-bench Machine Learning, Feature Importance

Optical spectral-temporal signatures extracted from videos of explosions provide information for identifying characteristics of the corresponding explosive devices. Currently, the identification is done using heuristic algorithms and direct subject matter expert review. An improvement in predictive performance may be obtained by using machine learning, but this application lends itself to high consequence national security decisions, so it is not only important to provide high accuracy but clear explanations for the predictions to garner confidence in the model. While much work has been done to develop explainability methods for machine learning models, not much of the work focuses on situations with input variables of the form of functional data such optical spectral-temporal signatures. We propose a procedure for explaining machine learning models fit using functional data that accounts for the functional nature the data. Our approach makes use of functional principal component analysis (fPCA) and permutation feature importance (PFI). fPCA is used to transform the functions to create uncorrelated functional principal components (fPCs). The model is trained using the fPCs as inputs, and PFI is applied to identify the fPCs important to the model for prediction. Visualizations are used to interpret the variability explained by the fPCs that are found to be important by PFI to determine the aspects of the functions that are important for prediction. We demonstrate the technique by explaining neural networks fit to explosion optical spectral-temporal signatures for predicting characteristics of the explosive devices.
Learning Patterns in Imaginary Vowels for an Intelligent Brain Computer Interface (BCI) Design
Authors: Parisa Ghane, Gahangir Hossain
Date: 2020-10-17 00:00:00
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG), Motor Imagery, Multi-class Classification

Technology advancements made it easy to measure non-invasive and high-quality electroencephalograph (EEG) signals from human's brain. Hence, development of robust and high-performance AI algorithms becomes crucial to properly process the EEG signals and recognize the patterns, which lead to an appropriate control signal. Despite the advancements in processing the motor imagery EEG signals, the healthcare applications, such as emotion detection, are still in the early stages of AI design. In this paper, we propose a modular framework for the recognition of vowels as the AI part of a brain computer interface system. We carefully designed the modules to discriminate the English vowels given the raw EEG signals, and meanwhile avoid the typical issued with the data-poor environments like most of the healthcare applications. The proposed framework consists of appropriate signal segmentation, filtering, extraction of spectral features, reducing the dimensions by means of principle component analysis, and finally a multi-class classification by decision-tree-based support vector machine (DT-SVM). The performance of our framework was evaluated by a combination of test-set and resubstitution (also known as apparent) error rates. We provide the algorithms of the proposed framework to make it easy for future researchers and developers who want to follow the same workflow.
Stochastic analysis of heterogeneous porous material with modified neural architecture search (NAS) based physics-informed neural networks using transfer learning
Authors: Hongwei Guo, Xiaoying Zhuang, Timon Rabczuk
Date: 2020-10-03 00:00:00
Tasks: Neural Architecture Search, Transfer Learning

In this work, a modified neural architecture search method (NAS) based physics-informed deep learning model is presented for stochastic analysis in heterogeneous porous material. Monte Carlo method based on a randomized spectral representation is first employed to construct a stochastic model for simulation of flow through porous media. To solve the governing equations for stochastic groundwater flow problem, we build a modified NAS model based on physics-informed neural networks (PINNs) with transfer learning in this paper that will be able to fit different partial differential equations (PDEs) with less calculation. The performance estimation strategies adopted is constructed from an error estimation model using the method of manufactured solutions. A sensitivity analysis is performed to obtain the prior knowledge of the PINNs model and narrow down the range of parameters for search space and use hyper-parameter optimization algorithms to further determine the values of the parameters. Further the NAS based PINNs model also saves the weights and biases of the most favorable architectures, then used in the fine-tuning process. It is found that the log-conductivity field using Gaussian correlation function will perform much better than exponential correlation case, which is more fitted to the PINNs model and the modified neural architecture search based PINNs model shows a great potential in approximating solutions to PDEs. Moreover, a three dimensional stochastic flow model is built to provide a benchmark to the simulation of groundwater flow in highly heterogeneous aquifers. The NAS model based deep collocation method is verified to be effective and accurate through numerical examples in different dimensions using different manufactured solutions.
Transferable Graph Optimizers for ML Compilers
Authors: Yanqi Zhou, Sudip Roy, Amirali Abdolrashidi, Daniel Wong, Peter Ma, Qiumin Xu, Hanxiao Liu, Phitchaya Mangpo Phothilimthana, Shen Wang, Anna Goldie, Azalia Mirhoseini, James Laudon
Date: 2020-10-21 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Deep Reinforcement Learning, Graph Neural Network

Most compilers for machine learning (ML) frameworks need to solve many correlated optimization problems to generate efficient machine code. Current ML compilers rely on heuristics based algorithms to solve these optimization problems one at a time. However, this approach is not only hard to maintain but often leads to sub-optimal solutions especially for newer model architectures. Existing learning based approaches in the literature are sample inefficient, tackle a single optimization problem, and do not generalize to unseen graphs making them infeasible to be deployed in practice. To address these limitations, we propose an end-to-end, transferable deep reinforcement learning method for computational graph optimization (GO), based on a scalable sequential attention mechanism over an inductive graph neural network. GO generates decisions on the entire graph rather than on each individual node autoregressively, drastically speeding up the search compared to prior methods. Moreover, we propose recurrent attention layers to jointly optimize dependent graph optimization tasks and demonstrate 33%-60% speedup on three graph optimization tasks compared to TensorFlow default optimization. On a diverse set of representative graphs consisting of up to 80,000 nodes, including Inception-v3, Transformer-XL, and WaveNet, GO achieves on average 21% improvement over human experts and 18% improvement over the prior state of the art with 15x faster convergence, on a device placement task evaluated in real systems.
Deep neural networks for collaborative learning analytics: Evaluating team collaborations using student gaze point prediction
Authors: Zang Guo, Roghayeh Barmaki
Date: 2020-10-16 00:00:00
Tasks: Anatomy

Automatic assessment and evaluation of team performance during collaborative tasks is key to the learning analytics and computer-supported cooperative work research. There is a growing interest in the use of gaze-oriented cues for evaluating the collaboration and cooperativeness of teams. However, collecting gaze data using eye-trackers is not always feasible due to time and cost constraints. In this paper, we introduce an automated team assessment tool based on gaze points and joint visual attention (JVA) information extracted by computer vision solutions. We then evaluate team collaborations in an undergraduate anatomy learning activity (N=60, 30 teams) as a test user-study. The results indicate that higher JVA was positively associated with student learning outcomes (r(30)=0.50,p<0.005). Moreover, teams who participated in two experimental groups, and used interactive 3-D anatomy models, had higher JVA (F(1,28)=6.65,p<0.05) and better knowledge retention (F(1,28) =7.56,p<0.05) than those in the control group. Also, no significant difference was observed based on JVA for different gender compositions of teams. The findings from this work offer implications in learning sciences and collaborative computing by providing a novel mutual attention-based measure to objectively evaluate team collaboration dynamics.
SAINT+: Integrating Temporal Features for EdNet Correctness Prediction
Authors: Dongmin Shin, Yugeun Shim, Hangyeol Yu, Seewoo Lee, Byungsoo Kim, Youngduck Choi
Date: 2020-10-19 00:00:00
Tasks: Decoder, Knowledge Tracing, Prediction

We propose SAINT+, a successor of SAINT which is a Transformer based knowledge tracing model that separately processes exercise information and student response information. Following the architecture of SAINT, SAINT+ has an encoder-decoder structure where the encoder applies self-attention layers to a stream of exercise embeddings, and the decoder alternately applies self-attention layers and encoder-decoder attention layers to streams of response embeddings and encoder output. Moreover, SAINT+ incorporates two temporal feature embeddings into the response embeddings: elapsed time, the time taken for a student to answer, and lag time, the time interval between adjacent learning activities. We empirically evaluate the effectiveness of SAINT+ on EdNet, the largest publicly available benchmark dataset in the education domain. Experimental results show that SAINT+ achieves state-of-the-art performance in knowledge tracing with an improvement of 1.25% in area under receiver operating characteristic curve compared to SAINT, the current state-of-the-art model in EdNet dataset.
Data-driven Regularized Inference Privacy
Authors: Chong Xiao Wang, Wee Peng Tay
Date: 2020-10-10 00:00:00
Tasks: Decision Making, Domain Adaptation, Privacy Preserving

Data is used widely by service providers as input to inference systems to perform decision making for authorized tasks. The raw data however allows a service provider to infer other sensitive information it has not been authorized for. We propose a data-driven inference privacy preserving framework to sanitize data so as to prevent leakage of sensitive information that is present in the raw data, while ensuring that the sanitized data is still compatible with the service provider's legacy inference system. We develop an inference privacy framework based on the variational method and include maximum mean discrepancy and domain adaption as techniques to regularize the domain of the sanitized data to ensure its legacy compatibility. However, the variational method leads to weak privacy in cases where the underlying data distribution is hard to approximate. It may also face difficulties when handling continuous private variables. To overcome this, we propose an alternative formulation of the privacy metric using maximal correlation and we present empirical methods to estimate it. Finally, we develop a deep learning model as an example of the proposed inference privacy framework. Numerical experiments verify the feasibility of our approach.
Legal Document Classification: An Application to Law Area Prediction of Petitions to Public Prosecution Service
Authors: Mariana Y. Noguti, Eduardo Vellasques, Luiz S. Oliveira
Date: 2020-10-13 00:00:00
Tasks: Classification, Document Classification, General Classification, Sentiment Analysis, Word Embeddings

In recent years, there has been an increased interest in the application of Natural Language Processing (NLP) to legal documents. The use of convolutional and recurrent neural networks along with word embedding techniques have presented promising results when applied to textual classification problems, such as sentiment analysis and topic segmentation of documents. This paper proposes the use of NLP techniques for textual classification, with the purpose of categorizing the descriptions of the services provided by the Public Prosecutor's Office of the State of Paran\'a to the population in one of the areas of law covered by the institution. Our main goal is to automate the process of assigning petitions to their respective areas of law, with a consequent reduction in costs and time associated with such process while allowing the allocation of human resources to more complex tasks. In this paper, we compare different approaches to word representations in the aforementioned task: including document-term matrices and a few different word embeddings. With regards to the classification models, we evaluated three different families: linear models, boosted trees and neural networks. The best results were obtained with a combination of Word2Vec trained on a domain-specific corpus and a Recurrent Neural Network (RNN) architecture (more specifically, LSTM), leading to an accuracy of 90\% and F1-Score of 85\% in the classification of eighteen categories (law areas).
Dynamically Tie the Right Offer to the Right Customer in Telecommunications Industry
Authors: Kunal Sawarkar, Sanket Jain
Date: 2020-10-18 00:00:00
Tasks: Marketing, Segmentation

For a successful business, engaging in an effective campaign is a key task for marketers. Most previous studies used various mathematical models to segment customers without considering the correlation between customer segmentation and a campaign. This work presents a conceptual model by studying the significant campaign-dependent variables of customer targeting in customer segmentation context. In this way, the processes of customer segmentation and targeting thus can be linked and solved together. The outcomes of customer segmentation of this study could be more meaningful and relevant for marketers. This investigation applies a customer life time value (LTV) model to assess the fitness between targeted customer groups and marketing strategies. To integrate customer segmentation and customer targeting, this work uses the genetic algorithm (GA) to determine the optimized marketing strategy. Later, we suggest using C&RT (Classification and Regression Tree) in SPSS PASW Modeler as the replacement to Genetic Algorithm technique to accomplish these results. We also suggest using LOSSYCOUNTING and Counting Bloom Filter to dynamically design the right and up-to-date offer to the right customer.
The power of pictures: using ML assisted image generation to engage the crowd in complex socioscientific problems
Authors: Janet Rafner, Lotte Philipsen, Sebastian Risi, Joel Simon, Jacob Sherson
Date: 2020-10-15 00:00:00
Tasks: Image Generation

Human-computer image generation using Generative Adversarial Networks (GANs) is becoming a well-established methodology for casual entertainment and open artistic exploration. Here, we take the interaction a step further by weaving in carefully structured design elements to transform the activity of ML-assisted imaged generation into a catalyst for large-scale popular dialogue on complex socioscientific problems such as the United Nations Sustainable Development Goals (SDGs) and as a gateway for public participation in research.
TUTA: Tree-based Transformers for Generally Structured Table Pre-training
Authors: Zhiruo Wang, Haoyu Dong, Ran Jia, Jia Li, Zhiyi Fu, Shi Han, Dongmei Zhang
Date: 2020-10-21 00:00:00

Tables are widely used with various structures to organize and present data. Recent attempts on table understanding mainly focus on relational tables, yet overlook to other common table structures. In this paper, we propose TUTA, a unified pre-training architecture for understanding generally structured tables. Noticing that understanding a table requires spatial, hierarchical, and semantic information, we enhance transformers with three novel structure-aware mechanisms. First, we devise a unified tree-based structure, called a bi-dimensional coordinate tree, to describe both the spatial and hierarchical information of generally structured tables. Upon this, we propose tree-based attention and position embedding to better capture the spatial and hierarchical information. Moreover, we devise three progressive pre-training objectives to enable representations at the token, cell, and table levels. We pre-train TUTA on a wide range of unlabeled web and spreadsheet tables and fine-tune it on two critical tasks in the field of table structure understanding: cell type classification and table type classification. Experiments show that TUTA is highly effective, achieving state-of-the-art on five widely-studied datasets.
Comprehensive Empirical Evaluation of Deep Learning Approaches for Session-based Recommendation in E-Commerce
Authors: Mohamed Maher, Perseverance Munga Ngoy, Aleksandrs Rebriks, Cagri Ozcinar, Josue Cuevas, Rajasekhar Sanagavarapu, Gholamreza Anbarjafari
Date: 2020-10-17 00:00:00
Tasks: Recommendation Systems, Session-Based Recommendations

Boosting sales of e-commerce services is guaranteed once users find more matching items to their interests in a short time. Consequently, recommendation systems have become a crucial part of any successful e-commerce services. Although various recommendation techniques could be used in e-commerce, a considerable amount of attention has been drawn to session-based recommendation systems during the recent few years. This growing interest is due to the security concerns in collecting personalized user behavior data, especially after the recent general data protection regulations. In this work, we present a comprehensive evaluation of the state-of-the-art deep learning approaches used in the session-based recommendation. In session-based recommendation, a recommendation system counts on the sequence of events made by a user within the same session to predict and endorse other items that are more likely to correlate with his/her preferences. Our extensive experiments investigate baseline techniques (\textit{e.g.,} nearest neighbors and pattern mining algorithms) and deep learning approaches (\textit{e.g.,} recurrent neural networks, graph neural networks, and attention-based networks). Our evaluations show that advanced neural-based models and session-based nearest neighbor algorithms outperform the baseline techniques in most of the scenarios. However, we found that these models suffer more in case of long sessions when there exists drift in user interests, and when there is no enough data to model different items correctly during training. Our study suggests that using hybrid models of different approaches combined with baseline algorithms could lead to substantial results in session-based recommendations based on dataset characteristics. We also discuss the drawbacks of current session-based recommendation algorithms and further open research directions in this field.
Exploring investor behavior in Bitcoin: a study of the disposition effect
Authors: Jürgen E. Schatzmann, Bernhard Haslhofer
Date: 2020-10-23 00:00:00

Investors commonly exhibit the disposition effect - the irrational tendency to sell their winning investments and hold onto their losing ones. While this phenomenon has been observed in many traditional markets, it remains unclear whether it also applies to atypical markets like cryptoassets. This paper investigates the prevalence of the disposition effect in Bitcoin using transactions targeting cryptoasset exchanges as proxies for selling transactions. Our findings suggest that investors in Bitcoin were indeed subject to the disposition effect, with varying intensity. They also show that the disposition effect was not consistently present throughout the observation period. Its prevalence was more evident from the boom and bust year 2017 onwards, as confirmed by various technical indicators. Our study suggests irrational investor behavior is also present in atypical markets like Bitcoin.
Direct Estimation of Firing Rates from Calcium Imaging Data
Authors: Elad Ganmor, Michael Krumin, Luigi F. Rossi, Matteo Carandini, Eero P. Simoncelli
Date: 2016-01-04 00:00:00

Two-photon imaging of calcium indicators allows simultaneous recording of responses of hundreds of neurons over hours and even days, but provides a relatively indirect measure of their spiking activity. Existing deconvolution algorithms attempt to recover spikes from observed imaging data, which are then commonly subjected to the same analyses that are applied to electrophysiologically recorded spikes (e.g., estimation of average firing rates, or tuning curves). Here we show, however, that in the presence of noise this approach is often heavily biased. We propose an alternative analysis that aims to estimate the underlying rate directly, by integrating over the unobserved spikes instead of committing to a single estimate of the spike train. This approach can be used to estimate average firing rates or tuning curves directly from the imaging data, and is sufficiently flexible to incorporate prior knowledge about tuning structure. We show that directly estimated rates are more accurate than those obtained from averaging of spikes estimated through deconvolution, both on simulated data and on imaging data acquired in mouse visual cortex.
Logical Signal Processing: a Fourier Analysis of Temporal Logic
Authors: Niraj Basnet, Houssam Abbas
Date: 2020-09-17 00:00:00

What is the frequency content of temporal logic formulas? That is, when we monitor a signal against a formula, which frequency bands of the signal are relevant to the logic and should be preserved, and which can be safely discarded? This question is relevant whenever signals are filtered or compressed before being monitored, which is almost always the case for analog signals. To answer this question, we focus on monitors that measure the robustness of a signal relative to a specification in Signal Temporal Logic. We prove that robustness monitors can be modeled using Volterra series. We then study the Fourier transforms of these Volterra representations, and provide a method to derive the Fourier transforms of entire formulas. We also make explicit the measurement process in temporal logic and re-define it on the basis of distributions to make it compatible with measurements in signal processing. Experiments illustrate these results. Beyond compression, this work enables the integration of temporal logic monitoring into common signal processing toolchains as just another signal processing operation, and enables a common formalism to study both logical and non-logical operations in the frequency domain, which we refer to as Logical Signal Processing.