pwc-archive/papers-with-abstracts huggingface dataset:
Self-Directed Online Machine Learning for Topology Optimization
Authors: Changyu Deng, Yizhou Wang, Can Qin, Yun Fu, Wei Lu
Date: 2020-02-04 00:00:00
Tasks: BIG-bench Machine Learning, Stochastic Optimization

Topology optimization by optimally distributing materials in a given domain requires non-gradient optimizers to solve highly complicated problems. However, with hundreds of design variables or more involved, solving such problems would require millions of Finite Element Method (FEM) calculations whose computational cost is huge and impractical. Here we report Self-directed Online Learning Optimization (SOLO) which integrates Deep Neural Network (DNN) with FEM calculations. A DNN learns and substitutes the objective as a function of design variables. A small number of training data is generated dynamically based on the DNN's prediction of the optimum. The DNN adapts to the new training data and gives better prediction in the region of interest until convergence. The optimum predicted by the DNN is proved to converge to the true global optimum through iterations. Our algorithm was tested by four types of problems including compliance minimization, fluid-structure optimization, heat transfer enhancement and truss optimization. It reduced the computational time by 2 ~ 5 orders of magnitude compared with directly using heuristic methods, and outperformed all state-of-the-art algorithms tested in our experiments. This approach enables solving large multi-dimensional optimization problems.
BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization
Authors: Henry B. Moss, Vatsal Aggarwal, Nishant Prateek, Javier González, Roberto Barra-Chicote
Date: 2020-02-04 00:00:00
Tasks: Bayesian Optimization, text-to-speech, Text to Speech

We present BOFFIN TTS (Bayesian Optimization For FIne-tuning Neural Text To Speech), a novel approach for few-shot speaker adaptation. Here, the task is to fine-tune a pre-trained TTS model to mimic a new speaker using a small corpus of target utterances. We demonstrate that there does not exist a one-size-fits-all adaptation strategy, with convincing synthesis requiring a corpus-specific configuration of the hyper-parameters that control fine-tuning. By using Bayesian optimization to efficiently optimize these hyper-parameter values for a target speaker, we are able to perform adaptation with an average 30% improvement in speaker similarity over standard techniques. Results indicate, across multiple corpora, that BOFFIN TTS can learn to synthesize new speakers using less than ten minutes of audio, achieving the same naturalness as produced for the speakers used to train the base model.
Neural network with data augmentation in multi-objective prediction of multi-stage pump
Authors: Hang Zhao
Date: 2020-02-04 00:00:00
Tasks: Data Augmentation, Prediction

A multi-objective prediction method of multi-stage pump method based on neural network with data augmentation is proposed. In order to study the highly nonlinear relationship between key design variables and centrifugal pump external characteristic values (head and power), the neural network model (NN) is built in comparison with the quadratic response surface model (RSF), the radial basis Gaussian response surface model (RBF), and the Kriging model (KRG). The numerical model validation experiment of another type of single stage centrifugal pump showed that numerical model based on CFD is quite accurate and fair. All of prediction models are trained by 60 samples under the different combination of three key variables in design range respectively. The accuracy of the head and power based on the four predictions models are analyzed comparing with the CFD simulation values. The results show that the neural network model has better performance in all external characteristic values comparing with other three surrogate models. Finally, a neural network model based on data augmentation (NNDA) is proposed for the reason that simulation cost is too high and data is scarce in mechanical simulation field especially in CFD problems. The model with data augmentation can triple the data by interpolation at each sample point of different attributes. It shows that the performance of neural network model with data augmentation is better than former neural network model. Therefore, the prediction ability of NN is enhanced without more simulation costs. With data augmentation it can be a better prediction model used in solving the optimization problems of multistage pump for next optimization and generalized to finite element analysis optimization problems in future.
Defending Adversarial Attacks via Semantic Feature Manipulation
Authors: Shuo Wang, Tianle Chen, Surya Nepal, Carsten Rudolph, Marthie Grobler, Shangyu Chen
Date: 2020-02-03 00:00:00
Tasks: General Classification

Machine learning models have demonstrated vulnerability to adversarial attacks, more specifically misclassification of adversarial examples. In this paper, we propose a one-off and attack-agnostic Feature Manipulation (FM)-Defense to detect and purify adversarial examples in an interpretable and efficient manner. The intuition is that the classification result of a normal image is generally resistant to non-significant intrinsic feature changes, e.g., varying thickness of handwritten digits. In contrast, adversarial examples are sensitive to such changes since the perturbation lacks transferability. To enable manipulation of features, a combo-variational autoencoder is applied to learn disentangled latent codes that reveal semantic features. The resistance to classification change over the morphs, derived by varying and reconstructing latent codes, is used to detect suspicious inputs. Further, combo-VAE is enhanced to purify the adversarial examples with good quality by considering both class-shared and class-unique features. We empirically demonstrate the effectiveness of detection and the quality of purified instance. Our experiments on three datasets show that FM-Defense can detect nearly $100\%$ of adversarial examples produced by different state-of-the-art adversarial attacks. It achieves more than $99\%$ overall purification accuracy on the suspicious instances that close the manifold of normal examples.
Towards Sharper First-Order Adversary with Quantized Gradients
Authors: Zhuanghua Liu, Ivor W. Tsang
Date: 2020-02-01 00:00:00
Tasks: Adversarial Robustness, Quantization

Despite the huge success of Deep Neural Networks (DNNs) in a wide spectrum of machine learning and data mining tasks, recent research shows that this powerful tool is susceptible to maliciously crafted adversarial examples. Up until now, adversarial training has been the most successful defense against adversarial attacks. To increase adversarial robustness, a DNN can be trained with a combination of benign and adversarial examples generated by first-order methods. However, in state-of-the-art first-order attacks, adversarial examples with sign gradients retain the sign information of each gradient component but discard the relative magnitude between components. In this work, we replace sign gradients with quantized gradients. Gradient quantization not only preserves the sign information, but also keeps the relative magnitude between components. Experiments show white-box first-order attacks with quantized gradients outperform their variants with sign gradients on multiple datasets. Notably, our BLOB\_QG attack achieves an accuracy of $88.32\%$ on the secret MNIST model from the MNIST Challenge and it outperforms all other methods on the leaderboard of white-box attacks.
A Survey on String Constraint Solving
Authors: Roberto Amadini
Date: 2020-01-31 00:00:00
Tasks: Survey

String constraint solving refers to solving combinatorial problems involving constraints over string variables. String solving approaches have become popular over the last years given the massive use of strings in different application domains like formal analysis, automated testing, database query processing, and cybersecurity. This paper reports a comprehensive survey on string constraint solving by exploring the large number of approaches that have been proposed over the last decades to solve string constraints.
Detecting Changes in Asset Co-Movement Using the Autoencoder Reconstruction Ratio
Authors: Bryan Lim, Stefan Zohren, Stephen Roberts
Date: 2020-01-23 00:00:00
Tasks: Denoising, Dimensionality Reduction, Management

Detecting changes in asset co-movements is of much importance to financial practitioners, with numerous risk management benefits arising from the timely detection of breakdowns in historical correlations. In this article, we propose a real-time indicator to detect temporary increases in asset co-movements, the Autoencoder Reconstruction Ratio, which measures how well a basket of asset returns can be modelled using a lower-dimensional set of latent variables. The ARR uses a deep sparse denoising autoencoder to perform the dimensionality reduction on the returns vector, which replaces the PCA approach of the standard Absorption Ratio, and provides a better model for non-Gaussian returns. Through a systemic risk application on forecasting on the CRSP US Total Market Index, we show that lower ARR values coincide with higher volatility and larger drawdowns, indicating that increased asset co-movement does correspond with periods of market weakness. We also demonstrate that short-term (i.e. 5-min and 1-hour) predictors for realised volatility and market crashes can be improved by including additional ARR inputs.
Zero-Shot Activity Recognition with Videos
Authors: Evin Pinar Ornek
Date: 2020-01-22 00:00:00
Tasks: Action Recognition, Activity Recognition, Word Embeddings, Zero-Shot Learning

In this paper, we examined the zero-shot activity recognition task with the usage of videos. We introduce an auto-encoder based model to construct a multimodal joint embedding space between the visual and textual manifolds. On the visual side, we used activity videos and a state-of-the-art 3D convolutional action recognition network to extract the features. On the textual side, we worked with GloVe word embeddings. The zero-shot recognition results are evaluated by top-n accuracy. Then, the manifold learning ability is measured by mean Nearest Neighbor Overlap. In the end, we provide an extensive discussion over the results and the future directions.
Scale bridging materials physics: Active learning workflows and integrable deep neural networks for free energy function representations in alloys
Authors: Gregory Teichert, Anirudh Natarajan, Anton Van der Ven, Krishna Garikipati
Date: 2020-01-30 00:00:00
Tasks: Active Learning

The free energy plays a fundamental role in descriptions of many systems in continuum physics. Notably, in multiphysics applications, it encodes thermodynamic coupling between different fields. It thereby gives rise to driving forces on the dynamics of interaction between the constituent phenomena. In mechano-chemically interacting materials systems, even consideration of only compositions, order parameters and strains can render the free energy to be reasonably high-dimensional. In proposing the free energy as a paradigm for scale bridging, we have previously exploited neural networks for their representation of such high-dimensional functions. Specifically, we have developed an integrable deep neural network (IDNN) that can be trained to free energy derivative data obtained from atomic scale models and statistical mechanics, then analytically integrated to recover a free energy density function. The motivation comes from the statistical mechanics formalism, in which certain free energy derivatives are accessible for control of the system, rather than the free energy itself. Our current work combines the IDNN with an active learning workflow to improve sampling of the free energy derivative data in a high-dimensional input space. Treated as input-output maps, machine learning accommodates role reversals between independent and dependent quantities as the mathematical descriptions change with scale bridging. As a prototypical system we focus on Ni-Al. Phase field simulations using the resulting IDNN representation for the free energy density of Ni-Al demonstrate that the appropriate physics of the material have been learned. To the best of our knowledge, this represents the most complete treatment of scale bridging, using the free energy for a practical materials system, that starts with electronic structure calculations and proceeds through statistical mechanics to continuum physics.
Predicting Bank Loan Default with Extreme Gradient Boosting
Authors: Rising Odegua
Date: 2020-01-18 00:00:00
Tasks: Prediction

Loan default prediction is one of the most important and critical problems faced by banks and other financial institutions as it has a huge effect on profit. Although many traditional methods exist for mining information about a loan application, most of these methods seem to be under-performing as there have been reported increases in the number of bad loans. In this paper, we use an Extreme Gradient Boosting algorithm called XGBoost for loan default prediction. The prediction is based on a loan data from a leading bank taking into consideration data sets from both the loan application and the demographic of the applicant. We also present important evaluation metrics such as Accuracy, Recall, precision, F1-Score and ROC area of the analysis. This paper provides an effective basis for loan credit approval in order to identify risky customers from a large number of loan applications using predictive modeling.
Detecting Network Anomalies using Rule-based machine learning within SNMP-MIB dataset
Authors: Abdalrahman Hwoij, Mouhammd Al-kasassbeh, Mustafa Al-Fayoumi
Date: 2020-01-18 00:00:00
Tasks: BIG-bench Machine Learning

One of the most effective threats that targeting cybercriminals to limit network performance is Denial of Service (DOS) attack. Thus, data security, completeness and efficiency could be greatly damaged by this type of attacks. This paper developed a network traffic system that relies on adopted dataset to differentiate the DOS attacks from normal traffic. The detection model is built with five Rule-based machine learning classifiers (DecisionTable, JRip, OneR, PART and ZeroR). The findings have shown that the ICMP variables are implemented in the identification of ICMP attack, HTTP flood attack, and Slowloris at a high accuracy of approximately 99.7% using PART classifier. In addition, PART classifier has succeeded in classifying normal traffic from different DOS attacks at 100%.
Covering the News with (AI) Style
Authors: Michele Merler, Cicero Nogueira dos santos, Mauro Martino, Alfio M. Gliozzo, John R. Smith
Date: 2020-01-05 00:00:00
Tasks: Articles

We introduce a multi-modal discriminative and generative frame-work capable of assisting humans in producing visual content re-lated to a given theme, starting from a collection of documents(textual, visual, or both). This framework can be used by edit or to generate images for articles, as well as books or music album covers. Motivated by a request from the The New York Times (NYT) seeking help to use AI to create art for their special section on Artificial Intelligence, we demonstrated the application of our system in producing such image.
EgoMap: Projective mapping and structured egocentric memory for Deep RL
Authors: Edward Beeching, Christian Wolf, Jilles Dibangoye, Olivier Simonin
Date: 2020-01-24 00:00:00
Tasks: Deep Reinforcement Learning, Memorization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Tasks involving localization, memorization and planning in partially observable 3D environments are an ongoing challenge in Deep Reinforcement Learning. We present EgoMap, a spatially structured neural memory architecture. EgoMap augments a deep reinforcement learning agent's performance in 3D environments on challenging tasks with multi-step objectives. The EgoMap architecture incorporates several inductive biases including a differentiable inverse projection of CNN feature vectors onto a top-down spatially structured map. The map is updated with ego-motion measurements through a differentiable affine transform. We show this architecture outperforms both standard recurrent agents and state of the art agents with structured memory. We demonstrate that incorporating these inductive biases into an agent's architecture allows for stable training with reward alone, circumventing the expense of acquiring and labelling expert trajectories. A detailed ablation study demonstrates the impact of key aspects of the architecture and through extensive qualitative analysis, we show how the agent exploits its structured internal memory to achieve higher performance.
A Stochastic Game Framework for Efficient Energy Management in Microgrid Networks
Authors: Shravan Nayak, Chanakya Ajit Ekbote, Annanya Pratap Singh Chauhan, Raghuram Bharadwaj Diddigi, Prishita Ray, Abhinava Sikdar, Sai Koti Reddy Danda, Shalabh Bhatnagar
Date: 2020-02-06 00:00:00
Tasks: energy management, energy trading, Management, Q-Learning, Scheduling

We consider the problem of energy management in microgrid networks. A microgrid is capable of generating a limited amount of energy from a renewable resource and is responsible for handling the demands of its dedicated customers. Owing to the variable nature of renewable generation and the demands of the customers, it becomes imperative that each microgrid optimally manages its energy. This involves intelligently scheduling the demands at the customer side, selling (when there is a surplus) and buying (when there is a deficit) the power from its neighboring microgrids depending on its current and future needs. Typically, the transaction of power among the microgrids happens at a pre-decided price by the central grid. In this work, we formulate the problems of demand and battery scheduling, energy trading and dynamic pricing (where we allow the microgrids to decide the price of the transaction depending on their current configuration of demand and renewable energy) in the framework of stochastic games. Subsequently, we propose a novel approach that makes use of independent learners Deep Q-learning algorithm to solve this problem. Through extensive empirical evaluation, we show that our proposed framework is more beneficial to the majority of the microgrids and we provide a detailed analysis of the results.
beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
Authors: Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner
Date: 2017-04-26 00:00:00
Proceeding: ICLR 2017 4
Tasks: Disentanglement

Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificial intelligence that is able to learn and reason in the same way that humans do. We introduce beta-VAE, a new state-of-the-art framework for automated discovery of interpretable factorised latent representations from raw image data in a completely unsupervised manner. Our approach is a modification of the variational autoencoder (VAE) framework. We introduce an adjustable hyperparameter beta that balances latent channel capacity and independence constraints with reconstruction accuracy. We demonstrate that beta-VAE with appropriately tuned beta > 1 qualitatively outperforms VAE (beta = 1), as well as state of the art unsupervised (InfoGAN) and semi-supervised (DC-IGN) approaches to disentangled factor learning on a variety of datasets (celebA, faces and chairs). Furthermore, we devise a protocol to quantitatively compare the degree of disentanglement learnt by different models, and show that our approach also significantly outperforms all baselines quantitatively. Unlike InfoGAN, beta-VAE is stable to train, makes few assumptions about the data and relies on tuning a single hyperparameter, which can be directly optimised through a hyper parameter search using weakly labelled data or through heuristic visual inspection for purely unsupervised data.
Non-linear prediction with LSTM recurrent neural networks for acoustic novelty detection
Authors: Erik Marchi ; Fabio Vesperini ; Felix Weninger ; Florian Eyben ; Stefano Squartini ; Björn Schuller
Date: 2015-10-01 00:00:00
Proceeding: 2015 International Joint Conference on Neural Networks (IJCNN) 2015 10
Tasks: Acoustic Novelty Detection, Denoising, Novelty Detection

Acoustic novelty detection aims at identifying abnormal/novel acoustic signals which differ from the reference/normal data that the system was trained with. In this paper we present a novel approach based on non-linear predictive denoising autoencoders. In our approach, auditory spectral features of the next short-term frame are predicted from the previous frames by means of Long-Short Term Memory (LSTM) recurrent denoising autoencoders. We show that this yields an effective generative model for audio. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. The autoencoder is trained on a public database which contains recordings of typical in-home situations such as talking, watching television, playing and eating. The evaluation was performed on more than 260 different abnormal events. We compare results with state-of-the-art methods and we conclude that our novel approach significantly outperforms existing methods by achieving up to 94.4% F-Measure.
A novel approach for automatic acoustic novelty detection using a denoising autoencoder with bidirectional LSTM neural networks
Authors: Erik Marchi ; Fabio Vesperini ; Florian Eyben ; Stefano Squartini ; Björn Schuller
Date: 2015-08-06 00:00:00
Proceeding: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015 8
Tasks: Acoustic Novelty Detection, Denoising, Novelty Detection

Acoustic novelty detection aims at identifying abnormal/novel acoustic signals which differ from the reference/normal data that the system was trained with. In this paper we present a novel unsupervised approach based on a denoising autoencoder. In our approach auditory spectral features are processed by a denoising autoencoder with bidirectional Long Short-Term Memory recurrent neural networks. We use the reconstruction error between the input and the output of the autoencoder as activation signal to detect novel events. The autoencoder is trained on a public database which contains recordings of typical in-home situations such as talking, watching television, playing and eating. The evaluation was performed on more than 260 different abnormal events. We compare results with state-of-the-art methods and we conclude that our novel approach significantly outperforms existing methods by achieving up to 93.4% F-Measure.