pwc-archive/papers-with-abstracts huggingface dataset:
Improving Device Directedness Classification of Utterances with Semantic Lexical Features
Authors: Kellen Gillespie, Ioannis C. Konstantakopoulos, Xingzhi Guo, Vishal Thanvantri Vasudevan, Abhinav Sethy
Date: 2020-09-29 00:00:00
Tasks: General Classification, Transfer Learning

User interactions with personal assistants like Alexa, Google Home and Siri are typically initiated by a wake term or wakeword. Several personal assistants feature "follow-up" modes that allow users to make additional interactions without the need of a wakeword. For the system to only respond when appropriate, and to ignore speech not intended for it, utterances must be classified as device-directed or non-device-directed. State-of-the-art systems have largely used acoustic features for this task, while others have used only lexical features or have added LM-based lexical features. We propose a directedness classifier that combines semantic lexical features with a lightweight acoustic feature and show it is effective in classifying directedness. The mixed-domain lexical and acoustic feature model is able to achieve 14% relative reduction of EER over a state-of-the-art acoustic-only baseline model. Finally, we successfully apply transfer learning and semi-supervised learning to the model to improve accuracy even further.
Gender prediction using limited Twitter Data
Authors: Maaike Burghoorn, Maaike H. T. de Boer, Stephan Raaijmakers
Date: 2020-09-29 00:00:00
Tasks: Gender Classification, Gender Prediction, Prediction

Transformer models have shown impressive performance on a variety of NLP tasks. Off-the-shelf, pre-trained models can be fine-tuned for specific NLP classification tasks, reducing the need for large amounts of additional training data. However, little research has addressed how much data is required to accurately fine-tune such pre-trained transformer models, and how much data is needed for accurate prediction. This paper explores the usability of BERT (a Transformer model for word embedding) for gender prediction on social media. Forensic applications include detecting gender obfuscation, e.g. males posing as females in chat rooms. A Dutch BERT model is fine-tuned on different samples of a Dutch Twitter dataset labeled for gender, varying in the number of tweets used per person. The results show that finetuning BERT contributes to good gender classification performance (80% F1) when finetuned on only 200 tweets per person. But when using just 20 tweets per person, the performance of our classifier deteriorates non-steeply (to 70% F1). These results show that even with relatively small amounts of data, BERT can be fine-tuned to accurately help predict the gender of Twitter users, and, consequently, that it is possible to determine gender on the basis of just a low volume of tweets. This opens up an operational perspective on the swift detection of gender.
Zero-Shot Clinical Acronym Expansion via Latent Meaning Cells
Authors: Griffin Adams, Mert Ketenci, Shreyas Bhave, Adler Perotte, Noémie Elhadad
Date: 2020-09-29 00:00:00
Tasks: Representation Learning

We introduce Latent Meaning Cells, a deep latent variable model which learns contextualized representations of words by combining local lexical context and metadata. Metadata can refer to granular context, such as section type, or to more global context, such as unique document ids. Reliance on metadata for contextualized representation learning is apropos in the clinical domain where text is semi-structured and expresses high variation in topics. We evaluate the LMC model on the task of zero-shot clinical acronym expansion across three datasets. The LMC significantly outperforms a diverse set of baselines at a fraction of the pre-training cost and learns clinically coherent representations. We demonstrate that not only is metadata itself very helpful for the task, but that the LMC inference algorithm provides an additional large benefit.
Towards ML Engineering: A Brief History Of TensorFlow Extended (TFX)
Authors: Konstantinos, Katsiapis, Abhijit Karmarkar, Ahmet Altay, Aleksandr Zaks, Neoklis Polyzotis, Anusha Ramesh, Ben Mathes, Gautam Vasudevan, Irene Giannoumis, Jarek Wilkiewicz, Jiri Simsa, Justin Hong, Mitch Trott, Noé Lutz, Pavel A. Dournov, Robert Crowe, Sarah Sirajuddin, Tris Brian Warkentin, Zhitao Li
Date: 2020-09-28 00:00:00

Software Engineering, as a discipline, has matured over the past 5+ decades. The modern world heavily depends on it, so the increased maturity of Software Engineering was an eventuality. Practices like testing and reliable technologies help make Software Engineering reliable enough to build industries upon. Meanwhile, Machine Learning (ML) has also grown over the past 2+ decades. ML is used more and more for research, experimentation and production workloads. ML now commonly powers widely-used products integral to our lives. But ML Engineering, as a discipline, has not widely matured as much as its Software Engineering ancestor. Can we take what we have learned and help the nascent field of applied ML evolve into ML Engineering the way Programming evolved into Software Engineering [1]? In this article we will give a whirlwind tour of Sibyl [2] and TensorFlow Extended (TFX) [3], two successive end-to-end (E2E) ML platforms at Alphabet. We will share the lessons learned from over a decade of applied ML built on these platforms, explain both their similarities and their differences, and expand on the shifts (both mental and technical) that helped us on our journey. In addition, we will highlight some of the capabilities of TFX that help realize several aspects of ML Engineering. We argue that in order to unlock the gains ML can bring, organizations should advance the maturity of their ML teams by investing in robust ML infrastructure and promoting ML Engineering education. We also recommend that before focusing on cutting-edge ML modeling techniques, product leaders should invest more time in adopting interoperable ML platforms for their organizations. In closing, we will also share a glimpse into the future of TFX.
A Physics-Informed Machine Learning Approach for Solving Heat Transfer Equation in Advanced Manufacturing and Engineering Applications
Authors: Navid Zobeiry, Keith D. Humfeld
Date: 2020-09-28 00:00:00
Tasks: Feature Engineering, Physics-informed machine learning

A physics-informed neural network is developed to solve conductive heat transfer partial differential equation (PDE), along with convective heat transfer PDEs as boundary conditions (BCs), in manufacturing and engineering applications where parts are heated in ovens. Since convective coefficients are typically unknown, current analysis approaches based on trial and error finite element (FE) simulations are slow. The loss function is defined based on errors to satisfy PDE, BCs and initial condition. An adaptive normalizing scheme is developed to reduce loss terms simultaneously. In addition, theory of heat transfer is used for feature engineering. The predictions for 1D and 2D cases are validated by comparing with FE results. It is shown that using engineered features, heat transfer beyond the training zone can be predicted. Trained model allows for fast evaluation of a range of BCs to develop feedback loops, realizing Industry 4.0 concept of active manufacturing control based on sensor data.
Deep Representational Similarity Learning for analyzing neural signatures in task-based fMRI dataset
Authors: Muhammad Yousefnezhad, Jeffrey Sawalha, Alessandro Selvitella, Daoqiang Zhang
Date: 2020-09-28 00:00:00
Tasks: Decision Making

Similarity analysis is one of the crucial steps in most fMRI studies. Representational Similarity Analysis (RSA) can measure similarities of neural signatures generated by different cognitive states. This paper develops Deep Representational Similarity Learning (DRSL), a deep extension of RSA that is appropriate for analyzing similarities between various cognitive tasks in fMRI datasets with a large number of subjects, and high-dimensionality -- such as whole-brain images. Unlike the previous methods, DRSL is not limited by a linear transformation or a restricted fixed nonlinear kernel function -- such as Gaussian kernel. DRSL utilizes a multi-layer neural network for mapping neural responses to linear space, where this network can implement a customized nonlinear transformation for each subject separately. Furthermore, utilizing a gradient-based optimization in DRSL can significantly reduce runtime of analysis on large datasets because it uses a batch of samples in each iteration rather than all neural responses to find an optimal solution. Empirical studies on multi-subject fMRI datasets with various tasks -- including visual stimuli, decision making, flavor, and working memory -- confirm that the proposed method achieves superior performance to other state-of-the-art RSA algorithms.
Machine Learning and Computer Vision Techniques to Predict Thermal Properties of Particulate Composites
Authors: Fazlolah Mohaghegh, Jayathi Murthy
Date: 2020-08-27 00:00:00
Tasks: BIG-bench Machine Learning

Accurate thermal analysis of composites and porous media requires detailed characterization of local thermal properties in small scale. For some important applications such as lithium-ion batteries, changes in the properties during the operation makes the analysis even more challenging, necessitating a rapid characterization. We propose a new method to characterize the thermal properties of particulate composites based on actual micro-images. Our computer-vision-based approach constructs 3D images from stacks of 2D SEM images and then extracts several representative elemental volumes (REVs) from the reconstructed images at random places, which leads to having a range of geometrical features for different REVs. A deep learning algorithm is designed based on convolutional neural nets to take the shape of the geometry and result in the effective conductivity of the REV. The training of the network is performed in two methods: First, based on implementing a coarser grid that uses the average values of conductivities from the fine grid and the resulted effective conductivity from the DNS solution of the fine grid. The other method uses conductivity values on cross sections from each REV in different directions. The results of training based on averaging show that using a coarser grid in the network does not have a meaningful effect on the network error; however, it decreases the training time up to three orders of magnitude. We showed that one general network can make accurate predictions using different types of electrode images, representing the difference in the geometry and constituents. Moreover, training based on averaging is more accurate than training based on cross sections. The study of the robustness of implementing a machine learning technique in predicting the thermal percolation shows the prediction error is almost half of the error from predictions based on the volume fraction.
Torchattacks: A PyTorch Repository for Adversarial Attacks
Authors: Hoki Kim
Date: 2020-09-24 00:00:00
Tasks: Deep Learning

Torchattacks is a PyTorch library that contains adversarial attacks to generate adversarial examples and to verify the robustness of deep learning models. The code can be found at https://github.com/Harry24k/adversarial-attacks-pytorch.
Orbital Graph Convolutional Neural Network for Material Property Prediction
Authors: Mohammadreza Karamad, Rishikesh Magar, Yuting Shi, Samira Siahrostami, Ian D. Gates, Amir Barati Farimani
Date: 2020-08-14 00:00:00
Tasks: BIG-bench Machine Learning, Property Prediction

Material representations that are compatible with machine learning models play a key role in developing models that exhibit high accuracy for property prediction. Atomic orbital interactions are one of the important factors that govern the properties of crystalline materials, from which the local chemical environments of atoms is inferred. Therefore, to develop robust machine learningmodels for material properties prediction, it is imperative to include features representing such chemical attributes. Here, we propose the Orbital Graph Convolutional Neural Network (OGCNN), a crystal graph convolutional neural network framework that includes atomic orbital interaction features that learns material properties in a robust way. In addition, we embedded an encoder-decoder network into the OGCNN enabling it to learn important features among basic atomic (elemental features), orbital-orbital interactions, and topological features. We examined the performance of this model on a broad range of crystalline material data to predict different properties. We benchmarked the performance of the OGCNN model with that of: 1) the crystal graph convolutional neural network (CGCNN), 2) other state-of-the-art descriptors for material representations including Many-body Tensor Representation (MBTR) and the Smooth Overlap of Atomic Positions (SOAP), and 3) other conventional regression machine learning algorithms where different crystal featurization methods have been used. We find that OGCNN significantly outperforms them. The OGCNN model with high predictive accuracy can be used to discover new materials among the immense phase and compound spaces of materials
A Finite-Time Technological Singularity Model With Artificial Intelligence Self-Improvement
Authors: Ihor Kendiukhov
Date: 2020-08-31 00:00:00

Recent advances in the development of artificial intelligence, technological progress acceleration, long-term trends of macroeconomic dynamics increase the relevance of technological singularity hypothesis. In this paper, we build a model of finite-time technological singularity assuming that artificial intelligence will replace humans for artificial intelligence engineers after some point in time when it is developed enough. This model implies the following: let A be the level of development of artificial intelligence. Then, the moment of technological singularity n is defined as the point in time where artificial intelligence development function approaches infinity. Thus, it happens in finite time. Although infinite level of development of artificial intelligence cannot be reached practically, this approximation is useful for several reasons, firstly because it allows modeling a phase transition or a change of regime. In the model, intelligence growth function appears to be hyperbolic function under relatively broad conditions which we list and compare. Subsequently, we also add a stochastic term (Brownian motion) to the model and investigate the changes in its behavior. The results can be applied for the modeling of dynamics of various processes characterized by multiplicative growth.
Measuring the Complexity of Domains Used to Evaluate AI Systems
Authors: Christopher Pereyda, Lawrence Holder
Date: 2020-09-18 00:00:00
Tasks: Benchmarking

There is currently a rapid increase in the number of challenge problem, benchmarking datasets and algorithmic optimization tests for evaluating AI systems. However, there does not currently exist an objective measure to determine the complexity between these newly created domains. This lack of cross-domain examination creates an obstacle to effectively research more general AI systems. We propose a theory for measuring the complexity between varied domains. This theory is then evaluated using approximations by a population of neural network based AI systems. The approximations are compared to other well known standards and show it meets intuitions of complexity. An application of this measure is then demonstrated to show its effectiveness as a tool in varied situations. The experimental results show this measure has promise as an effective tool for aiding in the evaluation of AI systems. We propose the future use of such a complexity metric for use in computing an AI system's intelligence.
VehicleNet: Learning Robust Feature Representation for Vehicle Re-identification
Authors: Zhedong Zheng ; Tao Ruan ; Yunchao Wei ; Yi Yang ; Tao Mei
Date: 2020-08-07 00:00:00
Tasks: Representation Learning, Vehicle Re-Identification

One fundamental challenge of vehicle re-identification (re-id) is to learn robust and discriminative visual representation, given the significant intra-class vehicle variations across different camera views. As the existing vehicle datasets are limited in terms of training images and viewpoints, we propose to build a unique large-scale vehicle dataset (called VehicleNet) by harnessing four public vehicle datasets, and design a novel yet effective two-stage progressive approach to learning more robust visual representation from VehicleNet. The first stage of our approach is to learn the generic representation for all domains (i.e., source vehicle datasets) by training with the conventional classification loss. This stage relaxes the full alignment between the training and testing domains, as it is agnostic to the target vehicle domain. The second stage is to fine-tune the trained model purely based on the target vehicle set, by minimizing the distribution discrepancy between our VehicleNet and any target domain. We discuss our proposed multi-source dataset VehicleNet and evaluate the effectiveness of the two-stage progressive representation learning through extensive experiments. We achieve the state-of-art accuracy of 86.07% mAP on the private test set of AICity Challenge, and competitive results on two other public vehicle re-id datasets, i.e., VeRi-776 and VehicleID. We hope this new VehicleNet dataset and the learned robust representations can pave the way for vehicle re-id in the real-world environments.
Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering
Authors: Wei Han, Hantao Huang, Tao Han
Date: 2020-10-06 00:00:00
Proceeding: COLING 2020 8
Tasks: Optical Character Recognition, Optical Character Recognition (OCR), Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Image text carries essential information to understand the scene and perform reasoning. Text-based visual question answering (text VQA) task focuses on visual questions that require reading text in images. Existing text VQA systems generate an answer by selecting from optical character recognition (OCR) texts or a fixed vocabulary. Positional information of text is underused and there is a lack of evidence for the generated answer. As such, this paper proposes a localization-aware answer prediction network (LaAP-Net) to address this challenge. Our LaAP-Net not only generates the answer to the question but also predicts a bounding box as evidence of the generated answer. Moreover, a context-enriched OCR representation (COR) for multimodal fusion is proposed to facilitate the localization task. Our proposed LaAP-Net outperforms existing approaches on three benchmark datasets for the text VQA task by a noticeable margin.
Automatically Optimized Gradient Boosting Trees for Classifying Large Volume High Cardinality Data Streams Under Concept Drift
Authors: Jobin Wilson, Amit Kumar Meher, Bivin Vinodkumar Bindu, Santanu Chaudhury, Brejesh lall, Manoj Sharma, Vishakha Pareek
Date: 2019-11-30 00:00:00
Tasks: AutoML, BIG-bench Machine Learning, Lifelong learning

Data abundance along with scarcity of machine learning experts and domain specialists necessitates progressive automation of end-to-end machine learning workflows. To this end, Automated Machine Learning (AutoML) has emerged as a prominent research area. Real world data often arrives as streams or batches, and data distribution evolves over time causing concept drift. Models need to handle data that is not independent and identically distributed (iid), and transfer knowledge across time through continuous self-evaluation and adaptation adhering to resource constraints. Creating autonomous self-maintaining models which not only discover an optimal pipeline, but also automatically adapt to concept drift to operate in a lifelong learning setting was the crux of NeurIPS 2018 AutoML challenge. We describe our winning solution to the challenge, entitled AutoGBT, which combines an adaptive self-optimized end-to-end machine learning pipeline based on gradient boosting trees with automatic hyper-parameter tuning using Sequential Model-Based Optimization (SMBO). We report experimental results on the challenge datasets as well as several benchmark datasets affected by concept drift and compare it with the baseline model for the challenge and Auto-sklearn. Results indicate the effectiveness of the proposed methodology in this context.
Towards Automatically-Tuned Deep Neural Networks
Authors: Hector Mendoza, Aaron Klein, Matthias Feurer, Jost Tobias Springenberg, Matthias Urban, Michael Burkart, Maximilian Dippel, Marius Lindauer, Frank Hutter
Date: 2019-05-18 00:00:00
Tasks: AutoML, BIG-bench Machine Learning

Recent advances in AutoML have led to automated tools that can compete with machine learning experts on supervised learning tasks. However, current AutoML tools do not yet support modern neural networks effectively. In this work, we present a first version of Auto-Net, which provides automatically-tuned feed-forward neural networks without any human intervention. We report results on datasets from the recent AutoML challenge showing that ensembling Auto-Net with Auto-sklearn often performs better than either alone and report the first results on winning competition datasets against human experts with automatically-tuned neural networks.
Neuraxle - A Python Framework for Neat Machine Learning Pipelines
Authors: Chevalier, Guillaume and Brillant, Alexandre and Hamel, Eric
Date: 2019-09-01 00:00:00
Tasks: AutoML, BIG-bench Machine Learning

Code Machine Learning Pipelines - The Right Way. The present new framework is for building neat pipelines, providing the right abstractions to both ease research, development, and deployment of your ML applications. Most open-source frameworks for machine learning offer abstractions that hardly allow automatic machine learning, code reuse, and deploying models to productions. The hyperparameter search space is often strongly coupled to the whole pipeline rather than being decoupled of the pipeline and only coupled to the individual estimator objects: the pipeline steps. The need for an open-source and scaleable pipeline library is strong.
Hashing-Based Undersampling Ensemble for Imbalanced Pattern Classification Problems
Authors: Wing WY Ng, Shichao Xu, Jianjun Zhang, Xing Tian, Tongwen Rong, Sam Kwong
Date: 2020-06-29 00:00:00
Tasks: Classification, General Classification, imbalanced classification

Undersampling is a popular method to solve imbalanced classification problems. However, sometimes it may remove too many majority samples which may lead to loss of informative samples. In this article, the hashing-based undersampling ensemble (HUE) is proposed to deal with this problem by constructing diversified training subspaces for undersampling. Samples in the majority class are divided into many subspaces by a hashing method. Each subspace corresponds to a training subset which consists of most of the samples from this subspace and a few samples from surrounding subspaces. These training subsets are used to train an ensemble of classification and regression tree classifiers with all minority class samples. The proposed method is tested on 25 UCI datasets against state-of-the-art methods. Experimental results show that the HUE outperforms other methods and yields good results on highly imbalanced datasets.