pwc-archive/papers-with-abstracts huggingface dataset:
O(1) Communication for Distributed SGD through Two-Level Gradient Averaging
Authors: Subhadeep Bhattacharya, Weikuan Yu, Fahim Tahmid Chowdhury
Date: 2020-06-12 00:00:00
Tasks: Quantization, Vocal Bursts Valence Prediction

Large neural network models present a hefty communication challenge to distributed Stochastic Gradient Descent (SGD), with a communication complexity of O(n) per worker for a model of n parameters. Many sparsification and quantization techniques have been proposed to compress the gradients, some reducing the communication complexity to O(k), where k << n. In this paper, we introduce a strategy called two-level gradient averaging (A2SGD) to consolidate all gradients down to merely two local averages per worker before the computation of two global averages for an updated model. A2SGD also retains local errors to maintain the variance for fast convergence. Our theoretical analysis shows that A2SGD converges similarly like the default distributed SGD algorithm. Our evaluation validates the theoretical conclusion and demonstrates that A2SGD significantly reduces the communication traffic per worker, and improves the overall training time of LSTM-PTB by 3.2x and 23.2x, respectively, compared to Top-K and QSGD. To the best of our knowledge, A2SGD is the first to achieve O(1) communication complexity per worker for distributed SGD.
Jointly Optimizing Dataset Size and Local Updates in Heterogeneous Mobile Edge Learning
Authors: Umair Mohammad, Sameh Sorour, Mohamed Hefeida
Date: 2020-06-12 00:00:00

This paper proposes to maximize the accuracy of a distributed machine learning (ML) model trained on learners connected via the resource-constrained wireless edge. We jointly optimize the number of local/global updates and the task size allocation to minimize the loss while taking into account heterogeneous communication and computation capabilities of each learner. By leveraging existing bounds on the difference between the training loss at any given iteration and the theoretically optimal loss, we derive an expression for the objective function in terms of the number of local updates. The resulting convex program is solved to obtain the optimal number of local updates which is used to obtain the total updates and batch sizes for each learner. The merits of the proposed solution, which is heterogeneity aware (HA), are exhibited by comparing its performance to the heterogeneity unaware (HU) approach.
Evaluating a Multi-sense Definition Generation Model for Multiple Languages
Authors: Arman Kabiri, Paul Cook
Date: 2020-06-12 00:00:00
Tasks: Word Embeddings

Most prior work on definition modeling has not accounted for polysemy, or has done so by considering definition modeling for a target word in a given context. In contrast, in this study, we propose a context-agnostic approach to definition modeling, based on multi-sense word embeddings, that is capable of generating multiple definitions for a target word. In further, contrast to most prior work, which has primarily focused on English, we evaluate our proposed approach on fifteen different datasets covering nine languages from several language families. To evaluate our approach we consider several variations of BLEU. Our results demonstrate that our proposed multi-sense model outperforms a single-sense model on all fifteen datasets.
The DeepFake Detection Challenge (DFDC) Dataset
Authors: Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, Cristian Canton Ferrer
Date: 2020-06-12 00:00:00
Tasks: DeepFake Detection, Face Swapping

Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. In addition to Deepfakes, a variety of GAN-based face swapping methods have also been published with accompanying code. To counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying DeepFake Detection Challenge (DFDC) Kaggle competition. Importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-swapped dataset. The DFDC dataset is by far the largest currently and publicly available face swap video dataset, with over 100,000 total clips sourced from 3,426 paid actors, produced with several Deepfake, GAN-based, and non-learned methods. In addition to describing the methods used to construct the dataset, we provide a detailed analysis of the top submissions from the Kaggle contest. We show although Deepfake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos. Training, validation and testing corpuses can be downloaded from https://ai.facebook.com/datasets/dfdc.
Formalizing Falsification for Theories of Consciousness Across Computational Hierarchies
Authors: Jake R. Hanson, Sara I. Walker
Date: 2020-06-12 00:00:00

The scientific study of consciousness is currently undergoing a critical transition in the form of a rapidly evolving scientific debate regarding whether or not currently proposed theories can be assessed for their scientific validity. At the forefront of this debate is Integrated Information Theory (IIT), widely regarded as the preeminent theory of consciousness because of its quantification of consciousness in terms a scalar mathematical measure called $\Phi$ that is, in principle, measurable. Epistemological issues in the form of the "unfolding argument" have provided a refutation of IIT by demonstrating how it permits functionally identical systems to have differences in their predicted consciousness. The implication is that IIT and any other proposed theory based on a system's causal structure may already be falsified even in the absence of experimental refutation. However, so far the arguments surrounding the issue of falsification of theories of consciousness are too abstract to readily determine the scope of their validity. Here, we make these abstract arguments concrete by providing a simple example of functionally equivalent machines realizable with table-top electronics that take the form of isomorphic digital circuits with and without feedback. This allows us to explicitly demonstrate the different levels of abstraction at which a theory of consciousness can be assessed. Within this computational hierarchy, we show how IIT is simultaneously falsified at the finite-state automaton (FSA) level and unfalsifiable at the combinatorial state automaton (CSA) level. We use this example to illustrate a more general set of criteria for theories of consciousness: to avoid being unfalsifiable or already falsified scientific theories of consciousness must be invariant with respect to changes that leave the inference procedure fixed at a given level in a computational hierarchy.
CoDeNet: Efficient Deployment of Input-Adaptive Object Detection on Embedded FPGAs
Authors: Zhen Dong, Dequan Wang, Qijing Huang, Yizhao Gao, Yaohui Cai, Tian Li, Bichen Wu, Kurt Keutzer, John Wawrzynek
Date: 2020-06-12 00:00:00
Tasks: image-classification, Image Classification, Novel Object Detection, Object, object-detection, Object Detection

Deploying deep learning models on embedded systems has been challenging due to limited computing resources. The majority of existing work focuses on accelerating image classification, while other fundamental vision problems, such as object detection, have not been adequately addressed. Compared with image classification, detection problems are more sensitive to the spatial variance of objects, and therefore, require specialized convolutions to aggregate spatial information. To address this need, recent work introduces dynamic deformable convolution to augment regular convolutions. However, this will lead to inefficient memory accesses of inputs with existing hardware. In this work, we harness the flexibility of FPGAs to develop a novel object detection pipeline with deformable convolutions. We show the speed-accuracy tradeoffs for a set of algorithm modifications including irregular-access versus limited-range and fixed-shape. We then Co-Design a Network CoDeNet with the modified deformable convolution and quantize it to 4-bit weights and 8-bit activations. With our high-efficiency implementation, our solution reaches 26.9 frames per second with a tiny model size of 0.76 MB while achieving 61.7 AP50 on the standard object detection dataset, Pascal VOC. With our higher accuracy implementation, our model gets to 67.1 AP50 on Pascal VOC with only 2.9 MB of parameters-20.9x smaller but 10% more accurate than Tiny-YOLO.
Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling
Authors: Sina Molavipour, Germán Bassi, Mikael Skoglund
Date: 2020-06-12 00:00:00

The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a long-standing problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and has shown improvements over conventional methods. One important challenge in this new approach is the need to obtain, given the original dataset, a different set where the samples are distributed according to a specific product density function. This is particularly challenging when estimating CMI. In this paper, we introduce a new technique, based on k nearest neighbors (k-NN), to perform the resampling and derive high-confidence concentration bounds for the sample average. Then the technique is employed to train a neural network classifier and the CMI is estimated accordingly. We propose three estimators using this technique and prove their consistency, make a comparison between them and similar approaches in the literature, and experimentally show improvements in estimating the CMI in terms of accuracy and variance of the estimators.
An efficient application of Bayesian optimization to an industrial MDO framework for aircraft design
Authors: Remy Priem, Hugo Gagnon, Ian Chittick, Stephane Dufresne, Youssef Diouane, Nathalie Bartoli
Date: 2020-06-12 00:00:00
Tasks: Bayesian Optimization, global-optimization, Mixture-of-Experts

The multi-level, multi-disciplinary and multi-fidelity optimization framework developed at Bombardier Aviation has shown great results to explore efficient and competitive aircraft configurations. This optimization framework has been developed within the Isight software, the latter offers a set of ready-to-use optimizers. Unfortunately, the computational effort required by the Isight optimizers can be prohibitive with respect to the requirements of an industrial context. In this paper, a constrained Bayesian optimization optimizer, namely the super efficient global optimization with mixture of experts, is used to reduce the optimization computational effort. The obtained results showed significant improvements compared to two of the popular Isight optimizers. The capabilities of the tested constrained Bayesian optimization solver are demonstrated on Bombardier research aircraft configuration study cases.
Modeling bike availability in a bike-sharing system using machine learning
Authors: Huthaifa I. Ashqar, Mohammed Elhenawy, Mohammed H. Almannaa, Ahmed Ghanem, Hesham A. Rakha, Leanna House
Date: 2020-06-12 00:00:00
Tasks: BIG-bench Machine Learning, Prediction, regression

This paper models the availability of bikes at San Francisco Bay Area Bike Share stations using machine learning algorithms. Random Forest (RF) and Least-Squares Boosting (LSBoost) were used as univariate regression algorithms, and Partial Least-Squares Regression (PLSR) was applied as a multivariate regression algorithm. The univariate models were used to model the number of available bikes at each station. PLSR was applied to reduce the number of required prediction models and reflect the spatial correlation between stations in the network. Results clearly show that univariate models have lower error predictions than the multivariate model. However, the multivariate model results are reasonable for networks with a relatively large number of spatially correlated stations. Results also show that station neighbors and the prediction horizon time are significant predictors. The most effective prediction horizon time that produced the least prediction error was 15 minutes.
ETHOS: an Online Hate Speech Detection Dataset
Authors: Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos, Grigorios Tsoumakas
Date: 2020-06-11 00:00:00
Tasks: Hate Speech Detection

Online hate speech is a recent problem in our society that is rising at a steady pace by leveraging the vulnerabilities of the corresponding regimes that characterise most social media platforms. This phenomenon is primarily fostered by offensive comments, either during user interaction or in the form of a posted multimedia context. Nowadays, giant corporations own platforms where millions of users log in every day, and protection from exposure to similar phenomena appears to be necessary in order to comply with the corresponding legislation and maintain a high level of service quality. A robust and reliable system for detecting and preventing the uploading of relevant content will have a significant impact on our digitally interconnected society. Several aspects of our daily lives are undeniably linked to our social profiles, making us vulnerable to abusive behaviours. As a result, the lack of accurate hate speech detection mechanisms would severely degrade the overall user experience, although its erroneous operation would pose many ethical concerns. In this paper, we present 'ETHOS', a textual dataset with two variants: binary and multi-label, based on YouTube and Reddit comments validated using the Figure-Eight crowdsourcing platform. Furthermore, we present the annotation protocol used to create this dataset: an active sampling procedure for balancing our data in relation to the various aspects defined. Our key assumption is that, even gaining a small amount of labelled data from such a time-consuming process, we can guarantee hate speech occurrences in the examined material.
An Unsupervised Machine Learning Approach to Assess the ZIP Code Level Impact of COVID-19 in NYC
Authors: Fadoua Khmaissia, Pegah Sagheb Haghighi, Aarthe Jayaprakash, Zhenwei Wu, Sokratis Papadopoulos, Yuan Lai, Freddy T. Nguyen
Date: 2020-06-11 00:00:00
Tasks: BIG-bench Machine Learning, Clustering

New York City has been recognized as the world's epicenter of the novel Coronavirus pandemic. To identify the key inherent factors that are highly correlated to the Increase Rate of COVID-19 new cases in NYC, we propose an unsupervised machine learning framework. Based on the assumption that ZIP code areas with similar demographic, socioeconomic, and mobility patterns are likely to experience similar outbreaks, we select the most relevant features to perform a clustering that can best reflect the spread, and map them down to 9 interpretable categories. We believe that our findings can guide policy makers to promptly anticipate and prevent the spread of the virus by taking the right measures.
Calibration of the von Wolffersdorff model using Genetic Algorithms
Authors: Francisco J. Mendez, Antonio Pasculli, Miguel A. Mendez, Nicola Sciarra
Date: 2020-06-10 00:00:00
Tasks: Sand, Uncertainty Quantification

This article proposes an optimization framework, based on Genetic Algorithms (GA), to calibrate the constitutive law of von Wolffersdorff. This constitutive law is known as Sand Hypoplasticity (SH), and allows for robust and accurate modeling of the soil behavior but requires a complex calibration involving eight parameters. The proposed optimization can automatically fit these parameters from the results of an oedometric and a triaxial drained compression test, by combining the GA with a numerical solver that integrates the SH in the test conditions. By repeating the same calibration several times, the stochastic nature of the optimizer enables the uncertainty quantification of the calibration parameters and allows studying their relative importance on the model prediction. After validating the numerical solver on the ExCaliber-Laboratory software from the SoilModels' website, the GA calibration is tested on a synthetic dataset to analyze the convergence and the statistics of the results. In particular, a correlation analysis reveals that two couples of the eight model parameters are strongly correlated. Finally, the calibration procedure is tested on the results from von Wolffersdorff, 1996, and Herle & Gudehus, 1999, on the Hochstetten sand. The model parameters identified by the Genetic Algorithm optimization improves the matching with the experimental data and hence lead to a better calibration.
Physics informed deep learning for computational elastodynamics without labeled data
Authors: Chengping Rao, Hao Sun, Yang Liu
Date: 2020-06-10 00:00:00
Tasks: Deep Learning, Philosophy

Numerical methods such as finite element have been flourishing in the past decades for modeling solid mechanics problems via solving governing partial differential equations (PDEs). A salient aspect that distinguishes these numerical methods is how they approximate the physical fields of interest. Physics-informed deep learning is a novel approach recently developed for modeling PDE solutions and shows promise to solve computational mechanics problems without using any labeled data. The philosophy behind it is to approximate the quantity of interest (e.g., PDE solution variables) by a deep neural network (DNN) and embed the physical law to regularize the network. To this end, training the network is equivalent to minimization of a well-designed loss function that contains the PDE residuals and initial/boundary conditions (I/BCs). In this paper, we present a physics-informed neural network (PINN) with mixed-variable output to model elastodynamics problems without resort to labeled data, in which the I/BCs are hardly imposed. In particular, both the displacement and stress components are taken as the DNN output, inspired by the hybrid finite element analysis, which largely improves the accuracy and trainability of the network. Since the conventional PINN framework augments all the residual loss components in a "soft" manner with Lagrange multipliers, the weakly imposed I/BCs cannot not be well satisfied especially when complex I/BCs are present. To overcome this issue, a composite scheme of DNNs is established based on multiple single DNNs such that the I/BCs can be satisfied forcibly in a "hard" manner. The propose PINN framework is demonstrated on several numerical elasticity examples with different I/BCs, including both static and dynamic problems as well as wave propagation in truncated domains. Results show the promise of PINN in the context of computational mechanics applications.
Jointly Predicting Job Performance, Personality, Cognitive Ability, Affect, and Well-Being
Authors: Pablo Robles-Granda, Suwen Lin, Xian Wu, Sidney D'Mello, Gonzalo J. Martinez, Koustuv Saha, Kari Nies, Gloria Mark, Andrew T. Campbell, Munmun De Choudhury, Anind D. Dey, Julie Gregg, Ted Grover, Stephen M. Mattingly, Shayan Mirjafari, Edward Moskal, Aaron Striegel, Nitesh V. Chawla
Date: 2020-06-10 00:00:00

Assessment of job performance, personalized health and psychometric measures are domains where data-driven and ubiquitous computing exhibits the potential of a profound impact in the future. Existing techniques use data extracted from questionnaires, sensors (wearable, computer, etc.), or other traits, to assess well-being and cognitive attributes of individuals. However, these techniques can neither predict individual's well-being and psychological traits in a global manner nor consider the challenges associated to processing the data available, that is incomplete and noisy. In this paper, we create a benchmark for predictive analysis of individuals from a perspective that integrates: physical and physiological behavior, psychological states and traits, and job performance. We design data mining techniques as benchmark and uses real noisy and incomplete data derived from wearable sensors to predict 19 constructs based on 12 standardized well-validated tests. The study included 757 participants who were knowledge workers in organizations across the USA with varied work roles. We developed a data mining framework to extract the meaningful predictors for each of the 19 variables under consideration. Our model is the first benchmark that combines these various instrument-derived variables in a single framework to understand people's behavior by leveraging real uncurated data from wearable, mobile, and social media sources. We verify our approach experimentally using the data obtained from our longitudinal study. The results show that our framework is consistently reliable and capable of predicting the variables under study better than the baselines when prediction is restricted to the noisy, incomplete data.
Tamil Vowel Recognition With Augmented MNIST-like Data Set
Authors: Muthiah Annamalai
Date: 2020-06-09 00:00:00
Tasks: Classification, General Classification, Optical Character Recognition (OCR)

We report generation of a MNIST [4] compatible data set [1] for Tamil vowels to enable building a classification DNN or other such ML/AI deep learning [2] models for Tamil OCR/Handwriting applications. We report the capability of the 60,000 grayscale, 28x28 pixel dataset to build a 92% accuracy (training) and 82% cross-validation 4-layer CNN, with 100,000+ parameters, in TensorFlow. We also report a top-1 classification accuracy of 70% and top-2 classification accuracy of 92% on handwritten vowels showing, for the same network.
An Augmented Translation Technique for low Resource language pair: Sanskrit to Hindi translation
Authors: Rashi Kumar, Piyush Jha, Vineet Sahula
Date: 2020-06-09 00:00:00
Tasks: Dimensionality Reduction, Machine Translation, NMT, Translation

Neural Machine Translation (NMT) is an ongoing technique for Machine Translation (MT) using enormous artificial neural network. It has exhibited promising outcomes and has shown incredible potential in solving challenging machine translation exercises. One such exercise is the best approach to furnish great MT to language sets with a little preparing information. In this work, Zero Shot Translation (ZST) is inspected for a low resource language pair. By working on high resource language pairs for which benchmarks are available, namely Spanish to Portuguese, and training on data sets (Spanish-English and English-Portuguese) we prepare a state of proof for ZST system that gives appropriate results on the available data. Subsequently the same architecture is tested for Sanskrit to Hindi translation for which data is sparse, by training the model on English-Hindi and Sanskrit-English language pairs. In order to prepare and decipher with ZST system, we broaden the preparation and interpretation pipelines of NMT seq2seq model in tensorflow, incorporating ZST features. Dimensionality reduction of word embedding is performed to reduce the memory usage for data storage and to achieve a faster training and translation cycles. In this work existing helpful technology has been utilized in an imaginative manner to execute our NLP issue of Sanskrit to Hindi translation. A Sanskrit-Hindi parallel corpus of 300 is constructed for testing. The data required for the construction of parallel corpus has been taken from the telecasted news, published on Department of Public Information, state government of Madhya Pradesh, India website.