pwc-archive/papers-with-abstracts huggingface dataset:
Interventions for Ranking in the Presence of Implicit Bias
Authors: L. Elisa Celis, Anay Mehrotra, Nisheeth K. Vishnoi
Date: 2020-01-23 00:00:00

Implicit bias is the unconscious attribution of particular qualities (or lack thereof) to a member from a particular social group (e.g., defined by gender or race). Studies on implicit bias have shown that these unconscious stereotypes can have adverse outcomes in various social contexts, such as job screening, teaching, or policing. Recently, (Kleinberg and Raghavan, 2018) considered a mathematical model for implicit bias and showed the effectiveness of the Rooney Rule as a constraint to improve the utility of the outcome for certain cases of the subset selection problem. Here we study the problem of designing interventions for the generalization of subset selection -- ranking -- that requires to output an ordered set and is a central primitive in various social and computational contexts. We present a family of simple and interpretable constraints and show that they can optimally mitigate implicit bias for a generalization of the model studied in (Kleinberg and Raghavan, 2018). Subsequently, we prove that under natural distributional assumptions on the utilities of items, simple, Rooney Rule-like, constraints can also surprisingly recover almost all the utility lost due to implicit biases. Finally, we augment our theoretical results with empirical findings on real-world distributions from the IIT-JEE (2009) dataset and the Semantic Scholar Research corpus.
Reducing Non-Normative Text Generation from Language Models
Authors: Xiangyu Peng, Siyan Li, Spencer Frazier, Mark Riedl
Date: 2020-01-23 00:00:00
Proceeding: INLG (ACL) 2020 12
Tasks: Language Modelling, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Generation

Large-scale, transformer-based language models such as GPT-2 are pretrained on diverse corpora scraped from the internet. Consequently, they are prone to generating non-normative text (i.e. in violation of social norms). We introduce a technique for fine-tuning GPT-2, using a policy gradient reinforcement learning technique and a normative text classifier to produce reward and punishment values. We evaluate our technique on five data sets using automated and human participant experiments. The normative text classifier is 81-90% accurate when compared to gold-standard human judgments of normative and non-normative generated text. Our normative fine-tuning technique is able to reduce non-normative text by 27-61%, depending on the data set.
Reducing the Representation Error of GAN Image Priors Using the Deep Decoder
Authors: Max Daniels, Paul Hand, Reinhard Heckel
Date: 2020-01-23 00:00:00
Tasks: Compressive Sensing, Decoder, Image Restoration

Generative models, such as GANs, learn an explicit low-dimensional representation of a particular class of images, and so they may be used as natural image priors for solving inverse problems such as image restoration and compressive sensing. GAN priors have demonstrated impressive performance on these tasks, but they can exhibit substantial representation error for both in-distribution and out-of-distribution images, because of the mismatch between the learned, approximate image distribution and the data generating distribution. In this paper, we demonstrate a method for reducing the representation error of GAN priors by modeling images as the linear combination of a GAN prior with a Deep Decoder. The deep decoder is an underparameterized and most importantly unlearned natural signal model similar to the Deep Image Prior. No knowledge of the specific inverse problem is needed in the training of the GAN underlying our method. For compressive sensing and image superresolution, our hybrid model exhibits consistently higher PSNRs than both the GAN priors and Deep Decoder separately, both on in-distribution and out-of-distribution images. This model provides a method for extensibly and cheaply leveraging both the benefits of learned and unlearned image recovery priors in inverse problems.
Compressive MRI quantification using convex spatiotemporal priors and deep auto-encoders
Authors: Mohammad Golbabaee, Guido Buonincontri, Carolin Pirkl, Marion Menzel, Bjoern Menze, Mike Davies, Pedro Gomez
Date: 2020-01-23 00:00:00
Tasks: compressed sensing, Physical Simulations, Quantitative MRI

We propose a dictionary-matching-free pipeline for multi-parametric quantitative MRI image computing. Our approach has two stages based on compressed sensing reconstruction and deep learned quantitative inference. The reconstruction phase is convex and incorporates efficient spatiotemporal regularisations within an accelerated iterative shrinkage algorithm. This minimises the under-sampling (aliasing) artefacts from aggressively short scan times. The learned quantitative inference phase is purely trained on physical simulations (Bloch equations) that are flexible for producing rich training samples. We propose a deep and compact auto-encoder network with residual blocks in order to embed Bloch manifold projections through multiscale piecewise affine approximations, and to replace the nonscalable dictionary-matching baseline. Tested on a number of datasets we demonstrate effectiveness of the proposed scheme for recovering accurate and consistent quantitative information from novel and aggressively subsampled 2D/3D quantitative MRI acquisition protocols.
Dense Residual Network: Enhancing Global Dense Feature Flow for Character Recognition
Authors: Zhao Zhang, Zemin Tang, Yang Wang, Zheng Zhang, Choujun Zhan, ZhengJun Zha, Meng Wang
Date: 2020-01-23 00:00:00

Deep Convolutional Neural Networks (CNNs), such as Dense Convolutional Networks (DenseNet), have achieved great success for image representation by discovering deep hierarchical information. However, most existing networks simply stacks the convolutional layers and hence failing to fully discover local and global feature information among layers. In this paper, we mainly explore how to enhance the local and global dense feature flow by exploiting hierarchical features fully from all the convolution layers. Technically, we propose an efficient and effective CNN framework, i.e., Fast Dense Residual Network (FDRN), for text recognition. To construct FDRN, we propose a new fast residual dense block (f-RDB) to retain the ability of local feature fusion and local residual learning of original RDB, which can reduce the computing efforts at the same time. After fully learning local residual dense features, we utilize the sum operation and several f-RDBs to define a new block termed global dense block (GDB) by imitating the construction of dense blocks to learn global dense residual features adaptively in a holistic way. Finally, we use two convolution layers to construct a down-sampling block to reduce the global feature size and extract deeper features. Extensive simulations show that FDRN obtains the enhanced recognition results, compared with other related models.
Text Extraction and Restoration of Old Handwritten Documents
Authors: Mayank Wadhwani, Debapriya Kundu, Deepayan Chakraborty, Bhabatosh Chanda
Date: 2020-01-23 00:00:00
Tasks: Image Restoration

Image restoration is very crucial computer vision task. This paper describes two novel methods for the restoration of old degraded handwritten documents using deep neural network. In addition to that, a small-scale dataset of 26 heritage letters images is introduced. The ground truth data to train the desired network is generated semi automatically involving a pragmatic combination of color transformation, Gaussian mixture model based segmentation and shape correction by using mathematical morphological operators. In the first approach, a deep neural network has been used for text extraction from the document image and later background reconstruction has been done using Gaussian mixture modeling. But Gaussian mixture modelling requires to set parameters manually, to alleviate this we propose a second approach where the background reconstruction and foreground extraction (which which includes extracting text with its original colour) both has been done using deep neural network. Experiments demonstrate that the proposed systems perform well on handwritten document images with severe degradations, even when trained with small dataset. Hence, the proposed methods are ideally suited for digital heritage preservation repositories. It is worth mentioning that, these methods can be extended easily for printed degraded documents.
Twenty Years of Network Science: A Bibliographic and Co-Authorship Network Analysis
Authors: Roland Molontay, Marcell Nagy
Date: 2020-01-23 00:00:00
Tasks: Data Visualization

Two decades ago three pioneering papers turned the attention to complex networks and initiated a new era of research, establishing an interdisciplinary field called network science. Namely, these highly-cited seminal papers were written by Watts&Strogatz, Barab\'asi&Albert, and Girvan&Newman on small-world networks, on scale-free networks and on the community structure of complex networks, respectively. In the past 20 years - due to the multidisciplinary nature of the field - a diverse but not divided network science community has emerged. In this paper, we investigate how this community has evolved over time with respect to speed, diversity and interdisciplinary nature as seen through the growing co-authorship network of network scientists (here the notion refers to a scholar with at least one paper citing at least one of the three aforementioned milestone papers). After providing a bibliographic analysis of 31,763 network science papers, we construct the co-authorship network of 56,646 network scientists and we analyze its topology and dynamics. We shed light on the collaboration patterns of the last 20 years of network science by investigating numerous structural properties of the co-authorship network and by using enhanced data visualization techniques. We also identify the most central authors, the largest communities, investigate the spatiotemporal changes, and compare the properties of the network to scientometric indicators.
Using a Generative Adversarial Network for CT Normalization and its Impact on Radiomic Features
Authors: Leihao Wei, Yannan Lin, William Hsu
Date: 2020-01-22 00:00:00
Tasks: Generative Adversarial Network, SSIM

Computer-Aided-Diagnosis (CADx) systems assist radiologists with identifying and classifying potentially malignant pulmonary nodules on chest CT scans using morphology and texture-based (radiomic) features. However, radiomic features are sensitive to differences in acquisitions due to variations in dose levels and slice thickness. This study investigates the feasibility of generating a normalized scan from heterogeneous CT scans as input. We obtained projection data from 40 low-dose chest CT scans, simulating acquisitions at 10%, 25% and 50% dose and reconstructing the scans at 1.0mm and 2.0mm slice thickness. A 3D generative adversarial network (GAN) was used to simultaneously normalize reduced dose, thick slice (2.0mm) images to normal dose (100%), thinner slice (1.0mm) images. We evaluated the normalized image quality using peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS). Our GAN improved perceptual similarity by 35%, compared to a baseline CNN method. Our analysis also shows that the GAN-based approach led to a significantly smaller error (p-value < 0.05) in nine studied radiomic features. These results indicated that GANs could be used to normalize heterogeneous CT images and reduce the variability in radiomic feature values.
Reliable and Energy Efficient MLC STT-RAM Buffer for CNN Accelerators
Authors: Masoomeh Jasemi, Shaahin Hessabi, Nader Bagherzadeh
Date: 2020-01-14 00:00:00

We propose a lightweight scheme where the formation of a data block is changed in such a way that it can tolerate soft errors significantly better than the baseline. The key insight behind our work is that CNN weights are normalized between -1 and 1 after each convolutional layer, and this leaves one bit unused in half-precision floating-point representation. By taking advantage of the unused bit, we create a backup for the most significant bit to protect it against the soft errors. Also, considering the fact that in MLC STT-RAMs the cost of memory operations (read and write), and reliability of a cell are content-dependent (some patterns take larger current and longer time, while they are more susceptible to soft error), we rearrange the data block to minimize the number of costly bit patterns. Combining these two techniques provides the same level of accuracy compared to an error-free baseline while improving the read and write energy by 9% and 6%, respectively.
Weakly Supervised Learning Meets Ride-Sharing User Experience Enhancement
Authors: Lan-Zhe Guo, Feng Kuang, Zhang-Xun Liu, Yu-Feng Li, Nan Ma, Xiao-Hu Qie
Date: 2020-01-19 00:00:00
Tasks: Weakly-supervised Learning

Weakly supervised learning aims at coping with scarce labeled data. Previous weakly supervised studies typically assume that there is only one kind of weak supervision in data. In many applications, however, raw data usually contains more than one kind of weak supervision at the same time. For example, in user experience enhancement from Didi, one of the largest online ride-sharing platforms, the ride comment data contains severe label noise (due to the subjective factors of passengers) and severe label distribution bias (due to the sampling bias). We call such a problem as "compound weakly supervised learning". In this paper, we propose the CWSL method to address this problem based on Didi ride-sharing comment data. Specifically, an instance reweighting strategy is employed to cope with severe label noise in comment data, where the weights for harmful noisy instances are small. Robust criteria like AUC rather than accuracy and the validation performance are optimized for the correction of biased data label. Alternating optimization and stochastic gradient methods accelerate the optimization on large-scale data. Experiments on Didi ride-sharing comment data clearly validate the effectiveness. We hope this work may shed some light on applying weakly supervised learning to complex real situations.
Identification of Chimera using Machine Learning
Authors: M. A. Ganaie, Saptarshi Ghosh, Naveen Mendola, M. Tanveer, Sarika Jalan
Date: 2020-01-16 00:00:00
Tasks: BIG-bench Machine Learning

Chimera state refers to coexistence of coherent and non-coherent phases in identically coupled dynamical units found in various complex dynamical systems. Identification of Chimera, on one hand is essential due to its applicability in various areas including neuroscience, and on other hand is challenging due to its widely varied appearance in different systems and the peculiar nature of its profile. Therefore, a simple yet universal method for its identification remains an open problem. Here, we present a very distinctive approach using machine learning techniques to characterize different dynamical phases and identify the chimera state from given spatial profiles generated using various different models. The experimental results show that the performance of the classification algorithms varies for different dynamical models. The machine learning algorithms, namely random forest, oblique random forest based on tikhonov, parallel-axis split and null space regularization achieved more than $96\% $ accuracy for the Kuramoto model. For the logistic-maps, random forest and tikhonov regularization based oblique random forest showed more than $90\%$ accuracy, and for the H\'enon-Map model, random forest, null-space and axis-parallel split regularization based oblique random forest achieved more than $80\%$ accuracy. The oblique random forest with null space regularization achieved consistent performance (more than $83\%$ accuracy) across different dynamical models while the auto-encoder based random vector functional link neural network showed relatively lower performance. This work provides a direction for employing machine learning techniques to identify dynamical patterns arising in coupled non-linear units on large-scale, and for characterizing complex spatio-temporal patterns in real-world systems for various applications.
Forecasting Corn Yield with Machine Learning Ensembles
Authors: Mohsen Shahhosseini, Guiping Hu, Sotirios V. Archontoulis
Date: 2020-01-18 00:00:00
Tasks: BIG-bench Machine Learning, Feature Importance

The emerge of new technologies to synthesize and analyze big data with high-performance computing, has increased our capacity to more accurately predict crop yields. Recent research has shown that Machine learning (ML) can provide reasonable predictions, faster, and with higher flexibility compared to simulation crop modeling. The earlier the prediction during the growing season the better, but this has not been thoroughly investigated as previous studies considered all data available to predict yields. This paper provides a machine learning based framework to forecast corn yields in three US Corn Belt states (Illinois, Indiana, and Iowa) considering complete and partial in-season weather knowledge. Several ensemble models are designed using blocked sequential procedure to generate out-of-bag predictions. The forecasts are made in county-level scale and aggregated for agricultural district, and state level scales. Results show that ensemble models based on weighted average of the base learners outperform individual models. Specifically, the proposed ensemble model could achieve best prediction accuracy (RRMSE of 7.8%) and least mean bias error (-6.06 bu/acre) compared to other developed models. Comparing our proposed model forecasts with the literature demonstrates the superiority of forecasts made by our proposed ensemble model. Results from the scenario of having partial in-season weather knowledge reveal that decent yield forecasts can be made as early as June 1st. To find the marginal effect of each input feature on the forecasts made by the proposed ensemble model, a methodology is suggested that is the basis for finding feature importance for the ensemble model. The findings suggest that weather features corresponding to weather in weeks 18-24 (May 1st to June 1st) are the most important input features.
Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes
Authors: Juergen Schmidhuber
Date: 2008-12-23 00:00:00

I argue that data becomes temporarily interesting by itself to some self-improving, but computationally limited, subjective observer once he learns to predict or compress the data in a better way, thus making it subjectively simpler and more beautiful. Curiosity is the desire to create or discover more non-random, non-arbitrary, regular data that is novel and surprising not in the traditional sense of Boltzmann and Shannon but in the sense that it allows for compression progress because its regularity was not yet known. This drive maximizes interestingness, the first derivative of subjective beauty or compressibility, that is, the steepness of the learning curve. It motivates exploring infants, pure mathematicians, composers, artists, dancers, comedians, yourself, and (since 1990) artificial systems.
An Attention-Enhanced Recurrent Graph Convolutional Network for Skeleton-Based Action Recognition
Authors: Xiaolu Ding, Kai Yang, Wai Chen
Date: 2019-11-27 00:00:00
Proceeding: SPML '19: Proceedings of the 2019 2nd International Conference on Signal Processing and Machine Learning 2019 11
Tasks: Action Recognition, Skeleton Based Action Recognition

Dynamic movements of human skeleton have attracted more and more attention as a robust modality for action recognition. As not all temporal stages and skeleton joints are informative for action recognition, and the irrelevant information often brings noise which can degrade the detection performance, extracting discriminative temporal and spatial features becomes an important task. In this paper, we propose a novel end-to-end attention-enhanced recurrent graph convolutional network (AR-GCN) for skeleton-based action recognition. An attention-enhanced mechanism is employed in AR-GCN to pay different levels of attention to different temporal stages and spatial joints. This approach overcomes the information loss caused by only using keyframes and key joints. In particular, AR-GCN combines the graph convolutional network (GCN) with the bidirectional recurrent neural network (BRNN), which retains the irregular joints expressive power of the original GCN, while promoting its sequential modeling ability by introducing a recurrent network. Experimental results demonstrate the effectiveness of our proposed model on the widely used NTU and Kinetics
Weighted Sum-Rate Optimization for Intelligent Reflecting Surface Enhanced Wireless Networks
Authors: Huayan Guo, Ying-Chang Liang, Jie Chen, Erik G. Larsson
Date: 2019-05-20 00:00:00

Intelligent reflecting surface (IRS) is a promising solution to build a programmable wireless environment for future communication systems. In practice, an IRS consists of massive low-cost elements, which can steer the incident signal in fully customizable ways by passive beamforming. In this paper, we consider an IRS-aided multiuser multiple-input single-output (MISO) downlink communication system. In particular, the weighted sum-rate of all users is maximized by joint optimizing the active beamforming at the base-station (BS) and the passive beamforming at the IRS. In addition, we consider a practical IRS assumption, in which the passive elements can only shift the incident signal to discrete phase levels. This non-convex problem is firstly decoupled via Lagrangian dual transform, and then the active and passive beamforming can be optimized alternatingly. The active beamforming at BS is optimized based on the fractional programming method. Then, three efficient algorithms with closed-form expressions are proposed for the passive beamforming at IRS. Simulation results have verified the effectiveness of the proposed algorithms as compared to different benchmark schemes.
Towards a Human-like Open-Domain Chatbot
Authors: Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, Quoc V. Le
Date: 2020-01-27 00:00:00
Tasks: Chatbot, Specificity

We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We also propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of a human-like multi-turn conversation. Our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72% on multi-turn evaluation) suggests that a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher in absolute SSA than the existing chatbots we evaluated.