OpenRaiser/Intern-Atlas huggingface dataset:
Feature Selection on Noisy Twitter Short Text Messages for Language Identification
Venue: arXiv cs.CL 2020
Year: 2020

The task of written language identification involves typically the detection of the languages present in a sample of text. Moreover, a sequence of text may not belong to a single inherent language but also may be mixture of text written in multiple languages. This kind of text is generated in large volumes from social media platforms due to its flexible and user friendly environment. Such text contains very large number of features which are essential for development of statistical, probabilistic as well as other kinds of language models. The large number of features have rich as well as irrelevant and redundant features which have diverse effect over the performance of the learning model. Therefore, feature selection methods are significant in choosing feature that are most relevant for an efficient model. In this article, we basically consider the Hindi-English language identification task as Hindi and English are often two most widely spoken languages of India. We apply different feature selection algorithms across various learning algorithms in order to analyze the effect of the algorithm as well as the number of features on the performance of the task. The methodology focuses on the word level language identification using a novel dataset of 6903 tweets extracted from Twitter. Various n-gram profiles are examined with different feature selection algorithms over many classifiers. Finally, an exhaustive comparative analysis is put forward with respect to the overall experiments conducted for the task.
Usefulness of interpretability methods to explain deep learning based plant stress phenotyping
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning techniques have been successfully deployed for automating plant stress identification and quantification. In recent years, there is a growing push towards training models that are interpretable -i.e. that justify their classification decisions by visually highlighting image features that were crucial for classification decisions. The expectation is that trained network models utilize image features that mimic visual cues used by plant pathologists. In this work, we compare some of the most popular interpretability methods: Saliency Maps, SmoothGrad, Guided Backpropogation, Deep Taylor Decomposition, Integrated Gradients, Layer-wise Relevance Propagation and Gradient times Input, for interpreting the deep learning model. We train a DenseNet-121 network for the classification of eight different soybean stresses (biotic and abiotic). Using a dataset consisting of 16,573 RGB images of healthy and stressed soybean leaflets captured under controlled conditions, we obtained an overall classification accuracy of 95.05 \%. For a diverse subset of the test data, we compared the important features with those identified by a human expert. We observed that most interpretability methods identify the infected regions of the leaf as important features for some -- but not all -- of the correctly classified images. For some images, the output of the interpretability methods indicated that spurious feature correlations may have been used to correctly classify them. Although the output explanation maps of these interpretability methods may be different from each other for a given image, we advocate the use of these interpretability methods as `hypothesis generation' mechanisms that can drive scientific insight.
Enhanced Behavioral Cloning Based self-driving Car Using Transfer Learning
Venue: arXiv cs.CV 2020
Year: 2020

With the growing phase of artificial intelligence and autonomous learning, the self-driving car is one of the promising area of research and emerging as a center of focus for automobile industries. Behavioral cloning is the process of replicating human behavior via visuomotor policies by means of machine learning algorithms. In recent years, several deep learning-based behavioral cloning approaches have been developed in the context of self-driving cars specifically based on the concept of transfer learning. Concerning the same, the present paper proposes a transfer learning approach using VGG16 architecture, which is fine tuned by retraining the last block while keeping other blocks as non-trainable. The performance of proposed architecture is further compared with existing NVIDIA architecture and its pruned variants (pruned by 22.2% and 33.85% using 1x1 filter to decrease the total number of parameters). Experimental results show that the VGG16 with transfer learning architecture has outperformed other discussed approaches with faster convergence.
Exploit the potential of Multi-column architecture for Crowd Counting
Venue: arXiv cs.CV 2020
Year: 2020

Crowd counting is an important yet challenging task in computer vision due to serious occlusions, complex background and large scale variations, etc. Multi-column architecture is widely adopted to overcome these challenges, yielding state-of-the-art performance in many public benchmarks. However, there still are two issues in such design: scale limitation and feature similarity. Further performance improvements are thus restricted. In this paper, we propose a novel crowd counting framework called Pyramid Scale Network (PSNet) to explicitly address these issues. Specifically, for scale limitation, we adopt three Pyramid Scale Modules (PSM) to efficiently capture multi-scale features, which integrate a message passing mechanism and an attention mechanism into multi-column architecture. Moreover, for feature similarity, a novel loss function named Multi-column variance loss is introduced to make the features learned by each column in PSM appropriately different from each other. To the best of our knowledge, PSNet is the first work to explicitly address scale limitation and feature similarity in multi-column design. Extensive experiments on five benchmark datasets demonstrate the effectiveness of the proposed innovations as well as the superior performance over the state-of-the-art. Our code is publicly available at: https://github.com/oahunc/Pyramid_Scale_Network
Generalization of Deep Convolutional Neural Networks -- A Case-study on Open-source Chest Radiographs
Venue: arXiv cs.CV 2020
Year: 2020

Deep Convolutional Neural Networks (DCNNs) have attracted extensive attention and been applied in many areas, including medical image analysis and clinical diagnosis. One major challenge is to conceive a DCNN model with remarkable performance on both internal and external data. We demonstrate that DCNNs may not generalize to new data, but increasing the quality and heterogeneity of the training data helps to improve the generalizibility factor. We use InceptionResNetV2 and DenseNet121 architectures to predict the risk of 5 common chest pathologies. The experiments were conducted on three publicly available databases: CheXpert, ChestX-ray14, and MIMIC Chest Xray JPG. The results show the internal performance of each of the 5 pathologies outperformed external performance on both of the models. Moreover, our strategy of exposing the models to a mix of different datasets during the training phase helps to improve model performance on the external dataset.
Optimizing Prediction Serving on Low-Latency Serverless Dataflow
Venue: arXiv cs.DC 2020
Year: 2020

Prediction serving systems are designed to provide large volumes of low-latency inferences machine learning models. These systems mix data processing and computationally intensive model inference and benefit from multiple heterogeneous processors and distributed computing resources. In this paper, we argue that a familiar dataflow API is well-suited to this latency-sensitive task, and amenable to optimization even with unmodified black-box ML models. We present the design of Cloudflow, a system that provides this API and realizes it on an autoscaling serverless backend. Cloudflow transparently implements performance-critical optimizations including operator fusion and competitive execution. Our evaluation shows that Cloudflow's optimizations yield significant performance improvements on synthetic workloads and that Cloudflow outperforms state-of-the-art prediction serving systems by as much as 2x on real-world prediction pipelines, meeting latency goals of demanding applications like real-time video analysis.
A subquadratic algorithm for the simultaneous conjugacy problem
Venue: arXiv cs.DS 2020
Year: 2020

The $d$-Simultaneous Conjugacy problem in the symmetric group $S_n$ asks whether there exists a permutation $τ\in S_n$ such that $b_j = τ^{-1}a_j τ$ holds for all $j = 1,2,\ldots, d$, where $a_1, a_2,\ldots , a_d$ and $b_1, b_2,\ldots , b_d$ are given sequences of permutations in $S_n$. The time complexity of existing algorithms for solving the problem is $O(dn^2)$. We show that for a given positive integer $d$ the $d$-Simultaneous Conjugacy problem in $S_n$ can be solved in $o(n^2)$ time.
Is Machine Learning Speaking my Language? A Critical Look at the NLP-Pipeline Across 8 Human Languages
Venue: arXiv cs.CL 2020
Year: 2020

Natural Language Processing (NLP) is increasingly used as a key ingredient in critical decision-making systems such as resume parsers used in sorting a list of job candidates. NLP systems often ingest large corpora of human text, attempting to learn from past human behavior and decisions in order to produce systems that will make recommendations about our future world. Over 7000 human languages are being spoken today and the typical NLP pipeline underrepresents speakers of most of them while amplifying the voices of speakers of other languages. In this paper, a team including speakers of 8 languages - English, Chinese, Urdu, Farsi, Arabic, French, Spanish, and Wolof - takes a critical look at the typical NLP pipeline and how even when a language is technically supported, substantial caveats remain to prevent full participation. Despite huge and admirable investments in multilingual support in many tools and resources, we are still making NLP-guided decisions that systematically and dramatically underrepresent the voices of much of the world.
Denoising as well as the best of any two denoisers
Venue: arXiv cs.IT 2020
Year: 2020

Given two arbitrary sequences of denoisers for block lengths tending to infinity we ask if it is possible to construct a third sequence of denoisers with an asymptotically vanishing (in block length) excess expected loss relative to the best expected loss of the two given denoisers for all clean channel input sequences. As in the setting of DUDE [1], which solves this problem when the given denoisers are sliding block denoisers, the construction is allowed to depend on the two given denoisers and the channel transition probabilities. We show that under certain restrictions on the two given denoisers the problem can be solved using a straightforward application of a known loss estimation paradigm. We then show by way of a counter-example that the loss estimation approach fails in the general case. Finally, we show that for the binary symmetric channel, combining the loss estimation with a randomization step leads to a solution to the stated problem under no restrictions on the given denoisers.
Does presence of social media plugins in a journal website result in higher social media attention of its research publications
Venue: arXiv cs.DL 2020
Year: 2020

Social media platforms have now emerged as an important medium for wider dissemination of research articles; with authors, readers and publishers creating different kinds of social media activity about the article. Some research studies have even shown that articles that get more social media attention may get higher visibility and citations. These factors are now persuading journal publishers to integrate social media plugins in their webpages to facilitate sharing and dissemination of articles in social media platforms. Many past studies have analyzed several factors (like journal impact factor, open access, collaboration etc.) that may impact social media attention of scholarly articles. However, there are no studies to analyze whether the presence of social media plugin in a journal could result in higher social media attention of articles published in the journal. This paper aims to bridge this gap in knowledge by analyzing a sufficiently large-sized sample of 99,749 articles from 100 different journals. Results obtained show that journals that have social media plugins integrated in their webpages get significantly higher social media mentions and shares for their articles as compared to journals that do not provide such plugins. Authors and readers visiting journal webpages appear to be a major contributor to social media activity around articles published in such journals. The results suggest that publishing houses should actively provide social media plugin integration in their journal webpages to increase social media visibility (altmetric impact) of their articles.
It Is Likely That Your Loss Should be a Likelihood
Venue: arXiv cs.LG 2020
Year: 2020

Many common loss functions such as mean-squared-error, cross-entropy, and reconstruction loss are unnecessarily rigid. Under a probabilistic interpretation, these common losses correspond to distributions with fixed shapes and scales. We instead argue for optimizing full likelihoods that include parameters like the normal variance and softmax temperature. Joint optimization of these "likelihood parameters" with model parameters can adaptively tune the scales and shapes of losses in addition to the strength of regularization. We explore and systematically evaluate how to parameterize and apply likelihood parameters for robust modeling, outlier-detection, and re-calibration. Additionally, we propose adaptively tuning $L_2$ and $L_1$ weights by fitting the scale parameters of normal and Laplace priors and introduce more flexible element-wise regularizers.
Recognizing $k$-Clique Extendible Orderings
Venue: arXiv cs.DS 2020
Year: 2020

A graph is $k$-clique-extendible if there is an ordering of the vertices such that whenever two $k$-sized overlapping cliques $A$ and $B$ have $k-1$ common vertices, and these common vertices appear between the two vertices $a,b\in (A\setminus B)\cup (B\setminus A)$ in the ordering, there is an edge between $a$ and $b$, implying that $A\cup B$ is a $(k+1)$-sized clique. Such an ordering is said to be a $k$-C-E ordering. These graphs arise in applications related to modelling preference relations. Recently, it has been shown that a maximum sized clique in such a graph can be found in $n^{O(k)}$ time when the ordering is given. When $k$ is $2$, such graphs are precisely the well-known class of comparability graphs and when $k$ is $3$ they are called triangle-extendible graphs. It has been shown that triangle-extendible graphs appear as induced subgraphs of visibility graphs of simple polygons, and the complexity of recognizing them has been mentioned as an open problem in the literature. While comparability graphs (i.e. $2$-C-E graphs) can be recognized in polynomial time, we show that recognizing $k$-C-E graphs is NP-hard for any fixed $k \geq 3$ and co-NP-hard when $k$ is part of the input. While our NP-hardness reduction for $k \geq 4$ is from the betweenness problem, for $k=3$, our reduction is an intricate one from the $3$-colouring problem. We also show that the problems of determining whether a given ordering of the vertices of a graph is a $k$-C-E ordering, and that of finding an $\ell$-sized (or maximum sized) clique in a $k$-C-E graph, given a $k$-C-E ordering, are complete for the parameterized complexity classes co-W[1] and W[1] respectively, when parameterized by $k$. However we show that the former is fixed-parameter tractable when parameterized by the treewidth of the graph.
VAFL: a Method of Vertical Asynchronous Federated Learning
Venue: arXiv cs.LG 2020
Year: 2020

Horizontal Federated learning (FL) handles multi-client data that share the same set of features, and vertical FL trains a better predictor that combine all the features from different clients. This paper targets solving vertical FL in an asynchronous fashion, and develops a simple FL method. The new method allows each client to run stochastic gradient algorithms without coordination with other clients, so it is suitable for intermittent connectivity of clients. This method further uses a new technique of perturbed local embedding to ensure data privacy and improve communication efficiency. Theoretically, we present the convergence rate and privacy level of our method for strongly convex, nonconvex and even nonsmooth objectives separately. Empirically, we apply our method to FL on various image and healthcare datasets. The results compare favorably to centralized and synchronous FL methods.
Unknown I.I.D. Prophets: Better Bounds, Streaming Algorithms, and a New Impossibility
Venue: arXiv cs.DS 2020
Year: 2020

A prophet inequality states, for some $α\in[0,1]$, that the expected value achievable by a gambler who sequentially observes random variables $X_1,\dots,X_n$ and selects one of them is at least an $α$ fraction of the maximum value in the sequence. We obtain three distinct improvements for a setting that was first studied by Correa et al. (EC, 2019) and is particularly relevant to modern applications in algorithmic pricing. In this setting, the random variables are i.i.d. from an unknown distribution and the gambler has access to an additional $βn$ samples for some $β\geq 0$. We first give improved lower bounds on $α$ for a wide range of values of $β$; specifically, $α\geq(1+β)/e$ when $β\leq 1/(e-1)$, which is tight, and $α\geq 0.648$ when $β=1$, which improves on a bound of around $0.635$ due to Correa et al. (SODA, 2020). Adding to their practical appeal, specifically in the context of algorithmic pricing, we then show that the new bounds can be obtained even in a streaming model of computation and thus in situations where the use of relevant data is complicated by the sheer amount of data available. We finally establish that the upper bound of $1/e$ for the case without samples is robust to additional information about the distribution, and applies also to sequences of i.i.d. random variables whose distribution is itself drawn, according to a known distribution, from a finite set of known candidate distributions. This implies a tight prophet inequality for exchangeable sequences of random variables, answering a question of Hill and Kertz (Contemporary Mathematics, 1992), but leaves open the possibility of better guarantees when the number of candidate distributions is small, a setting we believe is of strong interest to applications.
Fine-Grained Crowd Counting
Venue: arXiv cs.CV 2020
Year: 2020

Current crowd counting algorithms are only concerned about the number of people in an image, which lacks low-level fine-grained information of the crowd. For many practical applications, the total number of people in an image is not as useful as the number of people in each sub-category. E.g., knowing the number of people waiting inline or browsing can help retail stores; knowing the number of people standing/sitting can help restaurants/cafeterias; knowing the number of violent/non-violent people can help police in crowd management. In this paper, we propose fine-grained crowd counting, which differentiates a crowd into categories based on the low-level behavior attributes of the individuals (e.g. standing/sitting or violent behavior) and then counts the number of people in each category. To enable research in this area, we construct a new dataset of four real-world fine-grained counting tasks: traveling direction on a sidewalk, standing or sitting, waiting in line or not, and exhibiting violent behavior or not. Since the appearance features of different crowd categories are similar, the challenge of fine-grained crowd counting is to effectively utilize contextual information to distinguish between categories. We propose a two branch architecture, consisting of a density map estimation branch and a semantic segmentation branch. We propose two refinement strategies for improving the predictions of the two branches. First, to encode contextual information, we propose feature propagation guided by the density map prediction, which eliminates the effect of background features during propagation. Second, we propose a complementary attention model to share information between the two branches. Experiment results confirm the effectiveness of our method.
Implementing the ICE Estimator in Multilayer Perceptron Classifiers
Venue: arXiv cs.LG 2020
Year: 2020

This paper describes the techniques used to implement the ICE estimator for a multilayer perceptron model, and reviews the performance of the resulting models. The ICE estimator is implemented in the Apache Spark MultilayerPerceptronClassifier, and shown in cross-validation to outperform the stock MultilayerPerceptronClassifier that uses unadjusted MLE (cross-entropy) loss. The resulting models have identical runtime performance, and similar fitting performance to the stock MLP implementations. Additionally, this approach requires no hyper-parameters, and is therefore viable as a drop-in replacement for cross-entropy optimizing multilayer perceptron classifiers wherever overfitting may be a concern.
Data from Model: Extracting Data from Non-robust and Robust Models
Venue: arXiv cs.CV 2020
Year: 2020

The essence of deep learning is to exploit data to train a deep neural network (DNN) model. This work explores the reverse process of generating data from a model, attempting to reveal the relationship between the data and the model. We repeat the process of Data to Model (DtM) and Data from Model (DfM) in sequence and explore the loss of feature mapping information by measuring the accuracy drop on the original validation dataset. We perform this experiment for both a non-robust and robust origin model. Our results show that the accuracy drop is limited even after multiple sequences of DtM and DfM, especially for robust models. The success of this cycling transformation can be attributed to the shared feature mapping existing in data and model. Using the same data, we observe that different DtM processes result in models having different features, especially for different network architecture families, even though they achieve comparable performance.
Embracing the Unreliability of Memory Devices for Neuromorphic Computing
Venue: arXiv cs.ET 2020
Year: 2020

The emergence of resistive non-volatile memories opens the way to highly energy-efficient computation near- or in-memory. However, this type of computation is not compatible with conventional ECC, and has to deal with device unreliability. Inspired by the architecture of animal brains, we present a manufactured differential hybrid CMOS/RRAM memory architecture suitable for neural network implementation that functions without formal ECC. We also show that using low-energy but error-prone programming conditions only slightly reduces network accuracy.