OpenRaiser/Intern-Atlas huggingface dataset:
Large Batch Training Does Not Need Warmup
Venue: arXiv cs.LG 2020
Year: 2020

Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications. However, the optimizer converges slowly at early epochs and there is a gap between large-batch deep learning optimization heuristics and theoretical underpinnings. In this paper, we propose a novel Complete Layer-wise Adaptive Rate Scaling (CLARS) algorithm for large-batch training. We also analyze the convergence rate of the proposed method by introducing a new fine-grained analysis of gradient-based methods. Based on our analysis, we bridge the gap and illustrate the theoretical insights for three popular large-batch training techniques, including linear learning rate scaling, gradual warmup, and layer-wise adaptive rate scaling. Extensive experiments demonstrate that the proposed algorithm outperforms gradual warmup technique by a large margin and defeats the convergence of the state-of-the-art large-batch optimizer in training advanced deep neural networks (ResNet, DenseNet, MobileNet) on ImageNet dataset.
Unsupervised Community Detection with a Potts Model Hamiltonian, an Efficient Algorithmic Solution, and Applications in Digital Pathology
Venue: arXiv cs.CV 2020
Year: 2020

Unsupervised segmentation of large images using a Potts model Hamiltonian is unique in that segmentation is governed by a resolution parameter which scales the sensitivity to small clusters. Here, the input image is first modeled as a graph, which is then segmented by minimizing a Hamiltonian cost function defined on the graph and the respective segments. However, there exists no closed form solution of this optimization, and using previous iterative algorithmic solution techniques, the problem scales quadratically in the Input Length. Therefore, while Potts model segmentation gives accurate segmentation, it is grossly underutilized as an unsupervised learning technique. We propose a fast statistical down-sampling of input image pixels based on the respective color features, and a new iterative method to minimize the Potts model energy considering pixel to segment relationship. This method is generalizable and can be extended for image pixel texture features as well as spatial features. We demonstrate that this new method is highly efficient, and outperforms existing methods for Potts model based image segmentation. We demonstrate the application of our method in medical microscopy image segmentation; particularly, in segmenting renal glomerular micro-environment in renal pathology. Our method is not limited to image segmentation, and can be extended to any image/data segmentation/clustering task for arbitrary datasets with discrete features.
Anomaly Detection by One Class Latent Regularized Networks
Venue: arXiv cs.CV 2020
Year: 2020

Anomaly detection is a fundamental problem in computer vision area with many real-world applications. Given a wide range of images belonging to the normal class, emerging from some distribution, the objective of this task is to construct the model to detect out-of-distribution images belonging to abnormal instances. Semi-supervised Generative Adversarial Networks (GAN)-based methods have been gaining popularity in anomaly detection task recently. However, the training process of GAN is still unstable and challenging. To solve these issues, a novel adversarial dual autoencoder network is proposed, in which the underlying structure of training data is not only captured in latent feature space, but also can be further restricted in the space of latent representation in a discriminant manner, leading to a more accurate detector. In addition, the auxiliary autoencoder regarded as a discriminator could obtain an more stable training process. Experiments show that our model achieves the state-of-the-art results on MNIST and CIFAR10 datasets as well as GTSRB stop signs dataset.
MKPipe: A Compiler Framework for Optimizing Multi-Kernel Workloads in OpenCL for FPGA
Venue: arXiv cs.DC 2020
Year: 2020

OpenCL for FPGA enables developers to design FPGAs using a programming model similar for processors. Recent works have shown that code optimization at the OpenCL level is important to achieve high computational efficiency. However, existing works either focus primarily on optimizing single kernels or solely depend on channels to design multi-kernel pipelines. In this paper, we propose a source-to-source compiler framework, MKPipe, for optimizing multi-kernel workloads in OpenCL for FPGA. Besides channels, we propose new schemes to enable multi-kernel pipelines. Our optimizing compiler employs a systematic approach to explore the tradeoffs of these optimizations methods. To enable more efficient overlapping between kernel execution, we also propose a novel workitem/workgroup-id remapping technique. Furthermore, we propose new algorithms for throughput balancing and resource balancing to tune the optimizations upon individual kernels in the multi-kernel workloads. Our results show that our compiler-optimized multi-kernels achieve up to 3.6x (1.4x on average) speedup over the baseline, in which the kernels have already been optimized individually.
MadDroid: Characterising and Detecting Devious Ad Content for Android Apps
Venue: arXiv cs.CR 2020
Year: 2020

Advertisement drives the economy of the mobile app ecosystem. As a key component in the mobile ad business model, mobile ad content has been overlooked by the research community, which poses a number of threats, e.g., propagating malware and undesirable contents. To understand the practice of these devious ad behaviors, we perform a large-scale study on the app contents harvested through automated app testing. In this work, we first provide a comprehensive categorization of devious ad contents, including five kinds of behaviors belonging to two categories: \emph{ad loading content} and \emph{ad clicking content}. Then, we propose MadDroid, a framework for automated detection of devious ad contents. MadDroid leverages an automated app testing framework with a sophisticated ad view exploration strategy for effectively collecting ad-related network traffic and subsequently extracting ad contents. We then integrate dedicated approaches into the framework to identify devious ad contents. We have applied MadDroid to 40,000 Android apps and found that roughly 6\% of apps deliver devious ad contents, e.g., distributing malicious apps that cannot be downloaded via traditional app markets. Experiment results indicate that devious ad contents are prevalent, suggesting that our community should invest more effort into the detection and mitigation of devious ads towards building a trustworthy mobile advertising ecosystem.
Parsing as Pretraining
Venue: arXiv cs.CL 2020
Year: 2020

Recent analyses suggest that encoders pretrained for language modeling capture certain morpho-syntactic structure. However, probing frameworks for word vectors still do not report results on standard setups such as constituent and dependency parsing. This paper addresses this problem and does full parsing (on English) relying only on pretraining architectures -- and no decoding. We first cast constituent and dependency parsing as sequence tagging. We then use a single feed-forward layer to directly map word vectors to labels that encode a linearized tree. This is used to: (i) see how far we can reach on syntax modelling with just pretrained encoders, and (ii) shed some light about the syntax-sensitivity of different word vectors (by freezing the weights of the pretraining network during training). For evaluation, we use bracketing F1-score and LAS, and analyze in-depth differences across representations for span lengths and dependency displacements. The overall results surpass existing sequence tagging parsers on the PTB (93.5%) and end-to-end EN-EWT UD (78.8%).
Entropy Minimization vs. Diversity Maximization for Domain Adaptation
Venue: arXiv cs.LG 2020
Year: 2020

Entropy minimization has been widely used in unsupervised domain adaptation (UDA). However, existing works reveal that entropy minimization only may result into collapsed trivial solutions. In this paper, we propose to avoid trivial solutions by further introducing diversity maximization. In order to achieve the possible minimum target risk for UDA, we show that diversity maximization should be elaborately balanced with entropy minimization, the degree of which can be finely controlled with the use of deep embedded validation in an unsupervised manner. The proposed minimal-entropy diversity maximization (MEDM) can be directly implemented by stochastic gradient descent without use of adversarial learning. Empirical evidence demonstrates that MEDM outperforms the state-of-the-art methods on four popular domain adaptation datasets.
Proceedings of the X International Workshop on Locational Analysis and Related Problems
Venue: arXiv cs.OH 2020
Year: 2020

The International Workshop on Locational Analysis and Related Problems will take place during January 23-24, 2020 in Seville (Spain). It is organized by the Spanish Location Network and the Location Group GELOCA from the Spanish Society of Statistics and Operations Research(SEIO). The Spanish Location Network is a group of more than 140 researchers from several Spanish universities organized into 7 thematic groups. The Network has been funded by the Spanish Government since 2003. One of the main activities of the Network is a yearly meeting aimed at promoting the communication among its members and between them and other researchers, and to contribute to the development of the location field and related problems. The last meetings have taken place in Cádiz (January 20-February 1, 2019), Segovia (September 27-29, 2017), Málaga (September 14-16, 2016), Barcelona (November 25-28, 2015), Sevilla (October 1-3, 2014), Torremolinos (Málaga, June 19-21, 2013), Granada (May 10-12, 2012), Las Palmas de Gran Canaria (February 2-5, 2011) and Sevilla (February 1-3, 2010). The topics of interest are location analysis and related problems. This includes location models, networks, transportation, logistics, exact and heuristic solution methods, and computational geometry, among others.
A Comprehensive Review of Smart Cities Components, Applications, and Technologies Based on Internet of Things
Venue: arXiv cs.NI 2020
Year: 2020

Smart city technology is making cities more effective which is necessary for the rapid growth in urban population. With the rapid increase in advanced metering infrastructure and other digital technologies, Smart cities have become smarter with efficient electronic devices and embedded sensors based on the Internet of Things (IoT). This paper provides a comprehensive review of the smart cities concept with its components and applications. Moreover, technologies of IoT used in smart cities infrastructure and some practically implemented smart cities in the world are mentioned as exemplary implementations. Some open issues and future directions concluded the paper.
Resource Allocation in Intelligent Reflecting Surface Assisted NOMA Systems
Venue: arXiv cs.IT 2020
Year: 2020

This paper investigates the downlink communications of intelligent reflecting surface (IRS) assisted non-orthogonal multiple access (NOMA) systems. To maximize the system throughput, we formulate a joint optimization problem over the channel assignment, decoding order of NOMA users, power allocation, and reflection coefficients. The formulated problem is proved to be NP-hard. To tackle this problem, a three-step novel resource allocation algorithm is proposed. Firstly, the channel assignment problem is solved by a many-to-one matching algorithm. Secondly, by considering the IRS reflection coefficients design, a low-complexity decoding order optimization algorithm is proposed. Thirdly, given a channel assignment and decoding order, a joint optimization algorithm is proposed for solving the joint power allocation and reflection coefficient design problem. Numerical results illustrate that: i) with the aid of IRS, the proposed IRS-NOMA system outperforms the conventional NOMA system without the IRS in terms of system throughput; ii) the proposed IRS-NOMA system achieves higher system throughput than the IRS assisted orthogonal multiple access (IRS-OMA) systems; iii) simulation results show that the performance gains of the IRS-NOMA and the IRS-OMA systems can be enhanced via carefully choosing the location of the IRS.
Human Posture Recognition and Gesture Imitation with a Humanoid Robot
Venue: arXiv cs.RO 2020
Year: 2020

This study proposes different approaches for static and dynamic gesture analysis and imitation with the social robot Nao
A Framework for In-Vivo Testing of Mobile Applications
Venue: arXiv cs.SE 2020
Year: 2020

The ecosystem in which mobile applications run is highly heterogeneous and configurable. All layers upon which mobile apps are built offer wide possibilities of variations, from the device and the hardware, to the operating system and middleware, up to the user preferences and settings. Testing all possible configurations exhaustively, before releasing the app, is unaffordable. As a consequence, the app may exhibit different, including faulty, behaviours when executed in the field, under specific configurations. In this paper, we describe a framework that can be instantiated to support in-vivo testing of a mobile app. The framework monitors the configuration in the field and triggers in-vivo testing when an untested configuration is recognized. Experimental results show that the overhead introduced by monitoring is unnoticeable to negligible (i.e., 0-6%) depending on the device being used (high- vs. low-end). In-vivo test execution required on average 3s: if performed upon screen lock activation, it introduces just a slight delay before locking the device.
Pure Resolutions, Linear Codes, and Betti Numbers
Venue: arXiv cs.IT 2020
Year: 2020

We consider the minimal free resolutions of Stanley-Reisner rings associated to linear codes and give an intrinsic characterization of linear codes having a pure resolution. We use this characterization to quickly deduce the minimal free resolutions of Stanley-Reisner rings associated to MDS codes as well as constant weight codes. We also deduce that the minimal free resolutions of Stanley-Reisner rings of first order Reed-Muller codes are pure, and explicitly describe the Betti numbers. Further, we show that in the case of higher order Reed-Muller codes, the minimal free resolutions are almost always not pure. The nature of the minimal free resolution of Stanley-Reisner rings corresponding to several classes of two-weight codes, besides the first order Reed-Muller codes, is also determined.
Online Primal Dual Meets Online Matching with Stochastic Rewards: Configuration LP to the Rescue
Venue: arXiv cs.DS 2020
Year: 2020

Mehta and Panigrahi (FOCS 2012) introduce the problem of online matching with stochastic rewards, where edges are associated with success probabilities and a match succeeds with the probability of the corresponding edge. It is one of the few online matching problems that have defied the randomized online primal dual framework by Devanur, Jain, and Kleinberg (SODA 2013) thus far. This paper unlocks the power of randomized online primal dual in online matching with stochastic rewards by employing the configuration linear program rather than the standard matching linear program used in previous works. Our main result is a 0.572 competitive algorithm for the case of vanishing and unequal probabilities, improving the best previous bound of 0.534 by Mehta, Waggoner, and Zadimoghaddam (SODA 2015) and, in fact, is even better than the best previous bound of 0.567 by Mehta and Panigrahi (FOCS 2012) for the more restricted case of vanishing and equal probabilities. For vanishing and equal probabilities, we get a better competitive ratio of 0.576. Our results further generalize to the vertex-weighted case due to the intrinsic robustness of the randomized online primal dual analysis.
Fracture in random quasibrittle media: I. Discrete mesoscale simulations of load capacity and fracture process zone
Venue: arXiv cs.CE 2020
Year: 2020

Numerical simulations of concrete fracture performed with a probabilistic mesoscale discrete model are presented. The model represents a substantial part of material randomness by assigning random locations to the largest aggregates. The remaining part of randomness is introduced by causing material parameters to fluctuate randomly via a homogeneous random field. An extensive numerical study performed with the model considers prisms loaded in uniaxial tension with both fixed and rotating platens, and also beams with and without a notch loaded in three point bending. The results show the nontrivial effect of (i) autocorrelation length and (ii) variance of the random field on the fracture behavior of the model. Statistics of the peak load are presented as well as the size and shape of the fracture process zone at the moment when the maximum load is attained. Local averaging within the fracture process zone and weakest-link are identified as underlying mechanisms explaining the reported results. The companion paper, Part II [64], introduces an analytical model capable of predicting the distribution of the peak load obtained with the probabilistic discrete model via the simple estimation of extremes of a random field obtained as moving average of local strength.
What prevents Finnish women from applying to software engineering roles? A preliminary analysis of survey data
Venue: arXiv cs.SE 2020
Year: 2020

Finland is considered a country with a good track record in gender equality. Whilst statistics support the notion that Finland is performing well compared to many other countries in terms of workplace equality, there are still many areas for improvement. This paper focuses on the problems that some women face in obtaining software engineering roles. We report a preliminary analysis of survey data from 252 respondents. These are mainly women who have shown an interest in gaining programming roles by joining the Mimmit koodaa initiative, which aims to increase equality and diversity within the software industry. The survey sought to understand what early experiences may influence later career choices and feelings of efficacy and confidence needed to pursue technology-related careers. These initial findings reveal that women's feelings of computing self-efficacy and attitudes towards software engineering are shaped by early experiences. More negative experiences decrease the likelihood of working in software engineering roles in the future, despite expressing an interest in the field.
Pure Differentially Private Summation from Anonymous Messages
Venue: arXiv cs.CR 2020
Year: 2020

The shuffled (aka anonymous) model has recently generated significant interest as a candidate distributed privacy framework with trust assumptions better than the central model but with achievable errors smaller than the local model. We study pure differentially private (DP) protocols in the shuffled model for summation, a basic and widely used primitive: - For binary summation where each of n users holds a bit as an input, we give a pure $ε$-DP protocol for estimating the number of ones held by the users up to an error of $O_ε(1)$, and each user sends $O_ε(\log n)$ messages each of 1 bit. This is the first pure protocol in the shuffled model with error $o(\sqrt{n})$ for constant $ε$. Using this protocol, we give a pure $ε$-DP protocol that performs summation of real numbers in $[0, 1]$ up to an error of $O_ε(1)$, and where each user sends $O_ε(\log^3 n)$ messages each of $O(\log\log n)$ bits. - In contrast, we show that for any pure $ε$-DP protocol for binary summation in the shuffled model having absolute error $n^{0.5-Ω(1)}$, the per user communication has to be at least $Ω_ε(\sqrt{\log n})$ bits. This implies the first separation between the (bounded-communication) multi-message shuffled model and the central model, and the first separation between pure and approximate DP protocols in the shuffled model. To prove our lower bound, we consider (a generalization of) the following question: given $γ$ in $(0, 1)$, what is the smallest m for which there are two random variables $X^0, X^1$ supported on $\{0, \dots ,m\}$ such that (i) the total variation distance between $X^0$ and $X^1$ is at least $1-γ$, and (ii) the moment generating functions of $X^0$ and $X^1$ are within a constant factor of each other everywhere? We show that the answer is $m = Θ(\sqrt{\log(1/γ)})$.
Defending Adversarial Attacks via Semantic Feature Manipulation
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning models have demonstrated vulnerability to adversarial attacks, more specifically misclassification of adversarial examples. In this paper, we propose a one-off and attack-agnostic Feature Manipulation (FM)-Defense to detect and purify adversarial examples in an interpretable and efficient manner. The intuition is that the classification result of a normal image is generally resistant to non-significant intrinsic feature changes, e.g., varying thickness of handwritten digits. In contrast, adversarial examples are sensitive to such changes since the perturbation lacks transferability. To enable manipulation of features, a combo-variational autoencoder is applied to learn disentangled latent codes that reveal semantic features. The resistance to classification change over the morphs, derived by varying and reconstructing latent codes, is used to detect suspicious inputs. Further, combo-VAE is enhanced to purify the adversarial examples with good quality by considering both class-shared and class-unique features. We empirically demonstrate the effectiveness of detection and the quality of purified instance. Our experiments on three datasets show that FM-Defense can detect nearly $100\%$ of adversarial examples produced by different state-of-the-art adversarial attacks. It achieves more than $99\%$ overall purification accuracy on the suspicious instances that close the manifold of normal examples.
Rényi Entropy Bounds on the Active Learning Cost-Performance Tradeoff
Venue: arXiv cs.IT 2020
Year: 2020

Semi-supervised classification, one of the most prominent fields in machine learning, studies how to combine the statistical knowledge of the often abundant unlabeled data with the often limited labeled data in order to maximize overall classification accuracy. In this context, the process of actively choosing the data to be labeled is referred to as active learning. In this paper, we initiate the non-asymptotic analysis of the optimal policy for semi-supervised classification with actively obtained labeled data. Considering a general Bayesian classification model, we provide the first characterization of the jointly optimal active learning and semi-supervised classification policy, in terms of the cost-performance tradeoff driven by the label query budget (number of data items to be labeled) and overall classification accuracy. Leveraging recent results on the Rényi Entropy, we derive tight information-theoretic bounds on such active learning cost-performance tradeoff.
Stimulating Creativity with FunLines: A Case Study of Humor Generation in Headlines
Venue: arXiv cs.AI 2020
Year: 2020

Building datasets of creative text, such as humor, is quite challenging. We introduce FunLines, a competitive game where players edit news headlines to make them funny, and where they rate the funniness of headlines edited by others. FunLines makes the humor generation process fun, interactive, collaborative, rewarding and educational, keeping players engaged and providing humor data at a very low cost compared to traditional crowdsourcing approaches. FunLines offers useful performance feedback, assisting players in getting better over time at generating and assessing humor, as our analysis shows. This helps to further increase the quality of the generated dataset. We show the effectiveness of this data by training humor classification models that outperform a previous benchmark, and we release this dataset to the public.