OpenRaiser/Intern-Atlas huggingface dataset:
An Incremental Construction of Deep Neuro Fuzzy System for Continual Learning of Non-stationary Data Streams
Venue: arXiv cs.AI 2019
Year: 2019

Existing FNNs are mostly developed under a shallow network configuration having lower generalization power than those of deep structures. This paper proposes a novel self-organizing deep FNN, namely DEVFNN. Fuzzy rules can be automatically extracted from data streams or removed if they play limited role during their lifespan. The structure of the network can be deepened on demand by stacking additional layers using a drift detection method which not only detects the covariate drift, variations of input space, but also accurately identifies the real drift, dynamic changes of both feature space and target space. DEVFNN is developed under the stacked generalization principle via the feature augmentation concept where a recently developed algorithm, namely gClass, drives the hidden layer. It is equipped by an automatic feature selection method which controls activation and deactivation of input attributes to induce varying subsets of input features. A deep network simplification procedure is put forward using the concept of hidden layer merging to prevent uncontrollable growth of dimensionality of input space due to the nature of feature augmentation approach in building a deep network structure. DEVFNN works in the sample-wise fashion and is compatible for data stream applications. The efficacy of DEVFNN has been thoroughly evaluated using seven datasets with non-stationary properties under the prequential test-then-train protocol. It has been compared with four popular continual learning algorithms and its shallow counterpart where DEVFNN demonstrates improvement of classification accuracy. Moreover, it is also shown that the concept drift detection method is an effective tool to control the depth of network structure while the hidden layer merging scenario is capable of simplifying the network complexity of a deep network with negligible compromise of generalization performance.
Bisplit graphs satisfy the Chen-Chvátal conjecture
Venue: arXiv cs.DM 2019
Year: 2019

In this paper, we give a lengthy proof of a small result! A graph is bisplit if its vertex set can be partitioned into three stable sets with two of them inducing a complete bipartite graph. We prove that these graphs satisfy the Chen-Chvátal conjecture: their metric space (in the usual sense) has a universal line (in an unusual sense) or at least as many lines as the number of vertices.
AutoFrame: Automatic Frame Inference for Object-Oriented Languages
Venue: arXiv cs.SE 2019
Year: 2019

Automatic program verification has made tremendous strides, but is not yet for the masses. How do we make it less painful? This article addresses one of the obstacles: the need to specify explicit "frame clauses", expressing what properties are left unchanged by an operation. It is fair enough to ask the would-be (human) prover to state what each operation changes, and how, but the (mechanical) prover also requires knowledge of what it does not change. The process of specifying and verifying these properties is tedious and error-prone, and must be repeated whenever the software evolves. it is also hard to justify, since all the information about what the code changes is in the code itself. The AutoFrame tool presented here performs this analysis entirely automatically. It applies to object-oriented programming, where the issue is compounded by aliasing: if x is aliased to y, any update to x.a also affects y.a, even though the updating instruction usually does not even mention y. This aspect turns out to be the most delicate, and is addressed in AutoFrame by taking advantage of a companion tool, AutoAlias, which performs sound and sufficiently precise alias analysis, also in an entirely automatic way. Some practical results of AutoFrame so far are: (1) the automatic reconstruction (in about 25 seconds on an ordinary laptop) of the exact frame clauses, a total of 169 clauses, for an 8,000-line data structures and algorithms library which was previously (with the manually written frame clauses) verified for functional correctness using a mechanical program prover; and (2) the automatic generation (in less than 4 minutes) of frame conditions for a 150,000-line graphical and GUI library. The source code of AutoFrame and these examples are available for download.
Sentence Embeddings in NLI with Iterative Refinement Encoders
Venue: arXiv cs.CL 2019
Year: 2019

Sentence-level representations are necessary for various NLP tasks. Recurrent neural networks have proven to be very effective in learning distributed representations and can be trained efficiently on natural language inference tasks. We build on top of one such model and propose a hierarchy of BiLSTM and max pooling layers that implements an iterative refinement strategy and yields state of the art results on the SciTail dataset as well as strong results for SNLI and MultiNLI. We can show that the sentence embeddings learned in this way can be utilized in a wide variety of transfer learning tasks, outperforming InferSent on 7 out of 10 and SkipThought on 8 out of 9 SentEval sentence embedding evaluation tasks. Furthermore, our model beats the InferSent model in 8 out of 10 recently published SentEval probing tasks designed to evaluate sentence embeddings' ability to capture some of the important linguistic properties of sentences.
Efficient Data Ingestion and Query Processing for LSM-Based Storage Systems
Venue: arXiv cs.DB 2019
Year: 2019

In recent years, the Log Structured Merge (LSM) tree has been widely adopted by NoSQL and NewSQL systems for its superior write performance. Despite its popularity, however, most existing work has focused on LSM-based key-value stores with only a primary LSM-tree index; auxiliary structures, which are critical for supporting ad-hoc queries, have received much less attention. In this paper, we focus on efficient data ingestion and query processing for general-purpose LSM-based storage systems. We first propose and evaluate a series of optimizations for efficient batched point lookups, significantly improving the range of applicability of LSM-based secondary indexes. We then present several new and efficient maintenance strategies for LSM-based storage systems. Finally, we have implemented and experimentally evaluated the proposed techniques in the context of the Apache AsterixDB system, and we present the results here.
Efficient size estimation and impossibility of termination in uniform dense population protocols
Venue: arXiv cs.DC 2019
Year: 2019

We study uniform population protocols: networks of anonymous agents whose pairwise interactions are chosen at random, where each agent uses an identical transition algorithm that does not depend on the population size $n$. Many existing polylog$(n)$ time protocols for leader election and majority computation are nonuniform: to operate correctly, they require all agents to be initialized with an approximate estimate of $n$ (specifically, the exact value $\lfloor \log n \rfloor$). Our first main result is a uniform protocol for calculating $\log(n) \pm O(1)$ with high probability in $O(\log^2 n)$ time and $O(\log^4 n)$ states ($O(\log \log n)$ bits of memory). The protocol is converging but not terminating: it does not signal when the estimate is close to the true value of $\log n$. If it could be made terminating, this would allow composition with protocols, such as those for leader election or majority, that require a size estimate initially, to make them uniform (though with a small probability of failure). We do show how our main protocol can be indirectly composed with others in a simple and elegant way, based on the leaderless phase clock, demonstrating that those protocols can in fact be made uniform. However, our second main result implies that the protocol cannot be made terminating, a consequence of a much stronger result: a uniform protocol for any task requiring more than constant time cannot be terminating even with probability bounded above 0, if infinitely many initial configurations are dense: any state present initially occupies $Ω(n)$ agents. (In particular, no leader is allowed.) Crucially, the result holds no matter the memory or time permitted. Finally, we show that with an initial leader, our size-estimation protocol can be made terminating with high probability, with the same asymptotic time and space bounds.
ARBEE: Towards Automated Recognition of Bodily Expression of Emotion In the Wild
Venue: arXiv cs.CV 2019
Year: 2019

Humans are arguably innately prepared to comprehend others' emotional expressions from subtle body movements. If robots or computers can be empowered with this capability, a number of robotic applications become possible. Automatically recognizing human bodily expression in unconstrained situations, however, is daunting given the incomplete understanding of the relationship between emotional expressions and body movements. The current research, as a multidisciplinary effort among computer and information sciences, psychology, and statistics, proposes a scalable and reliable crowdsourcing approach for collecting in-the-wild perceived emotion data for computers to learn to recognize body languages of humans. To accomplish this task, a large and growing annotated dataset with 9,876 video clips of body movements and 13,239 human characters, named BoLD (Body Language Dataset), has been created. Comprehensive statistical analysis of the dataset revealed many interesting insights. A system to model the emotional expressions based on bodily movements, named ARBEE (Automated Recognition of Bodily Expression of Emotion), has also been developed and evaluated. Our analysis shows the effectiveness of Laban Movement Analysis (LMA) features in characterizing arousal, and our experiments using LMA features further demonstrate computability of bodily expression. We report and compare results of several other baseline methods which were developed for action recognition based on two different modalities, body skeleton, and raw image. The dataset and findings presented in this work will likely serve as a launchpad for future discoveries in body language understanding that will enable future robots to interact and collaborate more effectively with humans.
Decentralized Detection with Robust Information Privacy Protection
Venue: arXiv cs.IT 2019
Year: 2019

We consider a decentralized detection network whose aim is to infer a public hypothesis of interest. However, the raw sensor observations also allow the fusion center to infer private hypotheses that we wish to protect. We consider the case where there are an uncountable number of private hypotheses belonging to an uncertainty set, and develop local privacy mappings at every sensor so that the sanitized sensor information minimizes the Bayes error of detecting the public hypothesis at the fusion center, while achieving information privacy for all private hypotheses. We introduce the concept of a most favorable hypothesis (MFH) and show how to find a MFH in the set of private hypotheses. By protecting the information privacy of the MFH, information privacy for every other private hypothesis is also achieved. We provide an iterative algorithm to find the optimal local privacy mappings, and derive some theoretical properties of these privacy mappings. Simulation results demonstrate that our proposed approach allows the fusion center to infer the public hypothesis with low error while protecting information privacy of all the private hypotheses.
DP-ADMM: ADMM-based Distributed Learning with Differential Privacy
Venue: arXiv cs.LG 2019
Year: 2019

Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local computation and message passing. Such an iterative process could cause privacy concerns of data owners. The goal of this paper is to provide differential privacy for ADMM-based distributed machine learning. Prior approaches on differentially private ADMM exhibit low utility under high privacy guarantee and often assume the objective functions of the learning problems to be smooth and strongly convex. To address these concerns, we propose a novel differentially private ADMM-based distributed learning algorithm called DP-ADMM, which combines an approximate augmented Lagrangian function with time-varying Gaussian noise addition in the iterative process to achieve higher utility for general objective functions under the same differential privacy guarantee. We also apply the moments accountant method to bound the end-to-end privacy loss. The theoretical analysis shows that DP-ADMM can be applied to a wider class of distributed learning problems, is provably convergent, and offers an explicit utility-privacy tradeoff. To our knowledge, this is the first paper to provide explicit convergence and utility properties for differentially private ADMM-based distributed learning algorithms. The evaluation results demonstrate that our approach can achieve good convergence and model accuracy under high end-to-end differential privacy guarantee.
IEA: Inner Ensemble Average within a convolutional neural network
Venue: arXiv cs.LG 2019
Year: 2019

Ensemble learning is a method of combining multiple trained models to improve model accuracy. We propose the usage of such methods, specifically ensemble average, inside Convolutional Neural Network (CNN) architectures by replacing the single convolutional layers with Inner Average Ensembles (IEA) of multiple convolutional layers. Empirical results on different benchmarking datasets show that CNN models using IEA outperform those with regular convolutional layers. A visual and a similarity score analysis of the features generated from IEA explains why it boosts the model performance.
A concise frictional contact formulation based on surface potentials and isogeometric discretization
Venue: arXiv cs.CE 2019
Year: 2019

This work presents a concise theoretical and computational framework for the finite element formulation of frictional contact problems with arbitrarily large deformation and sliding. The aim of this work is to extend the contact theory based on surface potentials (Sauer and De Lorenzis, 2013) to account for friction. Coulomb friction under isothermal conditions is considered here. For a consistent friction formulation, we start with the first and second laws of thermodynamics and derive the governing equations at the contact interface. A so-called interacting gap can then be defined as a kinematic variable unifying both sliding/sticking and normal/tangential contact. A variational principle for the frictional system can then be formulated based on a purely kinematical constraint. The direct elimination approach applied to the tangential part of this constraint leads to the so-called moving friction cone approach of Wriggers and Haraldsson (2003). Compared with existing friction formulations, our approach reduces the theoretical and computational complexity. Several numerical examples are presented to demonstrate the accuracy and robustness of the proposed friction formulation.
On $Z_pZ_{p^k}$-additive codes and their duality
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, two different Gray-like maps from $Z_p^α\times Z_{p^k}^β$, where $p$ is prime, to $Z_p^n$, $n={α+βp^{k-1}}$, denoted by $ϕ$ and $Φ$, respectively, are presented. We have determined the connection between the weight enumerators among the image codes under these two mappings. We show that if $C$ is a $Z_p Z_{p^k}$-additive code, and $C^\bot$ is its dual, then the weight enumerators of the image $p$-ary codes $ϕ(C)$ and $Φ(C^\bot)$ are formally dual. This is a partial generalization of [On $Z_{2^k}$-dual binary codes, arXiv:math/0509325], and the result is generalized to odd characteristic $p$ and mixed alphabet. Additionally, a construction of $1$-perfect additive codes in the mixed $Z_p Z_{p^2} ... Z_{p^k}$ alphabet is given.
Bilinear Recovery using Adaptive Vector-AMP
Venue: arXiv cs.IT 2019
Year: 2019

We consider the problem of jointly recovering the vector $\boldsymbol{b}$ and the matrix $\boldsymbol{C}$ from noisy measurements $\boldsymbol{Y} = \boldsymbol{A}(\boldsymbol{b})\boldsymbol{C} + \boldsymbol{W}$, where $\boldsymbol{A}(\cdot)$ is a known affine linear function of $\boldsymbol{b}$ (i.e., $\boldsymbol{A}(\boldsymbol{b})=\boldsymbol{A}_0+\sum_{i=1}^Q b_i \boldsymbol{A}_i$ with known matrices $\boldsymbol{A}_i$). This problem has applications in matrix completion, robust PCA, dictionary learning, self-calibration, blind deconvolution, joint-channel/symbol estimation, compressive sensing with matrix uncertainty, and many other tasks. To solve this bilinear recovery problem, we propose the Bilinear Adaptive Vector Approximate Message Passing (BAd-VAMP) algorithm. We demonstrate numerically that the proposed approach is competitive with other state-of-the-art approaches to bilinear recovery, including lifted VAMP and Bilinear GAMP.
Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora
Venue: arXiv cs.CL 2019
Year: 2019

In this work we introduce dual conditional cross-entropy filtering for noisy parallel data. For each sentence pair of the noisy parallel corpus we compute cross-entropy scores according to two inverse translation models trained on clean data. We penalize divergent cross-entropies and weigh the penalty by the cross-entropy average of both models. Sorting or thresholding according to these scores results in better subsets of parallel data. We achieve higher BLEU scores with models trained on parallel data filtered only from Paracrawl than with models trained on clean WMT data. We further evaluate our method in the context of the WMT2018 shared task on parallel corpus filtering and achieve the overall highest ranking scores of the shared task, scoring top in three out of four subtasks.
Car Monitoring System in Apartment Garages by Small Autonomous Car using Deep Learning
Venue: arXiv cs.CV 2019
Year: 2019

Currently, there is an increase in the number of Peruvian families living in apartments instead of houses for the lots of advantage; However, in some cases there are troubles such as robberies of goods that are usually left at the parking lots or the entrance of strangers that use the tenants parking lots (this last trouble sometimes is related to kidnappings or robberies in building apartments). Due to these problems, the use of a self-driving mini-car is proposed to implement a monitoring system of license plates in an underground garage inside a building using a deep learning model with the aim of recording the vehicles and identifying their owners if they were tenants or not. In addition, the small robot has its own location system using beacons that allow us to identify the position of the parking lot corresponding to each tenant of the building while the mini-car is on its way. Finally, one of the objectives of this work is to build a low-cost mini-robot that would replace expensive cameras or work together in order to keep safe the goods of tenants.
Machine learning for predicting thermal power consumption of the Mars Express Spacecraft
Venue: arXiv cs.LG 2019
Year: 2019

The thermal subsystem of the Mars Express (MEX) spacecraft keeps the on-board equipment within its pre-defined operating temperatures range. To plan and optimize the scientific operations of MEX, its operators need to estimate in advance, as accurately as possible, the power consumption of the thermal subsystem. The remaining power can then be allocated for scientific purposes. We present a machine learning pipeline for efficiently constructing accurate predictive models for predicting the power of the thermal subsystem on board MEX. In particular, we employ state-of-the-art feature engineering approaches for transforming raw telemetry data, in turn used for constructing accurate models with different state-of-the-art machine learning methods. We show that the proposed pipeline considerably improve our previous (competition-winning) work in terms of time efficiency and predictive performance. Moreover, while achieving superior predictive performance, the constructed models also provide important insight into the spacecraft's behavior, allowing for further analyses and optimal planning of MEX's operation.
Emergence of Communication in an Interactive World with Consistent Speakers
Venue: arXiv cs.CL 2019
Year: 2019

Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction. Prior work on the topic focused on simple environments, where training using policy gradient was feasible despite the non-stationarity of the agents during training. In this paper, we present a more challenging environment for testing the emergence of communication from raw pixels, where training using policy gradient fails. We propose a new model and training algorithm, that utilizes the structure of a learned representation space to produce more consistent speakers at the initial phases of training, which stabilizes learning. We empirically show that our algorithm substantially improves performance compared to policy gradient. We also propose a new alignment-based metric for measuring context-independence in emerged communication and find our method increases context-independence compared to policy gradient and other competitive baselines.
Adversarial Attack Type I: Cheat Classifiers by Significant Changes
Venue: arXiv cs.LG 2019
Year: 2019

Despite the great success of deep neural networks, the adversarial attack can cheat some well-trained classifiers by small permutations. In this paper, we propose another type of adversarial attack that can cheat classifiers by significant changes. For example, we can significantly change a face but well-trained neural networks still recognize the adversarial and the original example as the same person. Statistically, the existing adversarial attack increases Type II error and the proposed one aims at Type I error, which are hence named as Type II and Type I adversarial attack, respectively. The two types of attack are equally important but are essentially different, which are intuitively explained and numerically evaluated. To implement the proposed attack, a supervised variation autoencoder is designed and then the classifier is attacked by updating the latent variables using gradient information. {Besides, with pre-trained generative models, Type I attack on latent spaces is investigated as well.} Experimental results show that our method is practical and effective to generate Type I adversarial examples on large-scale image datasets. Most of these generated examples can pass detectors designed for defending Type II attack and the strengthening strategy is only efficient with a specific type attack, both implying that the underlying reasons for Type I and Type II attack are different.
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
Venue: arXiv cs.CL 2019
Year: 2019

We propose to decompose instruction execution to goal prediction and action generation. We design a model that maps raw visual observations to goals using LINGUNET, a language-conditioned image generation network, and then generates the actions required to complete them. Our model is trained from demonstration only without external resources. To evaluate our approach, we introduce two benchmarks for instruction following: LANI, a navigation task; and CHAI, where an agent executes household instructions. Our evaluation demonstrates the advantages of our model decomposition, and illustrates the challenges posed by our new benchmarks.