OpenRaiser/Intern-Atlas huggingface dataset:
Deep multi-class learning from label proportions
Venue: arXiv cs.LG 2019
Year: 2019

We propose a learning algorithm capable of learning from label proportions instead of direct data labels. In this scenario, our data are arranged into various bags of a certain size, and only the proportions of each label within a given bag are known. This is a common situation in cases where per-data labeling is lengthy, but a more general label is easily accessible. Several approaches have been proposed to learn in this setting with linear models in the multiclass setting, or with nonlinear models in the binary classification setting. Here we investigate the more general nonlinear multiclass setting, and compare two differentiable loss functions to train end-to-end deep neural networks from bags with label proportions. We illustrate the relevance of our methods on an image classification benchmark, and demonstrate the possibility to learn accurate image classifiers from bags of images.
A Study on Software Metrics and its Impact on Software Quality
Venue: arXiv cs.SE 2019
Year: 2019

Software metrics offer a quantitative basis for predicting the software development process. In this way, software quality can be improved very easily. Software quality should be achieved to satisfy the customer with decreasing the software cost and improve there liability of the software product. In this research, we have discussed how the software metrics affect the quality of the software and which stages of its development software metrics have applied. We discussed the different software metrics and how these metrics have an impact on software quality and reliability. These techniques have been used for improving the quality of software and increase the revenue.
Skew constacyclic codes over a non-chain ring $\mathbb{F}_{q}[u,v]/\langle f(u),g(v), uv-vu\rangle$
Venue: arXiv cs.IT 2019
Year: 2019

Let $f(u)$ and $g(v)$ be two polynomials of degree $k$ and $\ell$ respectively, not both linear, which split into distinct linear factors over $\mathbb{F}_{q}$. Let $\mathcal{R}=\mathbb{F}_{q}[u,v]/\langle f(u),g(v),\\uv-vu\rangle$ be a finite commutative non-chain ring. In this paper, we study $ψ$-skew cyclic and $θ_t$-skew constacyclic codes over the ring $\mathcal{R}$ where $ψ$ and $θ_t$ are two automorphisms defined on $\mathcal{R}$.
A Hippocampus Model for Online One-Shot Storage of Pattern Sequences
Venue: arXiv cs.NE 2019
Year: 2019

We present a computational model based on the CRISP theory (Content Representation, Intrinsic Sequences, and Pattern completion) of the hippocampus that allows to continuously store pattern sequences online in a one-shot fashion. Rather than storing a sequence in CA3, CA3 provides a pre-trained sequence that is hetero-associated with the input sequence, which allows the system to perform one-shot learning. Plasticity on a short time scale therefore only happens in the incoming and outgoing connections of CA3. Stored sequences can later be recalled from a single cue pattern. We identify the pattern separation performed by subregion DG to be necessary for storing sequences that contain correlated patterns. A design principle of the model is that we use a single learning rule named Hebbiand-escent to train all parts of the system. Hebbian-descent has an inherent forgetting mechanism that allows the system to continuously memorize new patterns while forgetting early stored ones. The model shows a plausible behavior when noisy and new patterns are presented and has a rather high capacity of about 40% in terms of the number of neurons in CA3. One notable property of our model is that it is capable of `boot-strapping' (improving) itself without external input in a process we refer to as `dreaming'. Besides artificially generated input sequences we also show that the model works with sequences of encoded handwritten digits or natural images. To our knowledge this is the first model of the hippocampus that allows to store correlated pattern sequences online in a one-shot fashion without a consolidation process, which can instantaneously be recalled later.
Interactive-predictive neural multimodal systems
Venue: arXiv cs.CV 2019
Year: 2019

Despite the advances achieved by neural models in sequence to sequence learning, exploited in a variety of tasks, they still make errors. In many use cases, these are corrected by a human expert in a posterior revision process. The interactive-predictive framework aims to minimize the human effort spent on this process by considering partial corrections for iteratively refining the hypothesis. In this work, we generalize the interactive-predictive approach, typically applied in to machine translation field, to tackle other multimodal problems namely, image and video captioning. We study the application of this framework to multimodal neural sequence to sequence models. We show that, following this framework, we approximately halve the effort spent for correcting the outputs generated by the automatic systems. Moreover, we deploy our systems in a publicly accessible demonstration, that allows to better understand the behavior of the interactive-predictive framework.
Visualization of AE's Training on Credit Card Transactions with Persistent Homology
Venue: arXiv cs.LG 2019
Year: 2019

Auto-encoders are among the most popular neural network architecture for dimension reduction. They are composed of two parts: the encoder which maps the model distribution to a latent manifold and the decoder which maps the latent manifold to a reconstructed distribution. However, auto-encoders are known to provoke chaotically scattered data distribution in the latent manifold resulting in an incomplete reconstructed distribution. Current distance measures fail to detect this problem because they are not able to acknowledge the shape of the data manifolds, i.e. their topological features, and the scale at which the manifolds should be analyzed. We propose Persistent Homology for Wasserstein Auto-Encoders, called PHom-WAE, a new methodology to assess and measure the data distribution of a generative model. PHom-WAE minimizes the Wasserstein distance between the true distribution and the reconstructed distribution and uses persistent homology, the study of the topological features of a space at different spatial resolutions, to compare the nature of the latent manifold and the reconstructed distribution. Our experiments underline the potential of persistent homology for Wasserstein Auto-Encoders in comparison to Variational Auto-Encoders, another type of generative model. The experiments are conducted on a real-world data set particularly challenging for traditional distance measures and auto-encoders. PHom-WAE is the first methodology to propose a topological distance measure, the bottleneck distance, for Wasserstein Auto-Encoders used to compare decoded samples of high quality in the context of credit card transactions.
The Bloom Clock
Venue: arXiv cs.DC 2019
Year: 2019

The bloom clock is a space-efficient, probabilistic data structure designed to determine the partial order of events in highly distributed systems. The bloom clock, like the vector clock, can autonomously detect causality violations by comparing its logical timestamps. Unlike the vector clock, the space complexity of the bloom clock does not depend on the number of nodes in a system. Instead it depends on a set of chosen parameters that determine its confidence interval, i.e. false positive rate. To reduce the space complexity from which the vector clock suffers, the bloom clock uses a 'moving window' in which the partial order of events can be inferred with high confidence. If two clocks are not comparable, the bloom clock can always deduce it, i.e. false negatives are not possible. If two clocks are comparable, the bloom clock can calculate the confidence of that statement, i.e. it can compute the false positive rate between comparable pairs of clocks. By choosing an acceptable threshold for the false positive rate, the bloom clock can properly compare the order of its timestamps, with that of other nodes in a highly accurate and space efficient way.
Somewhere Around That Number: An Interview Study of How Spreadsheet Users Manage Uncertainty
Venue: arXiv cs.HC 2019
Year: 2019

Spreadsheet users regularly deal with uncertainty in their data, for example due to errors and estimates. While an insight into data uncertainty can help in making better informed decisions, prior research suggests that people often use informal heuristics to reason with probabilities, which leads to incorrect conclusions. Moreover, people often ignore or simplify uncertainty. To understand how people currently encounter and deal with uncertainty in spreadsheets, we conducted an interview study with 11 spreadsheet users from a range of domains. We found that how people deal with uncertainty is influenced by the role the spreadsheet plays in people's work and the user's aims. Spreadsheets are used as a database, template, calculation tool, notepad and exploration tool. In doing so, participants' aims were to compute and compare different scenarios, understand something about the nature of the uncertainty in their situation, and translate the complexity of data uncertainty into simplified presentations to other people, usually decision-makers. Spreadsheets currently provide limited tools to support these aims, and participants had various workarounds.
TS-RNN: Text Steganalysis Based on Recurrent Neural Networks
Venue: arXiv cs.MM 2019
Year: 2019

With the rapid development of natural language processing technologies, more and more text steganographic methods based on automatic text generation technology have appeared in recent years. These models use the powerful self-learning and feature extraction ability of the neural networks to learn the feature expression of massive normal texts. Then they can automatically generate dense steganographic texts which conform to such statistical distribution based on the learned statistical patterns. In this paper, we observe that the conditional probability distribution of each word in the automatically generated steganographic texts will be distorted after embedded with hidden information. We use Recurrent Neural Networks (RNNs) to extract these feature distribution differences and then classify those features into cover text and stego text categories. Experimental results show that the proposed model can achieve high detection accuracy. Besides, the proposed model can even make use of the subtle differences of the feature distribution of texts to estimate the amount of hidden information embedded in the generated steganographic text.
On Post-Processing the Results of Quantum Optimizers
Venue: arXiv cs.ET 2019
Year: 2019

The use of quantum computing for applications involving optimization has been regarded as one of the areas it may prove to be advantageous (against classical computation). To further improve the quality of the solutions, post-processing techniques are often used on the results of quantum optimization. One such recent approach is the Multi Qubit Correction (MQC) algorithm by Dorband. In this paper, we will discuss and analyze the strengths and weaknesses of this technique. Then based on our discussion, we perform an experiment on how pairing heuristics on the input of MQC can affect the results of a quantum optimizer and a comparison between MQC and the built-in optimization method that D-wave Systems offers. Among our results, we are able to show that the built-in post-processing rarely beats MQC in our tests. We hope that by using the ideas and insights presented in this paper, researchers and developers will be able to make a more informed decision on what kind of post-processing methods to use for their quantum optimization needs.
Leadership in Congestion Games: Multiple User Classes and Non-Singleton Actions (Extended Version)
Venue: arXiv cs.GT 2019
Year: 2019

We study the problem of finding Stackelberg equilibria in games with a massive number of players. So far, the only known game instances in which the problem is solved in polynomial time are some particular congestion games. However, a complete characterization of hard and easy instances is still lacking. In this paper, we extend the state of the art along two main directions. First, we focus on games where players' actions are made of multiple resources, and we prove that the problem is NP-hard and not in Poly-APX unless P = NP, even in the basic case in which players are symmetric, their actions are made of only two resources, and the cost functions are monotonic. Second, we focus on games with singleton actions where the players are partitioned into classes, depending on which actions they have available. In this case, we provide a dynamic programming algorithm that finds an equilibrium in polynomial time, when the number of classes is fixed and the leader plays pure strategies. Moreover, we prove that, if we allow for leader's mixed strategies, then the problem becomes NP-hard even with only four classes and monotonic costs. Finally, for both settings, we provide mixed-integer linear programming formulations, and we experimentally evaluate their scalability on both random game instances and worst-case instances based on our hardness reductions.
Recommendation from Raw Data with Adaptive Compound Poisson Factorization
Venue: arXiv cs.IR 2019
Year: 2019

Count data are often used in recommender systems: they are widespread (song play counts, product purchases, clicks on web pages) and can reveal user preference without any explicit rating from the user. Such data are known to be sparse, over-dispersed and bursty, which makes their direct use in recommender systems challenging, often leading to pre-processing steps such as binarization. The aim of this paper is to build recommender systems from these raw data, by means of the recently proposed compound Poisson Factorization (cPF). The paper contributions are three-fold: we present a unified framework for discrete data (dcPF), leading to an adaptive and scalable algorithm; we show that our framework achieves a trade-off between Poisson Factorization (PF) applied to raw and binarized data; we study four specific instances that are relevant to recommendation and exhibit new links with combinatorics. Experiments with three different datasets show that dcPF is able to effectively adjust to over-dispersion, leading to better recommendation scores when compared with PF on either raw or binarized data.
Defining Admissible Rewards for High Confidence Policy Evaluation
Venue: arXiv cs.LG 2019
Year: 2019

A key impediment to reinforcement learning (RL) in real applications with limited, batch data is defining a reward function that reflects what we implicitly know about reasonable behaviour for a task and allows for robust off-policy evaluation. In this work, we develop a method to identify an admissible set of reward functions for policies that (a) do not diverge too far from past behaviour, and (b) can be evaluated with high confidence, given only a collection of past trajectories. Together, these ensure that we propose policies that we trust to be implemented in high-risk settings. We demonstrate our approach to reward design on synthetic domains as well as in a critical care context, for a reward that consolidates clinical objectives to learn a policy for weaning patients from mechanical ventilation.
Persistent homology detects curvature
Venue: arXiv cs.CG 2019
Year: 2019

In topological data analysis, persistent homology is used to study the "shape of data". Persistent homology computations are completely characterized by a set of intervals called a bar code. It is often said that the long intervals represent the "topological signal" and the short intervals represent "noise". We give evidence to dispute this thesis, showing that the short intervals encode geometric information. Specifically, we prove that persistent homology detects the curvature of disks from which points have been sampled. We describe a general computational framework for solving inverse problems using the average persistence landscape, a continuous mapping from metric spaces with a probability measure to a Hilbert space. In the present application, the average persistence landscapes of points sampled from disks of constant curvature results in a path in this Hilbert space which may be learned using standard tools from statistical and machine learning.
Exploiting Uncertainty of Loss Landscape for Stochastic Optimization
Venue: arXiv cs.LG 2019
Year: 2019

We introduce novel variants of momentum by incorporating the variance of the stochastic loss function. The variance characterizes the confidence or uncertainty of the local features of the averaged loss surface across the i.i.d. subsets of the training data defined by the mini-batches. We show two applications of the gradient of the variance of the loss function. First, as a bias to the conventional momentum update to encourage conformity of the local features of the loss function (e.g. local minima) across mini-batches to improve generalization and the cumulative training progress made per epoch. Second, as an alternative direction for "exploration" in the parameter space, especially, for non-convex objectives, that exploits both the optimistic and pessimistic views of the loss function in the face of uncertainty. We also introduce a novel data-driven stochastic regularization technique through the parameter update rule that is model-agnostic and compatible with arbitrary architectures. We further establish connections to probability distributions over loss functions and the REINFORCE policy gradient update with baseline in RL. Finally, we incorporate the new variants of momentum proposed into Adam, and empirically show that our methods improve the rate of convergence of training based on our experiments on the MNIST and CIFAR-10 datasets.
Cospectral Bipartite Graphs with the Same Degree Sequences but with Different Number of Large Cycles
Venue: arXiv cs.DM 2019
Year: 2019

Finding the multiplicity of cycles in bipartite graphs is a fundamental problem of interest in many fields including the analysis and design of low-density parity-check (LDPC) codes. Recently, Blake and Lin computed the number of shortest cycles ($g$-cycles, where $g$ is the girth of the graph) in a bi-regular bipartite graph, in terms of the degree sequences and the spectrum (eigenvalues of the adjacency matrix) of the graph [{\em IEEE Trans. Inform. Theory 64(10):6526--6535, 2018}]. This result was subsequently extended in [{\em IEEE Trans. Inform. Theory, accepted for publication, Dec. 2018}] to cycles of length $g+2, \ldots, 2g-2$, in bi-regular bipartite graphs, as well as $4$-cycles and $6$-cycles in irregular and half-regular bipartite graphs, with $g \geq 4$ and $g \geq 6$, respectively. In this paper, we complement these positive results with negative results demonstrating that the information of the degree sequences and the spectrum of a bipartite graph is, in general, insufficient to count (a) the $i$-cycles, $i \geq 2g$, in bi-regular graphs, (b) the $i$-cycles for any $i > g$, regardless of the value of $g$, and $g$-cycles for $g \geq 6$, in irregular graphs, and (c) the $i$-cycles for any $i > g$, regardless of the value of $g$, and $g$-cycles for $g \geq 8$, in half-regular graphs. To obtain these results, we construct counter-examples using the Godsil-McKay switching.
Using Metrics Suites to Improve the Measurement of Privacy in Graphs
Venue: arXiv cs.CR 2019
Year: 2019

Social graphs are widely used in research (e.g., epidemiology) and business (e.g., recommender systems). However, sharing these graphs poses privacy risks because they contain sensitive information about individuals. Graph anonymization techniques aim to protect individual users in a graph, while graph de-anonymization aims to re-identify users. The effectiveness of anonymization and de-anonymization algorithms is usually evaluated with privacy metrics. However, it is unclear how strong existing privacy metrics are when they are used in graph privacy. In this paper, we study 26 privacy metrics for graph anonymization and de-anonymization and evaluate their strength in terms of three criteria: monotonicity indicates whether the metric indicates lower privacy for stronger adversaries; for within-scenario comparisons, evenness indicates whether metric values are spread evenly; and for between-scenario comparisons, shared value range indicates whether metrics use a consistent value range across scenarios. Our extensive experiments indicate that no single metric fulfills all three criteria perfectly. We therefore use methods from multi-criteria decision analysis to aggregate multiple metrics in a metrics suite, and we show that these metrics suites improve monotonicity compared to the best individual metric. This important result enables more monotonic, and thus more accurate, evaluations of new graph anonymization and de-anonymization algorithms.
New Algorithms for Functional Distributed Constraint Optimization Problems
Venue: arXiv cs.MA 2019
Year: 2019

The Distributed Constraint Optimization Problem (DCOP) formulation is a powerful tool to model multi-agent coordination problems that are distributed by nature. The formulation is suitable for problems where variables are discrete and constraint utilities are represented in tabular form. However, many real-world applications have variables that are continuous and tabular forms thus cannot accurately represent constraint utilities. To overcome this limitation, researchers have proposed the Functional DCOP (F-DCOP) model, which are DCOPs with continuous variables. But existing approaches usually come with some restrictions on the form of constraint utilities and are without quality guarantees. Therefore, in this paper, we (i) propose exact algorithms to solve a specific subclass of F-DCOPs; (ii) propose approximation methods with quality guarantees to solve general F-DCOPs; and (iii) empirically show that our algorithms outperform existing state-of-the-art F-DCOP algorithms on randomly generated instances when given the same communication limitations.
Identifying Classes Susceptible to Adversarial Attacks
Venue: arXiv cs.LG 2019
Year: 2019

Despite numerous attempts to defend deep learning based image classifiers, they remain susceptible to the adversarial attacks. This paper proposes a technique to identify susceptible classes, those classes that are more easily subverted. To identify the susceptible classes we use distance-based measures and apply them on a trained model. Based on the distance among original classes, we create mapping among original classes and adversarial classes that helps to reduce the randomness of a model to a significant amount in an adversarial setting. We analyze the high dimensional geometry among the feature classes and identify the k most susceptible target classes in an adversarial attack. We conduct experiments using MNIST, Fashion MNIST, CIFAR-10 (ImageNet and ResNet-32) datasets. Finally, we evaluate our techniques in order to determine which distance-based measure works best and how the randomness of a model changes with perturbation.
Optimal Timing of Moving Target Defense: A Stackelberg Game Model
Venue: arXiv cs.GT 2019
Year: 2019

As an effective approach to thwarting advanced attacks, moving target defense (MTD) has been applied to various domains. Previous works on MTD, however, mainly focus on deciding the sequence of system configurations to be used and have largely ignored the equally important timing problem. Given that both the migration cost and attack time vary over system configurations, it is crucial to jointly optimize the spatial and temporal decisions in MTD to better protect the system from persistent threats. In this work, we propose a Stackelberg game model for MTD where the defender commits to a joint migration and timing strategy to cope with configuration-dependent migration cost and attack time distribution. The defender's problem is formulated as a semi-Markovian decision process and a nearly optimal MTD strategy is derived by exploiting the unique structure of the game.