OpenRaiser/Intern-Atlas huggingface dataset:
Performance optimization and modeling of fine-grained irregular communication in UPC
Venue: arXiv cs.DC 2019
Year: 2019

The UPC programming language offers parallelism via logically partitioned shared memory, which typically spans physically disjoint memory sub-systems. One convenient feature of UPC is its ability to automatically execute between-thread data movement, such that the entire content of a shared data array appears to be freely accessible by all the threads. The programmer friendliness, however, can come at the cost of substantial performance penalties. This is especially true when indirectly indexing the elements of a shared array, for which the induced between-thread data communication can be irregular and have a fine-grained pattern. In this paper we study performance enhancement strategies specifically targeting such fine-grained irregular communication in UPC. Starting from explicit thread privatization, continuing with block-wise communication, and arriving at message condensing and consolidation, we obtained considerable performance improvement of UPC programs that originally require fine-grained irregular communication. Besides the performance enhancement strategies, the main contribution of the present paper is to propose performance models for the different scenarios, in form of quantifiable formulas that hinge on the actual volumes of various data movements plus a small number of easily obtainable hardware characteristic parameters. These performance models help to verify the enhancements obtained, while also providing insightful predictions of similar parallel implementations, not limited to UPC, that also involve between-thread or between-process irregular communication. As a further validation, we also apply our performance modeling methodology and hardware characteristic parameters to an existing UPC code for solving a 2D heat equation on a uniform mesh.
Augmented Replay Memory in Reinforcement Learning With Continuous Control
Venue: arXiv cs.LG 2019
Year: 2019

Online reinforcement learning agents are currently able to process an increasing amount of data by converting it into a higher order value functions. This expansion of the information collected from the environment increases the agent's state space enabling it to scale up to a more complex problems but also increases the risk of forgetting by learning on redundant or conflicting data. To improve the approximation of a large amount of data, a random mini-batch of the past experiences that are stored in the replay memory buffer is often replayed at each learning step. The proposed work takes inspiration from a biological mechanism which act as a protective layer of human brain higher cognitive functions: active memory consolidation mitigates the effect of forgetting of previous memories by dynamically processing the new ones. The similar dynamics are implemented by a proposed augmented memory replay AMR capable of optimizing the replay of the experiences from the agent's memory structure by altering or augmenting their relevance. Experimental results show that an evolved AMR augmentation function capable of increasing the significance of the specific memories is able to further increase the stability and convergence speed of the learning algorithms dealing with the complexity of continuous action domains.
Worst-Case Optimal Radix Triejoin
Venue: arXiv cs.DB 2019
Year: 2019

Relatively recently, the field of join processing has been swayed by the discovery of a new class of multi-way join algorithms. The new algorithms join multiple relations simultaneously rather than perform a series of pairwise joins. The new join algorithms satisfy stronger worst-case runtime complexity guarantees than any of the existing approaches based on pairwise joins -- they are worst-case optimal in data complexity. These research efforts have resulted in a flurry of papers documenting theoretical and some practical contributions. However, there is still the quest of making the new worst-case optimal join algorithms truly practical in terms of (1) ease of implementation and (2) secondary index efficiency in terms of number of indexes created to answer a query. In this paper, we present a simple worst-case optimal multi-way join algorithm called the radix triejoin. Radix triejoin uses a binary encoding for reducing the domain of a database. Our main technical contribution is that domain reduction allows a bit-interleaving of attribute values that gives rise to a query-independent relation representation, permitting the computation of multiple queries over the same relations worst-case optimally without having to construct additional secondary indexes. We also generalise the core algorithm to conjunctive queries with inequality constraints and provide a new proof technique for the worst-case optimal join result.
Scalable Influence Estimation Without Sampling
Venue: arXiv cs.SI 2019
Year: 2019

In a diffusion process on a network, how many nodes are expected to be influenced by a set of initial spreaders? This natural problem, often referred to as influence estimation, boils down to computing the marginal probability that a given node is active at a given time when the process starts from specified initial condition. Among many other applications, this task is crucial for a well-studied problem of influence maximization: finding optimal spreaders in a social network that maximize the influence spread by a certain time horizon. Indeed, influence estimation needs to be called multiple times for comparing candidate seed sets. Unfortunately, in many models of interest an exact computation of marginals is #P-hard. In practice, influence is often estimated using Monte-Carlo sampling methods that require a large number of runs for obtaining a high-fidelity prediction, especially at large times. It is thus desirable to develop analytic techniques as an alternative to sampling methods. Here, we suggest an algorithm for estimating the influence function in popular independent cascade model based on a scalable dynamic message-passing approach. This method has a computational complexity of a single Monte-Carlo simulation and provides an upper bound on the expected spread on a general graph, yielding exact answer for treelike networks. We also provide dynamic message-passing equations for a stochastic version of the linear threshold model. The resulting saving of a potentially large sampling factor in the running time compared to simulation-based techniques hence makes it possible to address large-scale problem instances.
PMC-GANs: Generating Multi-Scale High-Quality Pedestrian with Multimodal Cascaded GANs
Venue: arXiv cs.CV 2019
Year: 2019

Recently, generative adversarial networks (GANs) have shown great advantages in synthesizing images, leading to a boost of explorations of using faked images to augment data. This paper proposes a multimodal cascaded generative adversarial networks (PMC-GANs) to generate realistic and diversified pedestrian images and augment pedestrian detection data. The generator of our model applies a residual U-net structure, with multi-scale residual blocks to encode features, and attention residual blocks to help decode and rebuild pedestrian images. The model constructs in a coarse-to-fine fashion and adopts cascade structure, which is beneficial to produce high-resolution pedestrians. PMC-GANs outperforms baselines, and when used for data augmentation, it improves pedestrian detection results.
Disentangled Representation Learning with Wasserstein Total Correlation
Venue: arXiv cs.LG 2019
Year: 2019

Unsupervised learning of disentangled representations involves uncovering of different factors of variations that contribute to the data generation process. Total correlation penalization has been a key component in recent methods towards disentanglement. However, Kullback-Leibler (KL) divergence-based total correlation is metric-agnostic and sensitive to data samples. In this paper, we introduce Wasserstein total correlation in both variational autoencoder and Wasserstein autoencoder settings to learn disentangled latent representations. A critic is adversarially trained along with the main objective to estimate the Wasserstein total correlation term. We discuss the benefits of using Wasserstein distance over KL divergence to measure independence and conduct quantitative and qualitative experiments on several data sets. Moreover, we introduce a new metric to measure disentanglement. We show that the proposed approach has comparable performances on disentanglement with smaller sacrifices in reconstruction abilities.
Robust Transceiver Design for Full-Duplex Decode-and-Forward Relay-Assisted MIMO Systems
Venue: arXiv cs.IT 2019
Year: 2019

Robust transceiver design against unresolvable system uncertainties is of crucial importance for reliable communication. For instance, full-duplex communication suffers from such uncertainties when canceling the self-interference, since some residual self-interference (RSI) remains uncanceled due to imperfect channel knowledge. We consider a MIMO multi-hop system, where the source, the relay and the destination are equipped with multiple antennas. The considered decode-and-forward (DF) hybrid relay can operate in either half-duplex or full-duplex mode, and the mode changes adaptively depending on the RSI strength. We investigate a robust transceiver design problem, which maximizes the throughput rate of the worstcase RSI under the self-interference channel uncertainty bound constraint. The yielded problem turns out to be a non-convex optimization problem, where the non-convex objective is optimized over the cone of semidefinite matrices. Without loss of generality, we simplify the problem to the optimization over multiple scalar parameters using majorization theory. Furthermore, we propose an efficient algorithm to obtain a local optimal solution iteratively. Eventually, we obtain insights on the optimal antenna allocation at the relay input-frontend and output-frontend, for relay reception and transmission, respectively. Interestingly, given a number of antennas at the relay, the robustness improves if more antennas are allocated to reception than to transmission.
A Game-Theoretic Algorithm for Link Prediction
Venue: arXiv cs.SI 2019
Year: 2019

Predicting edges in networks is a key problem in social network analysis and involves reasoning about the relationships between nodes based on the structural properties of a network. In particular, link prediction can be used to analyse how a network will develop or - given incomplete information about relationships - to discover "missing" links. Our approach to this problem is rooted in cooperative game theory, where we propose a new, quasi-local approach (i.e., one which considers nodes within some radius k) that combines generalised group closeness centrality and semivalue interaction indices. We develop fast algorithms for computing our measure and evaluate it on a number of real-world networks, where it outperforms a selection of other state-of-the-art methods from the literature. Importantly, choosing the optimal radius k for quasi-local methods is difficult, and there is no assurance that the choice is optimal. Additionally, when compared to other quasi-local methods, ours achieves very good results even when given a suboptimal radius k as a parameter.
Wisdom of collaborators: a peer-review approach to performance appraisal
Venue: arXiv cs.SI 2019
Year: 2019

Individual performance and reputation within a company are major factors that influence wage distribution, promotion and firing. Due to the complexity and collaborative nature of contemporary business processes, the evaluation of individual impact in the majority of organizations is an ambiguous and non-trivial task. Existing performance appraisal approaches are often affected by individuals biased judgements, and organizations are dissatisfied with the results of evaluations. We assert that employees can provide accurate measurement of their peer performance in a complex collaborative environment. We propose a novel metric, the Peer Rank Score (PRS), that evaluates individual reputations and the non-quantifiable individual impact. PRS is based on pairwise comparisons of employees. We show high robustness of the algorithm on simulations and empirically validate it for a genetic testing company on more than one thousand employees using peer reviews over the course of three years.
Intuitionistic Linear Temporal Logics
Venue: arXiv cs.LO 2019
Year: 2019

We consider intuitionistic variants of linear temporal logic with `next', `until' and `release' based on expanding posets: partial orders equipped with an order-preserving transition function. This class of structures gives rise to a logic which we denote $\iltl$, and by imposing additional constraints we obtain the logics $\itlb$ of persistent posets and $\itlht$ of here-and-there temporal logic, both of which have been considered in the literature. We prove that $\iltl$ has the effective finite model property and hence is decidable, while $\itlb$ does not have the finite model property. We also introduce notions of bounded bisimulations for these logics and use them to show that the `until' and `release' operators are not definable in terms of each other, even over the class of persistent posets.
A New Approach for Explainable Multiple Organ Annotation with Few Data
Venue: arXiv cs.CV 2019
Year: 2019

Despite the recent successes of deep learning, such models are still far from some human abilities like learning from few examples, reasoning and explaining decisions. In this paper, we focus on organ annotation in medical images and we introduce a reasoning framework that is based on learning fuzzy relations on a small dataset for generating explanations. Given a catalogue of relations, it efficiently induces the most relevant relations and combines them for building constraints in order to both solve the organ annotation task and generate explanations. We test our approach on a publicly available dataset of medical images where several organs are already segmented. A demonstration of our model is proposed with an example of explained annotations. It was trained on a small training set containing as few as a couple of examples.
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
Venue: arXiv cs.DC 2019
Year: 2019

Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embedding operations with unique irregular memory access patterns that pose a fundamental challenge to accelerate. This paper proposes a lightweight, commodity DRAM compliant, near-memory processing solution to accelerate personalized recommendation inference. The in-depth characterization of production-grade recommendation models shows that embedding operations with high model-, operator- and data-level parallelism lead to memory bandwidth saturation, limiting recommendation inference performance. We propose RecNMP which provides a scalable solution to improve system throughput, supporting a broad range of sparse embedding models. RecNMP is specifically tailored to production environments with heavy co-location of operators on a single server. Several hardware/software co-optimization techniques such as memory-side caching, table-aware packet scheduling, and hot entry profiling are studied, resulting in up to 9.8x memory latency speedup over a highly-optimized baseline. Overall, RecNMP offers 4.2x throughput improvement and 45.8% memory energy savings.
The Challenge of Unifying Semantic and Syntactic Inference Restrictions
Venue: arXiv cs.LO 2019
Year: 2019

While syntactic inference restrictions don't play an important role for SAT, they are an essential reasoning technique for more expressive logics, such as first-order logic, or fragments thereof. In particular, they can result in short proofs or model representations. On the other hand, semantically guided inference systems enjoy important properties, such as the generation of solely non-redundant clauses. I discuss to what extend the two paradigms may be unifiable.
A New Burrows Wheeler Transform Markov Distance
Venue: arXiv cs.CR 2019
Year: 2019

Prior work inspired by compression algorithms has described how the Burrows Wheeler Transform can be used to create a distance measure for bioinformatics problems. We describe issues with this approach that were not widely known, and introduce our new Burrows Wheeler Markov Distance (BWMD) as an alternative. The BWMD avoids the shortcomings of earlier efforts, and allows us to tackle problems in variable length DNA sequence clustering. BWMD is also more adaptable to other domains, which we demonstrate on malware classification tasks. Unlike other compression-based distance metrics known to us, BWMD works by embedding sequences into a fixed-length feature vector. This allows us to provide significantly improved clustering performance on larger malware corpora, a weakness of prior methods.
Leveraging Semi-Supervised Learning for Fairness using Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

There has been a growing concern about the fairness of decision-making systems based on machine learning. The shortage of labeled data has been always a challenging problem facing machine learning based systems. In such scenarios, semi-supervised learning has shown to be an effective way of exploiting unlabeled data to improve upon the performance of model. Notably, unlabeled data do not contain label information which itself can be a significant source of bias in training machine learning systems. This inspired us to tackle the challenge of fairness by formulating the problem in a semi-supervised framework. In this paper, we propose a semi-supervised algorithm using neural networks benefiting from unlabeled data to not just improve the performance but also improve the fairness of the decision-making process. The proposed model, called SSFair, exploits the information in the unlabeled data to mitigate the bias in the training data.
Automated Testing for Deep Learning Systems with Differential Behavior Criteria
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we conducted a study on building an automated testing system for deep learning systems based on differential behavior criteria. The automated testing goals were achieved by jointly optimizing two objective functions: maximizing differential behaviors from models under testing and maximizing neuron coverage. By observing differential behaviors from three pre-trained models during each testing iteration, the input image that triggered erroneous feedback was registered as a corner-case. The generated corner-cases can be used to examine the robustness of DNNs and consequently improve model accuracy. A project called DeepXplore was also used as a baseline model. After we fully implemented and optimized the baseline system, we explored its application as an augmenting training dataset with newly generated corner cases. With the GTRSB dataset, by retraining the model based on automated generated corner cases, the accuracy of three generic models increased by 259.2%, 53.6%, and 58.3%, respectively. Further, to extend the capability of automated testing, we explored other approaches based on differential behavior criteria to generate photo-realistic images for deep learning systems. One approach was to apply various transformations to the seed images for the deep learning framework. The other approach was to utilize the Generative Adversarial Networks (GAN) technique, which was implemented on MNIST and Driving datasets. The style transferring capability has been observed very effective in adding additional visual effects, replacing image elements, and style-shifting (virtual image to real images). The GAN-based testing sample generation system was shown to be the next frontier for automated testing for deep learning systems.
Graph Realizations: Maximum and Minimum Degree in Vertex Neighborhoods
Venue: arXiv cs.DS 2019
Year: 2019

The classical problem of degree sequence realizability asks whether or not a given sequence of $n$ positive integers is equal to the degree sequence of some $n$-vertex undirected simple graph. While the realizability problem of degree sequences has been well studied for different classes of graphs, there has been relatively little work concerning the realizability of other types of information profiles, such as the vertex neighborhood profiles. In this paper, we initiate the study of neighborhood degree profiles. We focus on the natural problem of realizing maximum and minimum neighborhood degrees. More specifically, we ask the following question: Given a sequence $D$ of $n$ non-negative integers $0\leq d_1\leq \cdots \leq d_n$, does there exist a simple graph with vertices $v_1,\ldots, v_n$ such that for every $1\le i \le n$, the maximum (resp. minimum) degree in the neighborhood of $v_i$ is exactly $d_i$? We provide in this work various results for both maximum as well as minimum neighborhood degree for general $n$ vertex graphs. Our results are first of its kind that studies extremal neighborhood degree profiles. For maximum neighborhood degree profiles, we provide a {\em complete realizability criteria}. In comparison, we observe that the minimum neighborhood profiles are not so well-behaved, for these our necessary and sufficient conditions for realizability {\em differ by a factor of at most two}.
Towards Neural-Guided Program Synthesis for Linear Temporal Logic Specifications
Venue: arXiv cs.AI 2019
Year: 2019

Synthesizing a program that realizes a logical specification is a classical problem in computer science. We examine a particular type of program synthesis, where the objective is to synthesize a strategy that reacts to a potentially adversarial environment while ensuring that all executions satisfy a Linear Temporal Logic (LTL) specification. Unfortunately, exact methods to solve so-called LTL synthesis via logical inference do not scale. In this work, we cast LTL synthesis as an optimization problem. We employ a neural network to learn a Q-function that is then used to guide search, and to construct programs that are subsequently verified for correctness. Our method is unique in combining search with deep learning to realize LTL synthesis. In our experiments the learned Q-function provides effective guidance for synthesis problems with relatively small specifications.