OpenRaiser/Intern-Atlas huggingface dataset:
Overlapping community detection in networks based on link partitioning and partitioning around medoids
Venue: arXiv cs.SI 2021
Year: 2021

In this paper, we present a new method for detecting overlapping communities in networks with a predefined number of clusters called LPAM (Link Partitioning Around Medoids). The overlapping communities in the graph are obtained by detecting the disjoint communities in the associated line graph employing link partitioning and partitioning around medoids which are done through the use of a distance function defined on the set of nodes. We consider both the commute distance and amplified commute distance as distance functions. The performance of the LPAM method is evaluated with computational experiments on real life instances, as well as synthetic network benchmarks. For small and medium-size networks, the exact solution was found, while for large networks we found solutions with a heuristic version of the LPAM method.
Performance-Complexity Tradeoffs in Greedy Weak Submodular Maximization with Random Sampling
Venue: arXiv cs.DM 2021
Year: 2021

Many problems in signal processing and machine learning can be formalized as weak submodular optimization tasks. For such problems, a simple greedy algorithm (\textsc{Greedy}) is guaranteed to find a solution achieving the objective with a value no worse than $1-e^{-1/c}$ of the optimal, where $c$ is the multiplicative weak-submodularity constant. Due to the high cost of querying large-scale systems, the complexity of \textsc{Greedy} becomes prohibitive in contemporary applications. In this work, we study the tradeoff between performance and complexity when one resorts to random sampling strategies to reduce the query complexity of \textsc{Greedy}. Specifically, we quantify the effect of uniform sampling strategies on \textsc{Greedy}'s performance through two metrics: (i) probability of identifying an optimal subset, and (ii) suboptimality with respect to the optimal solution. The latter implies that uniform sampling strategies with a fixed sampling size achieve a non-trivial approximation factor; however, we show that with overwhelming probability, these methods fail to find the optimal subset. Our analysis shows that the failure of uniform sampling strategies with fixed sample size can be circumvented by successively increasing the size of the search space. Building upon this insight, we propose a simple progressive stochastic greedy algorithm and study its approximation guarantees. Moreover, we demonstrate effectiveness of the proposed method in dimensionality reduction applications and feature selection tasks for clustering and object tracking.
Translating between the representations of a ranked convex geometry
Venue: arXiv cs.DM 2021
Year: 2021

It is well known that every closure system can be represented by an implicational base, or by the set of its meet-irreducible elements. In Horn logic, these are respectively known as the Horn expressions and the characteristic models. In this paper, we consider the problem of translating between the two representations in acyclic convex geometries. Quite surprisingly, we show that the problem in this context is already harder than the dualization in distributive lattices, a generalization of the well-known hypergraph dualization problem for which the existence of an output quasi-polynomial time algorithm is open. In light of this result, we consider a proper subclass of acyclic convex geometries, namely ranked convex geometries, as those that admit a ranked implicational base analogous to that of ranked posets. For this class, we provide output quasi-polynomial time algorithms based on hypergraph dualization for translating between the two representations. This improves the understanding of a long-standing open problem.
Model-Driven Deep Learning for MIMO Detection
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we investigate the model-driven deep learning (DL) for MIMO detection. In particular, the MIMO detector is specially designed by unfolding an iterative algorithm and adding some trainable parameters. Since the number of trainable parameters is much fewer than the data-driven DL based signal detector, the model-driven DL based MIMO detector can be rapidly trained with a much smaller data set. The proposed MIMO detector can be extended to soft-input soft-output detection easily. Furthermore, we investigate joint MIMO channel estimation and signal detection (JCESD), where the detector takes channel estimation error and channel statistics into consideration while channel estimation is refined by detected data and considers the detection error. Based on numerical results, the model-driven DL based MIMO detector significantly improves the performance of corresponding traditional iterative detector, outperforms other DL-based MIMO detectors and exhibits superior robustness to various mismatches.
From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation
Venue: arXiv cs.CV 2021
Year: 2021

Estimating accurate depth from a single image is challenging because it is an ill-posed problem as infinitely many 3D scenes can be projected to the same 2D scene. However, recent works based on deep convolutional neural networks show great progress with plausible results. The convolutional neural networks are generally composed of two parts: an encoder for dense feature extraction and a decoder for predicting the desired depth. In the encoder-decoder schemes, repeated strided convolution and spatial pooling layers lower the spatial resolution of transitional outputs, and several techniques such as skip connections or multi-layer deconvolutional networks are adopted to recover the original resolution for effective dense prediction. In this paper, for more effective guidance of densely encoded features to the desired depth prediction, we propose a network architecture that utilizes novel local planar guidance layers located at multiple stages in the decoding phase. We show that the proposed method outperforms the state-of-the-art works with significant margin evaluating on challenging benchmarks. We also provide results from an ablation study to validate the effectiveness of the proposed method.
LayoutVAE: Stochastic Scene Layout Generation From a Label Set
Venue: arXiv cs.CV 2021
Year: 2021

Recently there is an increasing interest in scene generation within the research community. However, models used for generating scene layouts from textual description largely ignore plausible visual variations within the structure dictated by the text. We propose LayoutVAE, a variational autoencoder based framework for generating stochastic scene layouts. LayoutVAE is a versatile modeling framework that allows for generating full image layouts given a label set, or per label layouts for an existing image given a new label. In addition, it is also capable of detecting unusual layouts, potentially providing a way to evaluate layout generation problem. Extensive experiments on MNIST-Layouts and challenging COCO 2017 Panoptic dataset verifies the effectiveness of our proposed framework.
Hybrid Boolean Networks as Physically Unclonable Functions
Venue: arXiv cs.CR 2021
Year: 2021

We introduce a Physically Unclonable Function (PUF) based on an ultra-fast chaotic network known as a Hybrid Boolean Network (HBN) implemented on a field programmable gate array. The network, consisting of $N$ coupled asynchronous logic gates displaying dynamics on the sub-nanosecond time scale, acts as a `digital fingerprint' by amplifying small manufacturing variations during a period of transient chaos. In contrast to other PUF designs, we use both $N$-bits per challenge and obtain $N$-bits per response by considering challenges to be initial states of the $N$-node network and responses to be states captured during the subsequent chaotic transient. We find that the presence of chaos amplifies the frozen-in randomness due to manufacturing differences and that the extractable entropy is approximately $50\%$ of the maximum of $N2^{N}$ bits. We obtain PUF uniqueness and reliability metrics $μ_{inter}$ = 0.40$\pm$0.01 and $μ_{intra}$ = 0.05$\pm$0.00, respectively, for an $N=256$ network. These metrics correspond to an expected Hamming distance of 102.4 bits per response. Moreover, a simple cherry-picking scheme that discards noisy bits yields $μ_{intra} < 0.01$ while still retaining $\sim200$ bits/response (corresponding to a Hamming distance of $\sim80$ bits/response). In addition to characterizing the uniqueness and reliability, we demonstrate super-exponential scaling in the entropy up to $N=512$ and demonstrate that PUFmeter, a recent PUF analysis tool, is unable to model our PUF. Finally, we characterize the temperature variation of the HBN-PUF and propose future improvements.
How Good is SGD with Random Shuffling?
Venue: arXiv cs.LG 2021
Year: 2021

We study the performance of stochastic gradient descent (SGD) on smooth and strongly-convex finite-sum optimization problems. In contrast to the majority of existing theoretical works, which assume that individual functions are sampled with replacement, we focus here on popular but poorly-understood heuristics, which involve going over random permutations of the individual functions. This setting has been investigated in several recent works, but the optimal error rates remain unclear. In this paper, we provide lower bounds on the expected optimization error with these heuristics (using SGD with any constant step size), which elucidate their advantages and disadvantages. In particular, we prove that after $k$ passes over $n$ individual functions, if the functions are re-shuffled after every pass, the best possible optimization error for SGD is at least $Ω\left(1/(nk)^2+1/nk^3\right)$, which partially corresponds to recently derived upper bounds. Moreover, if the functions are only shuffled once, then the lower bound increases to $Ω(1/nk^2)$. Since there are strictly smaller upper bounds for repeated reshuffling, this proves an inherent performance gap between SGD with single shuffling and repeated shuffling. As a more minor contribution, we also provide a non-asymptotic $Ω(1/k^2)$ lower bound (independent of $n$) for the incremental gradient method, when no random shuffling takes place. Finally, we provide an indication that our lower bounds are tight, by proving matching upper bounds for univariate quadratic functions.
Deduction Theorem: The Problematic Nature of Common Practice in Game Theory
Venue: arXiv cs.AI 2021
Year: 2021

We consider the Deduction Theorem used in the literature of game theory to run a purported proof by contradiction. In the context of game theory, it is stated that if we have a proof of $ϕ\vdash φ$, then we also have a proof of $ϕ\Rightarrow φ$. Hence, the proof of $ϕ\Rightarrow φ$ is deduced from a previously known statement. However, we argue that one has to manage to establish that a proof exists for the clauses $ϕ$ and $φ$, i.e., they are known true statements in order to show that $ϕ\vdash φ$ is provable, and that therefore $ϕ\Rightarrow φ$ is provable as well. Thus, we are not allowed to assume that the clause $ϕ$ or $φ$ is a true statement. This leads immediately to a wrong conclusion. Apart from this, we stress to other facts why the Deduction Theorem is not applicable to run a proof by contradiction. Finally, we present an example from industrial cooperation where the Deduction Theorem is not correctly applied with the consequence that the obtained result contradicts the well-known aggregation issue.
Computing the inverse geodesic length in planar graphs and graphs of bounded treewidth
Venue: arXiv cs.DS 2021
Year: 2021

The inverse geodesic length of a graph $G$ is the sum of the inverse of the distances between all pairs of distinct vertices of $G$. In some domains it is known as the Harary index or the global efficiency of the graph. We show that, if $G$ is planar and has $n$ vertices, then the inverse geodesic length of $G$ can be computed in roughly $O(n^{9/5})$ time. We also show that, if $G$ has $n$ vertices and treewidth at most $k$, then the inverse geodesic length of $G$ can be computed in $O(n \log^{O(k)}n)$ time. In both cases we use techniques developed for computing the sum of the distances, which does not have "inverse" component, together with batched evaluations of rational functions.
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
Venue: arXiv cs.CV 2021
Year: 2021

Alternatively inferring on the visual facts and commonsense is fundamental for an advanced VQA system. This ability requires models to go beyond the literal understanding of commonsense. The system should not just treat objects as the entrance to query background knowledge, but fully ground commonsense to the visual world and imagine the possible relationships between objects, e.g., "fork, can lift, food". To comprehensively evaluate such abilities, we propose a VQA benchmark, CRIC, which introduces new types of questions about Compositional Reasoning on vIsion and Commonsense, and an evaluation metric integrating the correctness of answering and commonsense grounding. To collect such questions and rich additional annotations to support the metric, we also propose an automatic algorithm to generate question samples from the scene graph associated with the images and the relevant knowledge graph. We further analyze several representative types of VQA models on the CRIC dataset. Experimental results show that grounding the commonsense to the image region and joint reasoning on vision and commonsense are still challenging for current approaches. The dataset is available at https://cricvqa.github.io.
Constrained domain adaptation for Image segmentation
Venue: arXiv cs.CV 2021
Year: 2021

We propose to adapt segmentation networks with a constrained formulation, which embeds domain-invariant prior knowledge about the segmentation regions. Such knowledge may take the form of simple anatomical information, e.g., structure size or shape, estimated from source samples or known a priori. Our method imposes domain-invariant inequality constraints on the network outputs of unlabeled target samples. It implicitly matches prediction statistics between target and source domains with permitted uncertainty of prior knowledge. We address our constrained problem with a differentiable penalty, fully suited for standard stochastic gradient descent approaches, removing the need for computationally expensive Lagrangian optimization with dual projections. Unlike current two-step adversarial training, our formulation is based on a single loss in a single network, which simplifies adaptation by avoiding extra adversarial steps, while improving convergence and quality of training. The comparison of our approach with state-of-the-art adversarial methods reveals substantially better performance on the challenging task of adapting spine segmentation across different MRI modalities. Our results also show a robustness to imprecision of size priors, approaching the accuracy of a fully supervised model trained directly in a target domain.Our method can be readily used for various constraints and segmentation problems.
Lightweight and Scalable Particle Tracking and Motion Clustering of 3D Cell Trajectories
Venue: arXiv cs.CV 2021
Year: 2021

Tracking cell particles in 3D microscopy videos is a challenging task but is of great significance for modeling the motion of cells. Proper characterization of the cell's shape, evolution, and their movement over time is crucial to understanding and modeling the mechanobiology of cell migration in many diseases. One in particular, toxoplasmosis is the disease caused by the parasite Toxoplasma gondii. Roughly, one-third of the world's population tests positive for T. gondii. Its virulence is linked to its lytic cycle, predicated on its motility and ability to enter and exit nucleated cells; therefore, studies elucidating its motility patterns are critical to the eventual development of therapeutic strategies. Here, we present a computational framework for fast and scalable detection, tracking, and identification of T. gondii motion phenotypes in 3D videos, in a completely unsupervised fashion. Our pipeline consists of several different modules including preprocessing, sparsification, cell detection, cell tracking, trajectories extraction, parametrization of the trajectories; and finally, a clustering step. Additionally, we identified the computational bottlenecks, and developed a lightweight and highly scalable pipeline through a combination of task distribution and parallelism. Our results prove both the accuracy and performance of our method.
A Review of Cooperative Multi-Agent Deep Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Deep Reinforcement Learning has made significant progress in multi-agent systems in recent years. In this review article, we have focused on presenting recent approaches on Multi-Agent Reinforcement Learning (MARL) algorithms. In particular, we have focused on five common approaches on modeling and solving cooperative multi-agent reinforcement learning problems: (I) independent learners, (II) fully observable critic, (III) value function factorization, (IV) consensus, and (IV) learn to communicate. First, we elaborate on each of these methods, possible challenges, and how these challenges were mitigated in the relevant papers. If applicable, we further make a connection among different papers in each category. Next, we cover some new emerging research areas in MARL along with the relevant recent papers. Due to the recent success of MARL in real-world applications, we assign a section to provide a review of these applications and corresponding articles. Also, a list of available environments for MARL research is provided in this survey. Finally, the paper is concluded with proposals on the possible research directions.
Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective
Venue: arXiv cs.AI 2021
Year: 2021

Can humans get arbitrarily capable reinforcement learning (RL) agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal? This question impacts how far RL can be scaled, and whether alternative paradigms must be developed in order to build safe artificial general intelligence. In this paper, we study when an RL agent has an instrumental goal to tamper with its reward process, and describe design principles that prevent instrumental goals for two different types of reward tampering (reward function tampering and RF-input tampering). Combined, the design principles can prevent both types of reward tampering from being instrumental goals. The analysis benefits from causal influence diagrams to provide intuitive yet precise formalizations.
Unconstrained Monotonic Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Monotonic neural networks have recently been proposed as a way to define invertible transformations. These transformations can be combined into powerful autoregressive flows that have been shown to be universal approximators of continuous probability distributions. Architectures that ensure monotonicity typically enforce constraints on weights and activation functions, which enables invertibility but leads to a cap on the expressiveness of the resulting transformations. In this work, we propose the Unconstrained Monotonic Neural Network (UMNN) architecture based on the insight that a function is monotonic as long as its derivative is strictly positive. In particular, this latter condition can be enforced with a free-form neural network whose only constraint is the positiveness of its output. We evaluate our new invertible building block within a new autoregressive flow (UMNN-MAF) and demonstrate its effectiveness on density estimation experiments. We also illustrate the ability of UMNNs to improve variational inference.
Nakamoto Consensus with Verifiable Delay Puzzle
Venue: arXiv cs.DC 2021
Year: 2021

This paper presents a new consensus protocol based on verifiable delay function. First, we introduce the concept of verifiable delay puzzle (VDP), which resembles the hashing puzzle used in the PoW mechanism but can only be solved sequentially. We then present a VDP implementation based on the continuous verifiable delay function. Further, we show that VDP can be combined with the Nakamoto consensus in a proof-of-stake/proof-of-delay hybrid protocol. We analyze the persistence and liveness of the protocol, and show that compared to PoW, our proposal consumes much less energy; compared to BFT leader-election based consensus algorithms, our proposal achieves better resistance to long-range attacks and DoS attacks targeting the block proposers.
Neural Architecture Search by Estimation of Network Structure Distributions
Venue: arXiv cs.NE 2021
Year: 2021

The influence of deep learning is continuously expanding across different domains, and its new applications are ubiquitous. The question of neural network design thus increases in importance, as traditional empirical approaches are reaching their limits. Manual design of network architectures from scratch relies heavily on trial and error, while using existing pretrained models can introduce redundancies or vulnerabilities. Automated neural architecture design is able to overcome these problems, but the most successful algorithms operate on significantly constrained design spaces, assuming the target network to consist of identical repeating blocks. While such approach allows for faster search, it does so at the cost of expressivity. We instead propose an alternative probabilistic representation of a whole neural network structure under the assumption of independence between layer types. Our matrix of probabilities is equivalent to the population of models, but allows for discovery of structural irregularities, while being simple to interpret and analyze. We construct an architecture search algorithm, inspired by the estimation of distribution algorithms, to take advantage of this representation. The probability matrix is tuned towards generating high-performance models by repeatedly sampling the architectures and evaluating the corresponding networks, while gradually increasing the model depth. Our algorithm is shown to discover non-regular models which cannot be expressed via blocks, but are competitive both in accuracy and computational cost, while not utilizing complex dataflows or advanced training techniques, as well as remaining conceptually simple and highly extensible.