OpenRaiser/Intern-Atlas huggingface dataset:
Counting Substructures with Higher-Order Graph Neural Networks: Possibility and Impossibility Results
Venue: arXiv cs.LG 2021
Year: 2021

While message passing Graph Neural Networks (GNNs) have become increasingly popular architectures for learning with graphs, recent works have revealed important shortcomings in their expressive power. In response, several higher-order GNNs have been proposed that substantially increase the expressive power, albeit at a large computational cost. Motivated by this gap, we explore alternative strategies and lower bounds. In particular, we analyze a new recursive pooling technique of local neighborhoods that allows different tradeoffs of computational cost and expressive power. First, we prove that this model can count subgraphs of size $k$, and thereby overcomes a known limitation of low-order GNNs. Second, we show how recursive pooling can exploit sparsity to reduce the computational complexity compared to the existing higher-order GNNs. More generally, we provide a (near) matching information-theoretic lower bound for counting subgraphs with graph representations that pool over representations of derived (sub-)graphs. We also discuss lower bounds on time complexity.
Maximum Entropy Subspace Clustering Network
Venue: arXiv cs.CV 2021
Year: 2021

Deep subspace clustering networks have attracted much attention in subspace clustering, in which an auto-encoder non-linearly maps the input data into a latent space, and a fully connected layer named self-expressiveness module is introduced to learn the affinity matrix via a typical regularization term (e.g., sparse or low-rank). However, the adopted regularization terms ignore the connectivity within each subspace, limiting their clustering performance. In addition, the adopted framework suffers from the coupling issue between the auto-encoder module and the self-expressiveness module, making the network training non-trivial. To tackle these two issues, we propose a novel deep subspace clustering method named Maximum Entropy Subspace Clustering Network (MESC-Net). Specifically, MESC-Net maximizes the entropy of the affinity matrix to promote the connectivity within each subspace, in which its elements corresponding to the same subspace are uniformly and densely distributed. Furthermore, we design a novel framework to explicitly decouple the auto-encoder module and the self-expressiveness module. We also theoretically prove that the learned affinity matrix satisfies the block-diagonal property under the independent subspaces. Extensive quantitative and qualitative results on commonly used benchmark datasets validate MESC-Net significantly outperforms state-of-the-art methods.
TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning has emerged as a new paradigm of collaborative machine learning; however, many prior studies have used global aggregation along a star topology without much consideration of the communication scalability or the diurnal property relied on clients' local time variety. In contrast, ring architecture can resolve the scalability issue and even satisfy the diurnal property by iterating nodes without an aggregation. Nevertheless, such ring-based algorithms can inherently suffer from the high-variance problem. To this end, we propose a novel algorithm called TornadoAggregate that improves both accuracy and scalability by facilitating the ring architecture. In particular, to improve the accuracy, we reformulate the loss minimization into a variance reduction problem and establish three principles to reduce variance: Ring-Aware Grouping, Small Ring, and Ring Chaining. Experimental results show that TornadoAggregate improved the test accuracy by up to 26.7% and achieved near-linear scalability.
Pilot-Based Unsourced Random Access with a Massive MIMO Receiver in the Quasi-Static Fading Regime
Venue: arXiv cs.IT 2021
Year: 2021

In this work we treat the unsourced random access problem on a Rayleigh block-fading AWGN channel with multiple receive antennas. Specifically, we consider the slowly fading scenario where the coherence block-length is large compared to the number of active users and the message can be transmitted in one coherence block. Unsourced random access refers to a form of grant-free random access where users are considered to be a-priori indistinguishable and the receiver recovers a list of transmitted messages up to permutation. In this work we show that, when the coherence block length is large enough, a conventional approach based on the transmission of non-orthogonal pilot sequences with subsequent channel estimation and Maximum-Ratio-Combining (MRC) provides a simple energy-efficient solution whose performance can be well approximated in closed form. For the finite block-length simulations we use a randomly sub-sampled DFT matrix as pilot matrix, a low-complexity approximate message passing algorithm for activity detection and a state-of-the-art polar code as single-user error correction code with a successive-cancellation-list decoder. These simulations prove the scalability of the presented approach and the quality of the analysis.
Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks excel at comprehending complex visual signals, delivering on par or even superior performance to that of human experts. However, ad-hoc visual explanations of model decisions often reveal an alarming level of reliance on exploiting non-causal visual cues that strongly correlate with the target label in training data. As such, deep neural nets suffer compromised generalization to novel inputs collected from different sources, and the reverse engineering of their decision rules offers limited interpretability. To overcome these limitations, we present a novel contrastive learning strategy called {\it Proactive Pseudo-Intervention} (PPI) that leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability. To demonstrate the utility of our proposals, we benchmark on both standard natural images and challenging medical image datasets. PPI-enhanced models consistently deliver superior performance relative to competing solutions, especially on out-of-domain predictions and data integration from heterogeneous sources. Further, our causally trained saliency maps are more succinct and meaningful relative to their non-causal counterparts.
Using topological autoencoders as a filtering function for global and local topology
Venue: arXiv cs.CG 2021
Year: 2021

Choosing a suitable filtering function for the Mapper algorithm can be difficult due to its arbitrariness and domain-specific requirements. Finding a general filtering function that can be applied across domains is therefore of interest, since it would improve the representation of manifolds in higher dimensions. In this extended abstract, we propose that topological autoencoders is a suitable candidate for this and report initial results strengthening this hypothesis for one set of high-dimensional manifolds. The results indicate a potential for an easier choice of filtering function when using the Mapper algorithm, allowing for a more general and descriptive representation of high-dimensional data.
Multi-agent Policy Optimization with Approximatively Synchronous Advantage Estimation
Venue: arXiv cs.LG 2021
Year: 2021

Cooperative multi-agent tasks require agents to deduce their own contributions with shared global rewards, known as the challenge of credit assignment. General methods for policy based multi-agent reinforcement learning to solve the challenge introduce differentiate value functions or advantage functions for individual agents. In multi-agent system, polices of different agents need to be evaluated jointly. In order to update polices synchronously, such value functions or advantage functions also need synchronous evaluation. However, in current methods, value functions or advantage functions use counter-factual joint actions which are evaluated asynchronously, thus suffer from natural estimation bias. In this work, we propose the approximatively synchronous advantage estimation. We first derive the marginal advantage function, an expansion from single-agent advantage function to multi-agent system. Further more, we introduce a policy approximation for synchronous advantage estimation, and break down the multi-agent policy optimization problem into multiple sub-problems of single-agent policy optimization. Our method is compared with baseline algorithms on StarCraft multi-agent challenges, and shows the best performance on most of the tasks.
Matching Distributions via Optimal Transport for Semi-Supervised Learning
Venue: arXiv cs.CV 2021
Year: 2021

Semi-Supervised Learning (SSL) approaches have been an influential framework for the usage of unlabeled data when there is not a sufficient amount of labeled data available over the course of training. SSL methods based on Convolutional Neural Networks (CNNs) have recently provided successful results on standard benchmark tasks such as image classification. In this work, we consider the general setting of SSL problem where the labeled and unlabeled data come from the same underlying probability distribution. We propose a new approach that adopts an Optimal Transport (OT) technique serving as a metric of similarity between discrete empirical probability measures to provide pseudo-labels for the unlabeled data, which can then be used in conjunction with the initial labeled data to train the CNN model in an SSL manner. We have evaluated and compared our proposed method with state-of-the-art SSL algorithms on standard datasets to demonstrate the superiority and effectiveness of our SSL algorithm.
GAEA: Graph Augmentation for Equitable Access via Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Disparate access to resources by different subpopulations is a prevalent issue in societal and sociotechnical networks. For example, urban infrastructure networks may enable certain racial groups to more easily access resources such as high-quality schools, grocery stores, and polling places. Similarly, social networks within universities and organizations may enable certain groups to more easily access people with valuable information or influence. Here we introduce a new class of problems, Graph Augmentation for Equitable Access (GAEA), to enhance equity in networked systems by editing graph edges under budget constraints. We prove such problems are NP-hard, and cannot be approximated within a factor of $(1-\tfrac{1}{3e})$. We develop a principled, sample- and time- efficient Markov Reward Process (MRP)-based mechanism design framework for GAEA. Our algorithm outperforms baselines on a diverse set of synthetic graphs. We further demonstrate the method on real-world networks, by merging public census, school, and transportation datasets for the city of Chicago and applying our algorithm to find human-interpretable edits to the bus network that enhance equitable access to high-quality schools across racial groups. Further experiments on Facebook networks of universities yield sets of new social connections that would increase equitable access to certain attributed nodes across gender groups.
Quantum LDPC Codes with Almost Linear Minimum Distance
Venue: arXiv cs.IT 2021
Year: 2021

We give a construction of quantum LDPC codes of dimension $Θ(\log N)$ and distance $Θ(N/\log N)$ as the code length $N\to\infty$. Using a product of chain complexes this construction also provides a family of quantum LDPC codes of distance $Ω(N^{1-α/2}/\log N)$ and dimension $Ω(N^α\log N)$, where $0 \le α< 1$. We also introduce and study a new operation called lifted product, which naturally generalizes the product operations for quantum codes and chain complexes. Moreover, as a simple byproduct of our results on quantum codes, we obtain a new result on classical codes. We show that for any fixed $R < 1$ there exists an asymptotically good family of classical quasi-cyclic LDPC codes of rate at least $R$ with, in some sense, optimal circulant size $Ω(N/\log N)$ as the code length $N\to\infty$.
Cross-lingual Transfer of Abstractive Summarizer to Less-resource Language
Venue: arXiv cs.CL 2021
Year: 2021

Automatic text summarization extracts important information from texts and presents the information in the form of a summary. Abstractive summarization approaches progressed significantly by switching to deep neural networks, but results are not yet satisfactory, especially for languages where large training sets do not exist. In several natural language processing tasks, a cross-lingual model transfer is successfully applied in less-resource languages. For summarization, the cross-lingual model transfer was not attempted due to a non-reusable decoder side of neural models that cannot correct target language generation. In our work, we use a pre-trained English summarization model based on deep neural networks and sequence-to-sequence architecture to summarize Slovene news articles. We address the problem of inadequate decoder by using an additional language model for the evaluation of the generated text in target language. We test several cross-lingual summarization models with different amounts of target data for fine-tuning. We assess the models with automatic evaluation measures and conduct a small-scale human evaluation. Automatic evaluation shows that the summaries of our best cross-lingual model are useful and of quality similar to the model trained only in the target language. Human evaluation shows that our best model generates summaries with high accuracy and acceptable readability. However, similar to other abstractive models, our models are not perfect and may occasionally produce misleading or absurd content.
Settling the complexity of Nash equilibrium in congestion games
Venue: arXiv cs.CC 2021
Year: 2021

We consider (i) the problem of finding a (possibly mixed) Nash equilibrium in congestion games, and (ii) the problem of finding an (exponential precision) fixed point of the gradient descent dynamics of a smooth function $f:[0,1]^n \rightarrow \mathbb{R}$. We prove that these problems are equivalent. Our result holds for various explicit descriptions of $f$, ranging from (almost general) arithmetic circuits, to degree-$5$ polynomials. By a very recent result of [Fearnley, Goldberg, Hollender, Savani '20] this implies that these problems are PPAD$\cap$PLS-complete. As a corollary, we also obtain the following equivalence of complexity classes: CCLS = PPAD$\cap$PLS.
Graph-based open-ended survey on concerns related to COVID-19
Venue: arXiv cs.SI 2021
Year: 2021

The COVID-19 pandemic is an unprecedented public health crisis with broad social and economic consequences. We conducted four surveys between April and August 2020 using the graph-based open-ended survey (GOS) framework, and investigated the most pressing concerns and issues for the general public in Japan. The GOS framework is a hybrid of the two traditional survey frameworks that allows respondents to post their opinions in a free-format style, which can subsequently serve as one of the choice items for other respondents, just as in a multiple-choice survey. As a result, this framework generates an opinion graph that relates opinions and respondents. We can also construct annotated opinion graphs to achieve a higher resolution. By clustering the annotated opinion graphs, we revealed the characteristic evolution of the response patterns as well as the interconnectedness and multi-faceted nature of opinions. Substantively, our notable finding is that "social pressure," not "infection risk," was one of the major concerns of our respondents. Social pressure refers to criticism and discrimination that they anticipate receiving from others should they contract COVID-19. It is possible that the collectivist nature of Japanese culture coupled with the government's policy of relying on personal responsibility to combat COVID-19 explains some of the above findings, as the latter has led to the emergence of vigilantes. The presence of mutual surveillance can contribute to growing skepticism toward others as well as fear of ostracism, which may have negative consequences at both the societal and individual levels.
On the geometry of geometric rank
Venue: arXiv cs.CC 2021
Year: 2021

We make a geometric study of the Geometric Rank of tensors recently introduced by Kopparty et al. Results include classification of tensors with degenerate geometric rank in $C^3\otimes C^3\otimes C^3$, classification of tensors with geometric rank two, and showing that upper bounds on geometric rank imply lower bounds on tensor rank.
Asynchronous Guessing Subject to Distortion
Venue: arXiv cs.IT 2021
Year: 2021

The problem of guessing subject to distortion is considered, and the performance of randomized guessing strategies is investigated. A one-shot achievability bound on the guessing moment (i.e., moment of the number of required queries) is given. Applying this result to i.i.d.~sources, it is shown that randomized strategies can asymptotically attain the optimal guessing moment. Further, a randomized guessing scheme which is feasible even when the block size is extremely large is proposed, and a single-letter characterization of the guessing moment achievable by the proposed scheme is obtained.
Strong but Simple Baseline with Dual-Granularity Triplet Loss for Visible-Thermal Person Re-Identification
Venue: arXiv cs.CV 2021
Year: 2021

In this letter, we propose a conceptually simple and effective dual-granularity triplet loss for visible-thermal person re-identification (VT-ReID). In general, ReID models are always trained with the sample-based triplet loss and identification loss from the fine granularity level. It is possible when a center-based loss is introduced to encourage the intra-class compactness and inter-class discrimination from the coarse granularity level. Our proposed dual-granularity triplet loss well organizes the sample-based triplet loss and center-based triplet loss in a hierarchical fine to coarse granularity manner, just with some simple configurations of typical operations, such as pooling and batch normalization. Experiments on RegDB and SYSU-MM01 datasets show that with only the global features our dual-granularity triplet loss can improve the VT-ReID performance by a significant margin. It can be a strong VT-ReID baseline to boost future research with high quality.
NSL: Hybrid Interpretable Learning From Noisy Raw Data
Venue: arXiv cs.LG 2021
Year: 2021

Inductive Logic Programming (ILP) systems learn generalised, interpretable rules in a data-efficient manner utilising existing background knowledge. However, current ILP systems require training examples to be specified in a structured logical format. Neural networks learn from unstructured data, although their learned models may be difficult to interpret and are vulnerable to data perturbations at run-time. This paper introduces a hybrid neural-symbolic learning framework, called NSL, that learns interpretable rules from labelled unstructured data. NSL combines pre-trained neural networks for feature extraction with FastLAS, a state-of-the-art ILP system for rule learning under the answer set semantics. Features extracted by the neural components define the structured context of labelled examples and the confidence of the neural predictions determines the level of noise of the examples. Using the scoring function of FastLAS, NSL searches for short, interpretable rules that generalise over such noisy examples. We evaluate our framework on propositional and first-order classification tasks using the MNIST dataset as raw data. Specifically, we demonstrate that NSL is able to learn robust rules from perturbed MNIST data and achieve comparable or superior accuracy when compared to neural network and random forest baselines whilst being more general and interpretable.
Robotic Communications for 5G and Beyond: Challenges and Research Opportunities
Venue: arXiv cs.RO 2021
Year: 2021

The ongoing surge in applications of robotics brings both opportunities and challenges for the fifth-generation (5G) and beyond (B5G) of communication networks. This article focuses on 5G/B5G-enabled terrestrial robotic communications with an emphasis on distinct characteristics of such communications. Firstly, signal and spatial modeling for robotic communications are presented. To elaborate further, both the benefits and challenges derived from robots' mobility are discussed. As a further advance, a novel simultaneous localization and radio mapping (SLARM) framework is proposed for integrating localization and communications into robotic networks. Furthermore, dynamic trajectory design and resource allocation for both indoor and outdoor robots are provided to verify the performance of robotic communications in the context of typical robotic application scenarios.
Setting the Record Straighter on Shadow Banning
Venue: arXiv cs.SI 2021
Year: 2021

Shadow banning consists for an online social network in limiting the visibility of some of its users, without them being aware of it. Twitter declares that it does not use such a practice, sometimes arguing about the occurrence of "bugs" to justify restrictions on some users. This paper is the first to address the plausibility or not of shadow banning on a major online platform, by adopting both a statistical and a graph topological approach. We first conduct an extensive data collection and analysis campaign, gathering occurrences of visibility limitations on user profiles (we crawl more than 2.5 million of them). In such a black-box observation setup, we highlight the salient user profile features that may explain a banning practice (using machine learning predictors). We then pose two hypotheses for the phenomenon: i) limitations are bugs, as claimed by Twitter, and ii) shadow banning propagates as an epidemic on user-interactions ego-graphs. We show that hypothesis i) is statistically unlikely with regards to the data we collected. We then show some interesting correlation with hypothesis ii), suggesting that the interaction topology is a good indicator of the presence of groups of shadow banned users on the service.