OpenRaiser/Intern-Atlas huggingface dataset:
Parameterized Inapproximability of Independent Set in $H$-Free Graphs
Venue: arXiv cs.CC 2022
Year: 2022

We study the Independent Set (IS) problem in $H$-free graphs, i.e., graphs excluding some fixed graph $H$ as an induced subgraph. We prove several inapproximability results both for polynomial-time and parameterized algorithms. Halldórsson [SODA 1995] showed that for every $δ>0$ IS has a polynomial-time $(\frac{d-1}{2}+δ)$-approximation in $K_{1,d}$-free graphs. We extend this result by showing that $K_{a,b}$-free graphs admit a polynomial-time $O(α(G)^{1-1/a})$-approximation, where $α(G)$ is the size of a maximum independent set in $G$. Furthermore, we complement the result of Halldórsson by showing that for some $γ=Θ(d/\log d),$ there is no polynomial-time $γ$-approximation for these graphs, unless NP = ZPP. Bonnet et al. [IPEC 2018] showed that IS parameterized by the size $k$ of the independent set is W[1]-hard on graphs which do not contain (1) a cycle of constant length at least $4$, (2) the star $K_{1,4}$, and (3) any tree with two vertices of degree at least $3$ at constant distance. We strengthen this result by proving three inapproximability results under different complexity assumptions for almost the same class of graphs (we weaken condition (2) that $G$ does not contain $K_{1,5}$). First, under the ETH, there is no $f(k)\cdot n^{o(k/\log k)}$ algorithm for any computable function $f$. Then, under the deterministic Gap-ETH, there is a constant $δ>0$ such that no $δ$-approximation can be computed in $f(k) \cdot n^{O(1)}$ time. Also, under the stronger randomized Gap-ETH there is no such approximation algorithm with runtime $f(k)\cdot n^{o(\sqrt{k})}$. Finally, we consider the parameterization by the excluded graph $H$, and show that under the ETH, IS has no $n^{o(α(H))}$ algorithm in $H$-free graphs and under Gap-ETH there is no $d/k^{o(1)}$-approximation for $K_{1,d}$-free graphs with runtime $f(d,k) n^{O(1)}$.
Shannon meets Myerson: Information Extraction from a Strategic Sender
Venue: arXiv cs.IT 2022
Year: 2022

We study a setting where a receiver must design a questionnaire to recover a sequence of symbols known to strategic sender, whose utility may not be incentive compatible. We allow the receiver the possibility of selecting the alternatives presented in the questionnaire, and thereby linking decisions across the components of the sequence. We show that, despite the strategic sender and the noise in the channel, the receiver can recover exponentially many sequences, but also that exponentially many sequences are unrecoverable even by the best strategy. We define the growth rate of the number of recovered sequences as the information extraction capacity. A generalization of the Shannon capacity, it characterizes the optimal amount of communication resources required. We derive bounds leading to an exact evaluation of the information extraction capacity in many cases. Our results form the building blocks of a novel, noncooperative regime of communication involving a strategic sender.
Zero-Shot Learning with Common Sense Knowledge Graphs
Venue: arXiv cs.LG 2022
Year: 2022

Zero-shot learning relies on semantic class representations such as hand-engineered attributes or learned embeddings to predict classes without any labeled examples. We propose to learn class representations by embedding nodes from common sense knowledge graphs in a vector space. Common sense knowledge graphs are an untapped source of explicit high-level knowledge that requires little human effort to apply to a range of tasks. To capture the knowledge in the graph, we introduce ZSL-KG, a general-purpose framework with a novel transformer graph convolutional network (TrGCN) for generating class representations. Our proposed TrGCN architecture computes non-linear combinations of node neighbourhoods. Our results show that ZSL-KG improves over existing WordNet-based methods on five out of six zero-shot benchmark datasets in language and vision.
Cyclic Differentiable Architecture Search
Venue: arXiv cs.CV 2022
Year: 2022

Differentiable ARchiTecture Search, i.e., DARTS, has drawn great attention in neural architecture search. It tries to find the optimal architecture in a shallow search network and then measures its performance in a deep evaluation network. The independent optimization of the search and evaluation networks, however, leaves room for potential improvement by allowing interaction between the two networks. To address the problematic optimization issue, we propose new joint optimization objectives and a novel Cyclic Differentiable ARchiTecture Search framework, dubbed CDARTS. Considering the structure difference, CDARTS builds a cyclic feedback mechanism between the search and evaluation networks with introspective distillation. First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized. Second, the architecture weights in the search network are further optimized by the label supervision in classification, as well as the regularization from the evaluation network through feature distillation. Repeating the above cycle results in joint optimization of the search and evaluation networks and thus enables the evolution of the architecture to fit the final evaluation network. The experiments and analysis on CIFAR, ImageNet and NAS-Bench-201 demonstrate the effectiveness of the proposed approach over the state-of-the-art ones. Specifically, in the DARTS search space, we achieve 97.52% top-1 accuracy on CIFAR10 and 76.3% top-1 accuracy on ImageNet. In the chain-structured search space, we achieve 78.2% top-1 accuracy on ImageNet, which is 1.1% higher than EfficientNet-B0. Our code and models are publicly available at https://github.com/microsoft/Cream.
On the Theoretical Equivalence of Several Trade-Off Curves Assessing Statistical Proximity
Venue: arXiv cs.LG 2022
Year: 2022

The recent advent of powerful generative models has triggered the renewed development of quantitative measures to assess the proximity of two probability distributions. As the scalar Frechet inception distance remains popular, several methods have explored computing entire curves, which reveal the trade-off between the fidelity and variability of the first distribution with respect to the second one. Several of such variants have been proposed independently and while intuitively similar, their relationship has not yet been made explicit. In an effort to make the emerging picture of generative evaluation more clear, we propose a unification of four curves known respectively as: the precision-recall (PR) curve, the Lorenz curve, the receiver operating characteristic (ROC) curve and a special case of Rényi divergence frontiers. In addition, we discuss possible links between PR / Lorenz curves with the derivation of domain adaptation bounds.
Metaheuristics for the Online Printing Shop Scheduling Problem
Venue: arXiv cs.AI 2022
Year: 2022

In this work, the online printing shop scheduling problem introduced in (Lunardi et al., Mixed Integer Linear Programming and Constraint Programming Models for the Online Printing Shop Scheduling Problem, Computers & Operations Research, to appear) is considered. This challenging real scheduling problem, that emerged in the nowadays printing industry, corresponds to a flexible job shop scheduling problem with sequencing flexibility; and it presents several complicating specificities such as resumable operations, periods of unavailability of the machines, sequence-dependent setup times, partial overlapping between operations with precedence constraints, and fixed operations, among others. A local search strategy and metaheuristic approaches for the problem are proposed and evaluated. Based on a common representation scheme, trajectory and populational metaheuristics are considered. Extensive numerical experiments with large-sized instances show that the proposed methods are suitable for solving practical instances of the problem; and that they outperform a half-heuristic-half-exact off-the-shelf solver by a large extent. Numerical experiments with classical instances of the flexible job shop scheduling problem show that the introduced methods are also competitive when applied to this particular case.
Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binning
Venue: arXiv cs.LG 2022
Year: 2022

Post-hoc multi-class calibration is a common approach for providing high-quality confidence estimates of deep neural network predictions. Recent work has shown that widely used scaling methods underestimate their calibration error, while alternative Histogram Binning (HB) methods often fail to preserve classification accuracy. When classes have small prior probabilities, HB also faces the issue of severe sample-inefficiency after the conversion into K one-vs-rest class-wise calibration problems. The goal of this paper is to resolve the identified issues of HB in order to provide calibrated confidence estimates using only a small holdout calibration dataset for bin optimization while preserving multi-class ranking accuracy. From an information-theoretic perspective, we derive the I-Max concept for binning, which maximizes the mutual information between labels and quantized logits. This concept mitigates potential loss in ranking performance due to lossy quantization, and by disentangling the optimization of bin edges and representatives allows simultaneous improvement of ranking and calibration performance. To improve the sample efficiency and estimates from a small calibration set, we propose a shared class-wise (sCW) calibration strategy, sharing one calibrator among similar classes (e.g., with similar class priors) so that the training sets of their class-wise calibration problems can be merged to train the single calibrator. The combination of sCW and I-Max binning outperforms the state of the art calibration methods on various evaluation metrics across different benchmark datasets and models, using a small calibration set (e.g., 1k samples for ImageNet).
Formalizing the Soundness of the Encoding Methods of SAT-based Model Checking
Venue: arXiv cs.LO 2022
Year: 2022

One of the effective model checking methods is to utilize the efficient decision procedure of SAT (or SMT) solvers. In a SAT-based model checking, a system and its property are encoded into a set of logic formulas and the safety is checked based on the satisfiability of the formulas. As the encoding methods are improved and crafted (e.g., k-induction and IC3/PDR), verifying their correctness becomes more important. This research aims at a formal verification of the SMC methods using the Coq proof assistant. Our contributions are twofold: (1) We specify the basic encoding methods, k-induction and (a simplified version of) IC3/PDR in Coq as a set of simple and modular encoding predicates. (2) We provide a formal proof of the soundness of the encoding methods based on our formalized lemmas on state sequences and paths.
Clustering with Tangles: Algorithmic Framework and Theoretical Guarantees
Venue: arXiv cs.LG 2022
Year: 2022

Originally, tangles were invented as an abstract tool in mathematical graph theory to prove the famous graph minor theorem. In this paper, we showcase the practical potential of tangles in machine learning applications. Given a collection of cuts of any dataset, tangles aggregate these cuts to point in the direction of a dense structure. As a result, a cluster is softly characterized by a set of consistent pointers. This highly flexible approach can solve clustering problems in various setups, ranging from questionnaires over community detection in graphs to clustering points in metric spaces. The output of our proposed framework is hierarchical and induces the notion of a soft dendrogram, which can help explore the cluster structure of a dataset. The computational complexity of aggregating the cuts is linear in the number of data points. Thus the bottleneck of the tangle approach is to generate the cuts, for which simple and fast algorithms form a sufficient basis. In our paper we construct the algorithmic framework for clustering with tangles, prove theoretical guarantees in various settings, and provide extensive simulations and use cases. Python code is available on github.
Can We Mitigate Backdoor Attack Using Adversarial Detection Methods?
Venue: arXiv cs.LG 2022
Year: 2022

Deep Neural Networks are well known to be vulnerable to adversarial attacks and backdoor attacks, where minor modifications on the input are able to mislead the models to give wrong results. Although defenses against adversarial attacks have been widely studied, investigation on mitigating backdoor attacks is still at an early stage. It is unknown whether there are any connections and common characteristics between the defenses against these two attacks. We conduct comprehensive studies on the connections between adversarial examples and backdoor examples of Deep Neural Networks to seek to answer the question: can we detect backdoor using adversarial detection methods. Our insights are based on the observation that both adversarial examples and backdoor examples have anomalies during the inference process, highly distinguishable from benign samples. As a result, we revise four existing adversarial defense methods for detecting backdoor examples. Extensive evaluations indicate that these approaches provide reliable protection against backdoor attacks, with a higher accuracy than detecting adversarial examples. These solutions also reveal the relations of adversarial examples, backdoor examples and normal samples in model sensitivity, activation space and feature space. This is able to enhance our understanding about the inherent features of these two attacks and the defense opportunities.
Online 3D Bin Packing with Constrained Deep Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In our problem, the agent has limited information about the items to be packed into the bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item's placement also subjects to the constraints of collision avoidance and physical stability. We formulate this online 3D-BPP as a constrained Markov decision process. To solve the problem, we propose an effective and easy-to-implement constrained deep reinforcement learning (DRL) method under the actor-critic framework. In particular, we introduce a feasibility predictor to predict the feasibility mask for the placement actions and use it to modulate the action probabilities output by the actor during training. Such supervisions and transformations to DRL facilitate the agent to learn feasible policies efficiently. Our method can also be generalized e.g., with the ability to handle lookahead or items with different orientations. We have conducted extensive evaluation showing that the learned policy significantly outperforms the state-of-the-art methods. A user study suggests that our method attains a human-level performance.
New Framework for Code-Mapping-based Reversible Data Hiding in JPEG Images
Venue: arXiv cs.MM 2022
Year: 2022

Code mapping (CM) is an efficient technique for reversible data hiding (RDH) in JPEG images, which embeds data by constructing a mapping relationship between the used and unused codes in the JPEG bitstream. This study presents a new framework for designing a CM-based RDH method. First, a new code mapping strategy is proposed to suppress file size expansion and improve applicability. Based on our proposed strategy, the mapped codes are redefined by creating a new Huffman table rather than selecting them from the unused codes in the original Huffman table. The critical issue of designing the CM-based RDH method, that is, constructing code mapping, is converted into a combinatorial optimization problem. This study proposes a novel CM-based RDH method that utilizes a genetic algorithm (GA). The experimental results demonstrate that the proposed method achieves a high embedding capacity with no signal distortion while suppressing file size expansion.
Truncated Channel Inversion Power Control to Enable One-Way URLLC with Imperfect Channel Reciprocity
Venue: arXiv cs.IT 2022
Year: 2022

We propose to use channel inversion power control (CIPC) to achieve one-way ultra-reliable and low-latency communications (URLLC), where only the transmission in one direction requires ultra reliability and low latency. Based on channel reciprocity, our proposed CIPC schemes guarantee the power of received signal that is used to decode the information to be a constant value $Q$, by varying the transmit signal and power, which relaxes the assumption of knowing channel state information (CSI) at the user. Thus, the CIPC schemes eliminate the overhead of CSI feedback, reduce communication latency, and explore the benefits of multiple antennas to significantly improve transmission reliability. We derive analytical expressions for the packet loss probability of the proposed CIPC schemes, based on which we determine a closed interval and a convex set for optimizing $Q$ in CIPC with imperfect and perfect channel reciprocity, respectively. Our results show that CIPC is an effective means to achieve one-way URLLC. The tradeoff among reliability, latency, and required resources (e.g., transmit antennas) is further revealed, which provides novel principles for designing one-way URLLC systems.
A Bayesian regularization-backpropagation neural network model for peeling computations
Venue: arXiv cs.CE 2022
Year: 2022

Bayesian regularization-backpropagation neural network (BR-BPNN) model is employed to predict some aspects of the gecko spatula peeling viz. the variation of the maximum normal and tangential pull-off forces and the resultant force angle at detachment with the peeling angle. K-fold cross validation is used to improve the effectiveness of the model. The input data is taken from finite element (FE) peeling results. The neural network is trained with 75% of the FE dataset. The remaining 25% are utilized to predict the peeling behavior. The training performance is evaluated for every change in the number of hidden layer neurons to determine the optimal network structure. The relative error is calculated to draw a clear comparison between predicted and FE results. It is shown that the BR-BPNN model in conjunction with k-fold technique has significant potential to estimate the peeling behavior.
Scaling Graph Neural Networks with Approximate PageRank
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks (GNNs) have emerged as a powerful approach for solving many network mining tasks. However, learning on large graphs remains a challenge - many recently proposed scalable GNN approaches rely on an expensive message-passing procedure to propagate information through the graph. We present the PPRGo model which utilizes an efficient approximation of information diffusion in GNNs resulting in significant speed gains while maintaining state-of-the-art prediction performance. In addition to being faster, PPRGo is inherently scalable, and can be trivially parallelized for large datasets like those found in industry settings. We demonstrate that PPRGo outperforms baselines in both distributed and single-machine training environments on a number of commonly used academic graphs. To better analyze the scalability of large-scale graph learning methods, we introduce a novel benchmark graph with 12.4 million nodes, 173 million edges, and 2.8 million node features. We show that training PPRGo from scratch and predicting labels for all nodes in this graph takes under 2 minutes on a single machine, far outpacing other baselines on the same graph. We discuss the practical application of PPRGo to solve large-scale node classification problems at Google.
Multi-Winner Voting with Approval Preferences
Venue: arXiv cs.GT 2022
Year: 2022

Multi-winner voting is the process of selecting a fixed-size set of representative candidates based on voters' preferences. It occurs in applications ranging from politics (parliamentary elections) to the design of modern computer applications (collaborative filtering, dynamic Q&A platforms, diversifying search results). All these applications share the problem of identifying a representative subset of alternatives -- and the study of multi-winner voting is the principled analysis of this task. This book provides a thorough and in-depth look at multi-winner voting based on approval preferences. One speaks of approval preferences if voters express their preferences by providing a set of candidates they approve. Approval preferences thus separate candidates in approved and disapproved ones, a simple, binary classification. The corresponding multi-winner voting rules are called approval-based committee (ABC) rules. Due to the simplicity of approval preferences, ABC rules are widely suitable for practical use. Recent years have seen a rising interest in ABC voting. While multi-winner voting has been originally a topic studied by economists and political scientists, a significant share of recent progress has occurred in the field of computational social choice. This discipline is situated in the intersection of artificial intelligence, computer science, economics, and (to a lesser degree) political science, combining insights and methods from these distinct fields. The goal of this book is to present fundamental concepts and results for ABC voting and to discuss the recent advances in computational social choice. The main focus is on axiomatic analysis, algorithmic results, and relevant applications.
Overlapping Spaces for Compact Graph Representations
Venue: arXiv cs.LG 2022
Year: 2022

Various non-trivial spaces are becoming popular for embedding structured data such as graphs, texts, or images. Following spherical and hyperbolic spaces, more general product spaces have been proposed. However, searching for the best configuration of product space is a resource-intensive procedure, which reduces the practical applicability of the idea. We generalize the concept of product space and introduce an overlapping space that does not have the configuration search problem. The main idea is to allow subsets of coordinates to be shared between spaces of different types (Euclidean, hyperbolic, spherical). As a result, parameter optimization automatically learns the optimal configuration. Additionally, overlapping spaces allow for more compact representations since their geometry is more complex. Our experiments confirm that overlapping spaces outperform the competitors in graph embedding tasks. Here, we consider both distortion setup, where the aim is to preserve distances, and ranking setup, where the relative order should be preserved. The proposed method effectively solves the problem and outperforms the competitors in both settings. We also perform an empirical analysis in a realistic information retrieval task, where we compare all spaces by incorporating them into DSSM. In this case, the proposed overlapping space consistently achieves nearly optimal results without any configuration tuning. This allows for reducing training time, which can be significant in large-scale applications.
Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

We present Wiki-CS, a novel dataset derived from Wikipedia for benchmarking Graph Neural Networks. The dataset consists of nodes corresponding to Computer Science articles, with edges based on hyperlinks and 10 classes representing different branches of the field. We use the dataset to evaluate semi-supervised node classification and single-relation link prediction models. Our experiments show that these methods perform well on a new domain, with structural properties different from earlier benchmarks. The dataset is publicly available, along with the implementation of the data pipeline and the benchmark experiments, at https://github.com/pmernyei/wiki-cs-dataset .