OpenRaiser/Intern-Atlas huggingface dataset:
Maximum Matchings in Geometric Intersection Graphs
Venue: arXiv cs.CG 2024
Year: 2024

Let $G$ be an intersection graph of $n$ geometric objects in the plane. We show that a maximum matching in $G$ can be found in $O(ρ^{3ω/2}n^{ω/2})$ time with high probability, where $ρ$ is the density of the geometric objects and $ω>2$ is a constant such that $n \times n$ matrices can be multiplied in $O(n^ω)$ time. The same result holds for any subgraph of $G$, as long as a geometric representation is at hand. For this, we combine algebraic methods, namely computing the rank of a matrix via Gaussian elimination, with the fact that geometric intersection graphs have small separators. We also show that in many interesting cases, the maximum matching problem in a general geometric intersection graph can be reduced to the case of bounded density. In particular, a maximum matching in the intersection graph of any family of translates of a convex object in the plane can be found in $O(n^{ω/2})$ time with high probability, and a maximum matching in the intersection graph of a family of planar disks with radii in $[1, Ψ]$ can be found in $O(Ψ^6\log^{11} n + Ψ^{12 ω} n^{ω/2})$ time with high probability.
Investigating Constraint Programming and Hybrid Methods for Real World Industrial Test Laboratory Scheduling
Venue: arXiv cs.AI 2024
Year: 2024

In this paper we deal with a complex real world scheduling problem closely related to the well-known Resource-Constrained Project Scheduling Problem (RCPSP). The problem concerns industrial test laboratories in which a large number of tests has to be performed by qualified personnel using specialised equipment, while respecting deadlines and other constraints. We present different constraint programming models and search strategies for this problem. Furthermore, we propose a Very Large Neighborhood Search approach based on our CP methods. Our models are evaluated using CP solvers and a MIP solver both on real-world test laboratory data and on a set of generated instances of different sizes based on the real-world data. Further, we compare the exact approaches with VLNS and a Simulated Annealing heuristic. We could find feasible solutions for all instances and several optimal solutions and we show that using VLNS we can improve upon the results of the other approaches.
Optimizing Cooperative path-finding: A Scalable Multi-Agent RRT* with Dynamic Potential Fields
Venue: arXiv cs.MA 2024
Year: 2024

Cooperative path-finding in multi-agent systems demands scalable solutions to navigate agents from their origins to destinations without conflict. Despite the breadth of research, scalability remains hampered by increased computational demands in complex environments. This study introduces the multi-agent RRT* potential field (MA-RRT*PF), an innovative algorithm that addresses computational efficiency and path-finding efficacy in dense scenarios. MA-RRT*PF integrates a dynamic potential field with a heuristic method, advancing obstacle avoidance and optimizing the expansion of random trees in congested spaces. The empirical evaluations highlight MA-RRT*PF's significant superiority over conventional multi-agent RRT* (MA-RRT*) in dense environments, offering enhanced performance and solution quality without compromising integrity. This work not only contributes a novel approach to the field of cooperative multi-agent path-finding but also offers a new perspective for practical applications in densely populated settings where traditional methods are less effective.
A Novel Visual Fault Detection and Classification System for Semiconductor Manufacturing Using Stacked Hybrid Convolutional Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

Automated visual inspection in the semiconductor industry aims to detect and classify manufacturing defects utilizing modern image processing techniques. While an earliest possible detection of defect patterns allows quality control and automation of manufacturing chains, manufacturers benefit from an increased yield and reduced manufacturing costs. Since classical image processing systems are limited in their ability to detect novel defect patterns, and machine learning approaches often involve a tremendous amount of computational effort, this contribution introduces a novel deep neural network based hybrid approach. Unlike classical deep neural networks, a multi-stage system allows the detection and classification of the finest structures in pixel size within high-resolution imagery. Consisting of stacked hybrid convolutional neural networks (SH-CNN) and inspired by current approaches of visual attention, the realized system draws the focus over the level of detail from its structures to more task-relevant areas of interest. The results of our test environment show that the SH-CNN outperforms current approaches of learning-based automated visual inspection, whereas a distinction depending on the level of detail enables the elimination of defect patterns in earlier stages of the manufacturing process.
The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems
Venue: arXiv cs.LG 2024
Year: 2024

Methods inspired by Artificial Intelligence (AI) are starting to fundamentally change computational science and engineering through breakthrough performances on challenging problems. However, reliability and trustworthiness of such techniques is a major concern. In inverse problems in imaging, the focus of this paper, there is increasing empirical evidence that methods may suffer from hallucinations, i.e., false, but realistic-looking artifacts; instability, i.e., sensitivity to perturbations in the data; and unpredictable generalization, i.e., excellent performance on some images, but significant deterioration on others. This paper provides a theoretical foundation for these phenomena. We give mathematical explanations for how and when such effects arise in arbitrary reconstruction methods, with several of our results taking the form of `no free lunch' theorems. Specifically, we show that (i) methods that overperform on a single image can wrongly transfer details from one image to another, creating a hallucination, (ii) methods that overperform on two or more images can hallucinate or be unstable, (iii) optimizing the accuracy-stability trade-off is generally difficult, (iv) hallucinations and instabilities, if they occur, are not rare events, and may be encouraged by standard training, (v) it may be impossible to construct optimal reconstruction maps for certain problems. Our results trace these effects to the kernel of the forward operator whenever it is nontrivial, but also apply to the case when the forward operator is ill-conditioned. Based on these insights, our work aims to spur research into new ways to develop robust and reliable AI-based methods for inverse problems in imaging.
Scattering and Sparse Partitions, and their Applications
Venue: arXiv cs.DS 2024
Year: 2024

A partition $\mathcal{P}$ of a weighted graph $G$ is $(σ,τ,Δ)$-sparse if every cluster has diameter at most $Δ$, and every ball of radius $Δ/σ$ intersects at most $τ$ clusters. Similarly, $\mathcal{P}$ is $(σ,τ,Δ)$-scattering if instead for balls we require that every shortest path of length at most $Δ/σ$ intersects at most $τ$ clusters. Given a graph $G$ that admits a $(σ,τ,Δ)$-sparse partition for all $Δ>0$, Jia et al. [STOC05] constructed a solution for the Universal Steiner Tree problem (and also Universal TSP) with stretch $O(τσ^2\log_τn)$. Given a graph $G$ that admits a $(σ,τ,Δ)$-scattering partition for all $Δ>0$, we construct a solution for the Steiner Point Removal problem with stretch $O(τ^3σ^3)$. We then construct sparse and scattering partitions for various different graph families, receiving many new results for the Universal Steiner Tree and Steiner Point Removal problems.
Making deep neural networks right for the right scientific reasons by interacting with their explanations
Venue: arXiv cs.LG 2024
Year: 2024

Deep neural networks have shown excellent performances in many real-world applications. Unfortunately, they may show "Clever Hans"-like behavior -- making use of confounding factors within datasets -- to achieve high performance. In this work, we introduce the novel learning setting of "explanatory interactive learning" (XIL) and illustrate its benefits on a plant phenotyping research task. XIL adds the scientist into the training loop such that she interactively revises the original model via providing feedback on its explanations. Our experimental results demonstrate that XIL can help avoiding Clever Hans moments in machine learning and encourages (or discourages, if appropriate) trust into the underlying model.
Asymptotically Optimal Competitive Ratio for Online Allocation of Reusable Resources
Venue: arXiv cs.DS 2024
Year: 2024

We consider the problem of online allocation (matching, budgeted allocations, and assortments) of reusable resources where an adversarial sequence of resource requests is revealed over time and any allocated resource is used/rented for a stochastic duration drawn independently from a resource dependent usage distribution. Previously, it was known that a greedy algorithm is 0.5--competitive against the clairvoyant benchmark that knows the entire sequence of requests in advance (Gong et al. (2021)). We give a novel algorithm that is $(1-1/e)$--competitive for arbitrary usage distributions when the starting capacity of each resource is large and the usage distributions are known. This is the best achievable competitive ratio guarantee for the problem, i.e., no online algorithm can have better competitive ratio. We also give a distribution oblivious online algorithm and show that it is $(1-1/e)$--competitive in special cases. At the heart of our algorithms is a new quantity that factors in the potential of reusability for each resource by (computationally) creating an asymmetry between identical units of the resource. We establish the performance guarantee for our algorithms by constructing a feasible solution to a novel system of inequalities that allows direct comparison with the clairvoyant benchmark instead of a linear programming (LP) relaxation of the benchmark. Our technique generalizes the primal-dual analysis framework for online resource allocation and may be of broader interest.
PAC Codes: Sequential Decoding vs List Decoding
Venue: arXiv cs.IT 2024
Year: 2024

In the Shannon lecture at the 2019 International Symposium on Information Theory (ISIT), Arıkan proposed to employ a one-to-one convolutional transform as a pre-coding step before the polar transform. The resulting codes of this concatenation are called polarization-adjusted convolutional (PAC) codes. In this scheme, a pair of polar mapper and demapper as pre- and postprocessing devices are deployed around a memoryless channel, which provides polarized information to an outer decoder leading to improved error correction performance of the outer code. In this paper, the list decoding and sequential decoding (including Fano decoding and stack decoding) are first adapted for use to decode PAC codes. Then, to reduce the complexity of sequential decoding of PAC/polar codes, we propose (i) an adaptive heuristic metric, (ii) tree search constraints for backtracking to avoid exploration of unlikely sub-paths, and (iii) tree search strategies consistent with the pattern of error occurrence in polar codes. These contribute to the reduction of the average decoding time complexity from 50% to 80%, trading with 0.05 to 0.3 dB degradation in error correction performance within FER=10^-3 range, respectively, relative to not applying the corresponding search strategies. Additionally, as an important ingredient in Fano decoding of PAC/polar codes, an efficient computation method for the intermediate LLRs and partial sums is provided. This method is effective in backtracking and avoids storing the intermediate information or restarting the decoding process. Eventually, all three decoding algorithms are compared in terms of performance, complexity, and resource requirements.
Bandits with Mean Bounds
Venue: arXiv cs.LG 2024
Year: 2024

We study a variant of the bandit problem where side information in the form of bounds on the mean of each arm is provided. We prove that these translate to tighter estimates of subgaussian factors and develop novel algorithms that exploit these estimates. In the linear setting, we present the Restricted-set OFUL (R-OFUL) algorithm that additionally uses the geometric properties of the problem to (potentially) restrict the set of arms being played and reduce exploration rates for suboptimal arms. In the stochastic case, we propose the non-optimistic Global Under-Explore (GLUE) algorithm which employs the inferred subgaussian estimates to adapt the rate of exploration for the arms. We analyze the regret of R-OFUL and GLUE, showing that our regret upper bounds are never worse than that of the standard OFUL and UCB algorithms respectively. Further, we also consider a practically motivated setting of learning from confounded logs where mean bounds appear naturally.
Extracting and Validating Explanatory Word Archipelagoes using Dual Entropy
Venue: arXiv cs.CL 2024
Year: 2024

The logical connectivity of text is represented by the connectivity of words that form archipelagoes. Here, each archipelago is a sequence of islands of the occurrences of a certain word. An island here means the local sequence of sentences where the word is emphasized, and an archipelago of a length comparable to the target text is extracted using the co-variation of entropy A (the window-based entropy) on the distribution of the word's occurrences with the width of each time window. Then, the logical connectivity of text is evaluated on entropy B (the graph-based entropy) computed on the distribution of sentences to connected word-clusters obtained on the co-occurrence of words. The results show the parts of the target text with words forming archipelagoes extracted on entropy A, without learned or prepared knowledge, form an explanatory part of the text that is of smaller entropy B than the parts extracted by the baseline methods.
On Biased Compression for Distributed Learning
Venue: arXiv cs.LG 2024
Year: 2024

In the last few years, various communication compression techniques have emerged as an indispensable tool helping to alleviate the communication bottleneck in distributed learning. However, despite the fact biased compressors often show superior performance in practice when compared to the much more studied and understood unbiased compressors, very little is known about them. In this work we study three classes of biased compression operators, two of which are new, and their performance when applied to (stochastic) gradient descent and distributed (stochastic) gradient descent. We show for the first time that biased compressors can lead to linear convergence rates both in the single node and distributed settings. We prove that distributed compressed SGD method, employed with error feedback mechanism, enjoys the ergodic rate $O\left( δL \exp \left[-\frac{μK}{δL}\right] + \frac{(C + δD)}{Kμ}\right)$, where $δ\ge 1$ is a compression parameter which grows when more compression is applied, $L$ and $μ$ are the smoothness and strong convexity constants, $C$ captures stochastic gradient noise ($C=0$ if full gradients are computed on each node) and $D$ captures the variance of the gradients at the optimum ($D=0$ for over-parameterized models). Further, via a theoretical study of several synthetic and empirical distributions of communicated gradients, we shed light on why and by how much biased compressors outperform their unbiased variants. Finally, we propose several new biased compressors with promising theoretical guarantees and practical performance.
Towards Data Auctions with Externalities
Venue: arXiv cs.GT 2024
Year: 2024

The design of data markets has gained importance as firms increasingly use machine learning models fueled by externally acquired training data. A key consideration is the externalities firms face when data, though inherently freely replicable, is allocated to competing firms. In this setting, we demonstrate that a data seller's optimal revenue increases as firms can pay to prevent allocations to others. To do so, we first reduce the combinatorial problem of allocating and pricing multiple datasets to the auction of a single digital good by modeling utility for data through the increase in prediction accuracy it provides. We then derive welfare and revenue maximizing mechanisms, highlighting how the form of firms' private information - whether the externalities one exerts on others is known, or vice-versa - affects the resulting structures. In all cases, under appropriate assumptions, the optimal allocation rule is a single threshold per firm, where either all data is allocated or none is.
Visual Grounding Methods for VQA are Working for the Wrong Reasons!
Venue: arXiv cs.CV 2024
Year: 2024

Existing Visual Question Answering (VQA) methods tend to exploit dataset biases and spurious statistical correlations, instead of producing right answers for the right reasons. To address this issue, recent bias mitigation methods for VQA propose to incorporate visual cues (e.g., human attention maps) to better ground the VQA models, showcasing impressive gains. However, we show that the performance improvements are not a result of improved visual grounding, but a regularization effect which prevents over-fitting to linguistic priors. For instance, we find that it is not actually necessary to provide proper, human-based cues; random, insensible cues also result in similar improvements. Based on this observation, we propose a simpler regularization scheme that does not require any external annotations and yet achieves near state-of-the-art performance on VQA-CPv2.
Removable Online Knapsack and Advice
Venue: arXiv cs.DS 2024
Year: 2024

In the knapsack problem, we are given a knapsack of some capacity and a set of items, each with a size and a value. The goal is to pack a selection of these items fitting the knapsack that maximizes the total value. The online version of this problem reveals the items one by one. For each item, the algorithm must decide immediately whether to pack it or not. We consider a natural variant of this problem, coined removable online knapsack. It differs from the classical variant by allowing the removal of packed items. Repacking is impossible, however: Once an item is removed, it is gone for good. We analyze the advice complexity of this problem. It measures how many advice bits an omniscient oracle needs to provide for an online algorithm to reach any given competitive ratio, which is, understood in its strict sense, just the approximation factor. We show that the competitive ratio jumps from unbounded without advice to near-optimal with just constantly many advice bits, a behavior unique among all problems examined so far. We also examine algorithms with barely any advice, for example just a single bit, and analyze the special case of the proportional knapsack problem, where an item's size always equals its value. We show that advice algorithms have various concrete applications and that lower bounds on the advice complexity of any problem are exceptionally strong. Our results improve some of the best known lower bounds on the competitive ratio for randomized algorithms and even for deterministic deterministic algorithms in established models such as knapsack with a resource buffer and various problems with multiple knapsacks. The seminal paper introducing knapsack with removability proposed such a problem for which we can even establish a one-to-one correspondence with the advice model; this paper therefore also provides a comprehensive analysis for this neglected problem.
High-Multiplicity Fair Allocation Using Parametric Integer Linear Programming
Venue: arXiv cs.GT 2024
Year: 2024

Using insights from parametric integer linear programming, we significantly improve on our previous work [Proc. ACM EC 2019] on high-multiplicity fair allocation. Therein, answering an open question from previous work, we proved that the problem of finding envy-free Pareto-efficient allocations of indivisible items is fixed-parameter tractable with respect to the combined parameter "number of agents" plus "number of item types." Our central improvement, compared to this result, is to break the condition that the corresponding utility and multiplicity values have to be encoded in unary required there. Concretely, we show that, while preserving fixed-parameter tractability, these values can be encoded in binary, thus greatly expanding the range of feasible values.
Universal Graph Compression: Stochastic Block Models
Venue: arXiv cs.IT 2024
Year: 2024

Motivated by the prevalent data science applications of processing large-scale graph data such as social networks and biological networks, this paper investigates lossless compression of data in the form of a labeled graph. Particularly, we consider a widely used random graph model, stochastic block model (SBM), which captures the clustering effects in social networks. An information-theoretic universal compression framework is applied, in which one aims to design a single compressor that achieves the asymptotically optimal compression rate, for every SBM distribution, without knowing the parameters of the SBM. Such a graph compressor is proposed in this paper, which universally achieves the optimal compression rate with polynomial time complexity for a wide class of SBMs. Existing universal compression techniques are developed mostly for stationary ergodic one-dimensional sequences. However, the adjacency matrix of SBM has complex two-dimensional correlations. The challenge is alleviated through a carefully designed transform that converts two-dimensional correlated data into almost i.i.d. submatrices. The sequence of submatrices is then compressed by a Krichevsky--Trofimov compressor, whose length analysis is generalized to identically distributed but arbitrarily correlated sequences. In four benchmark graph datasets, the compressed files from competing algorithms take 2.4 to 27 times the space needed by the proposed scheme.
Geometry-Aware Instance Segmentation with Disparity Maps
Venue: arXiv cs.CV 2024
Year: 2024

Most previous works of outdoor instance segmentation for images only use color information. We explore a novel direction of sensor fusion to exploit stereo cameras. Geometric information from disparities helps separate overlapping objects of the same or different classes. Moreover, geometric information penalizes region proposals with unlikely 3D shapes thus suppressing false positive detections. Mask regression is based on 2D, 2.5D, and 3D ROI using the pseudo-lidar and image-based representations. These mask predictions are fused by a mask scoring process. However, public datasets only adopt stereo systems with shorter baseline and focal legnth, which limit measuring ranges of stereo cameras. We collect and utilize High-Quality Driving Stereo (HQDS) dataset, using much longer baseline and focal length with higher resolution. Our performance attains state of the art. Please refer to our project page. The full paper is available here.