OpenRaiser/Intern-Atlas huggingface dataset:
End-to-End Video Text Spotting with Transformer
Venue: arXiv cs.CV 2022
Year: 2022

Recent video text spotting methods usually require the three-staged pipeline, i.e., detecting text in individual images, recognizing localized text, tracking text streams with post-processing to generate final results. These methods typically follow the tracking-by-match paradigm and develop sophisticated pipelines. In this paper, rooted in Transformer sequence modeling, we propose a simple, but effective end-to-end video text DEtection, Tracking, and Recognition framework (TransDETR). TransDETR mainly includes two advantages: 1) Different from the explicit match paradigm in the adjacent frame, TransDETR tracks and recognizes each text implicitly by the different query termed text query over long-range temporal sequence (more than 7 frames). 2) TransDETR is the first end-to-end trainable video text spotting framework, which simultaneously addresses the three sub-tasks (e.g., text detection, tracking, recognition). Extensive experiments in four video text datasets (i.e.,ICDAR2013 Video, ICDAR2015 Video, Minetto, and YouTube Video Text) are conducted to demonstrate that TransDETR achieves state-of-the-art performance with up to around 8.0% improvements on video text spotting tasks. The code of TransDETR can be found at https://github.com/weijiawu/TransDETR.
Multi-Agent Terraforming: Efficient Multi-Agent Path Finding via Environment Manipulation
Venue: arXiv cs.AI 2022
Year: 2022

Multi-agent pathfinding (MAPF) is concerned with planning collision-free paths for a team of agents from their start to goal locations in an environment cluttered with obstacles. Typical approaches for MAPF consider the locations of obstacles as being fixed, which limits their effectiveness in automated warehouses, where obstacles (representing pods or shelves) can be moved out of the way by agents (representing robots) to relieve bottlenecks and introduce shorter routes. In this work we initiate the study of MAPF with movable obstacles. In particular, we introduce a new extension of MAPF, which we call Terraforming MAPF (tMAPF), where some agents are responsible for moving obstacles to clear the way for other agents. Solving tMAPF is extremely challenging as it requires reasoning not only about collisions between agents, but also where and when obstacles should be moved. We present extensions of two state-of-the-art algorithms, CBS and PBS, in order to tackle tMAPF, and demonstrate that they can consistently outperform the best solution possible under a static-obstacle setting.
Accelerating Integrated Task and Motion Planning with Neural Feasibility Checking
Venue: arXiv cs.RO 2022
Year: 2022

As robots play an increasingly important role in the industrial, the expectations about their applications for everyday living tasks are getting higher. Robots need to perform long-horizon tasks that consist of several sub-tasks that need to be accomplished. Task and Motion Planning (TAMP) provides a hierarchical framework to handle the sequential nature of manipulation tasks by interleaving a symbolic task planner that generates a possible action sequence, with a motion planner that checks the kinematic feasibility in the geometric world, generating robot trajectories if several constraints are satisfied, e.g., a collision-free trajectory from one state to another. Hence, the reasoning about the task plan's geometric grounding is taken over by the motion planner. However, motion planning is computationally intense and is usability as feasibility checker casts TAMP methods inapplicable to real-world scenarios. In this paper, we introduce neural feasibility classifier (NFC), a simple yet effective visual heuristic for classifying the feasibility of proposed actions in TAMP. Namely, NFC will identify infeasible actions of the task planner without the need for costly motion planning, hence reducing planning time in multi-step manipulation tasks. NFC encodes the image of the robot's workspace into a feature map thanks to convolutional neural network (CNN). We train NFC using simulated data from TAMP problems and label the instances based on IK feasibility checking. Our empirical results in different simulated manipulation tasks show that our NFC generalizes to the entire robot workspace and has high prediction accuracy even in scenes with multiple obstructions. When combined with state-of-the-art integrated TAMP, our NFC enhances its performance while reducing its planning time.
Towards Large-Scale Interpretable Knowledge Graph Reasoning for Dialogue Systems
Venue: arXiv cs.CL 2022
Year: 2022

Users interacting with voice assistants today need to phrase their requests in a very specific manner to elicit an appropriate response. This limits the user experience, and is partly due to the lack of reasoning capabilities of dialogue platforms and the hand-crafted rules that require extensive labor. One possible way to improve user experience and relieve the manual efforts of designers is to build an end-to-end dialogue system that can do reasoning itself while perceiving user's utterances. In this work, we propose a novel method to incorporate the knowledge reasoning capability into dialogue systems in a more scalable and generalizable manner. Our proposed method allows a single transformer model to directly walk on a large-scale knowledge graph to generate responses. To the best of our knowledge, this is the first work to have transformer models generate responses by reasoning over differentiable knowledge graphs. We investigate the reasoning abilities of the proposed method on both task-oriented and domain-specific chit-chat dialogues. Empirical results show that this method can effectively and efficiently incorporate a knowledge graph into a dialogue system with fully-interpretable reasoning paths.
Differentiable Reasoning over Long Stories -- Assessing Systematic Generalisation in Neural Models
Venue: arXiv cs.CL 2022
Year: 2022

Contemporary neural networks have achieved a series of developments and successes in many aspects; however, when exposed to data outside the training distribution, they may fail to predict correct answers. In this work, we were concerned about this generalisation issue and thus analysed a broad set of models systematically and robustly over long stories. Related experiments were conducted based on the CLUTRR, which is a diagnostic benchmark suite that can analyse generalisation of natural language understanding (NLU) systems by training over small story graphs and testing on larger ones. In order to handle the multi-relational story graph, we consider two classes of neural models: "E-GNN", the graph-based models that can process graph-structured data and consider the edge attributes simultaneously; and "L-Graph", the sequence-based models which can process linearized version of the graphs. We performed an extensive empirical evaluation, and we found that the modified recurrent neural network yield surprisingly accurate results across every systematic generalisation tasks which outperform the modified graph neural network, while the latter produced more robust models.
Calibration of Machine Reading Systems at Scale
Venue: arXiv cs.CL 2022
Year: 2022

In typical machine learning systems, an estimate of the probability of the prediction is used to assess the system's confidence in the prediction. This confidence measure is usually uncalibrated; i.e.\ the system's confidence in the prediction does not match the true probability of the predicted output. In this paper, we present an investigation into calibrating open setting machine reading systems such as open-domain question answering and claim verification systems. We show that calibrating such complex systems which contain discrete retrieval and deep reading components is challenging and current calibration techniques fail to scale to these settings. We propose simple extensions to existing calibration approaches that allows us to adapt them to these settings. Our experimental results reveal that the approach works well, and can be useful to selectively predict answers when question answering systems are posed with unanswerable or out-of-the-training distribution questions.
Red Domination in Perfect Elimination Bipartite Graphs
Venue: arXiv cs.DS 2022
Year: 2022

The $k$ red domination problem for a bipartite graph $G=(X,Y,E)$ is to find a subset $D \subseteq X$ of cardinality at most $k$ that dominates vertices of $Y$. The decision version of this problem is NP-complete for general bipartite graphs but solvable in polynomial time for chordal bipartite graphs. We strengthen that result by showing that it is NP-complete for perfect elimination bipartite graphs. We present a tight upper bound on the number of such sets in bipartite graphs, and show that we can calculate that number in linear time for convex bipartite graphs. We present a linear space linear delay enumeration algorithm that needs only linear preprocessing time.
5G-Enabled Pseudonymity for Cooperative Intelligent Transportation System
Venue: arXiv cs.NI 2022
Year: 2022

Cooperative Intelligent Transportation Systems (C-ITS) enable communications between vehicles, road-side infrastructures, and road-users to improve users' safety and to efficiently manage traffic. Most, if not all, of the intelligent vehicles-to-everything (V2X) applications, often rely on continuous collection and sharing of sensitive information such as detailed location information which raises privacy concerns. In this light, a common approach to concealing the long-term identity of C-ITS vehicles is using multiple temporary identifiers, called pseudonyms. However, the legacy pseudonyms management approach is prone to linking attacks. The introduction of 5G network to V2X offers enhanced location accuracy, better clock synchronisation, improved modular service-based architecture, and enhanced security and privacy preservation controls. Motivated by the above enhancements, we study 5G-enabled pseudonyms for protecting vehicle identity privacy in C-ITS. We highlight the gaps in the current standards of pseudonyms management. We further provide recommendations regarding the pseudonyms management life-cycle.
A Policy Driven AI-Assisted PoW Framework
Venue: arXiv cs.CR 2022
Year: 2022

Proof of Work (PoW) based cyberdefense systems require incoming network requests to expend effort solving an arbitrary mathematical puzzle. Current state of the art is unable to differentiate between trustworthy and untrustworthy connections, requiring all to solve complex puzzles. In this paper, we introduce an Artificial Intelligence (AI)-assisted PoW framework that utilizes IP traffic based features to inform an adaptive issuer which can then generate puzzles with varying hardness. The modular framework uses these capabilities to ensure that untrustworthy clients solve harder puzzles thereby incurring longer latency than authentic requests to receive a response from the server. Our preliminary findings reveal our approach effectively throttles untrustworthy traffic.
Programming Language Agnostic Mining of Code and Language Pairs with Sequence Labeling Based Question Answering
Venue: arXiv cs.CL 2022
Year: 2022

Mining aligned natural language (NL) and programming language (PL) pairs is a critical task to NL-PL understanding. Existing methods applied specialized hand-crafted features or separately-trained models for each PL. However, they usually suffered from low transferability across multiple PLs, especially for niche PLs with less annotated data. Fortunately, a Stack Overflow answer post is essentially a sequence of text and code blocks and its global textual context can provide PL-agnostic supplementary information. In this paper, we propose a Sequence Labeling based Question Answering (SLQA) method to mine NL-PL pairs in a PL-agnostic manner. In particular, we propose to apply the BIO tagging scheme instead of the conventional binary scheme to mine the code solutions which are often composed of multiple blocks of a post. Experiments on current single-PL single-block benchmarks and a manually-labeled cross-PL multi-block benchmark prove the effectiveness and transferability of SLQA. We further present a parallel NL-PL corpus named Lang2Code automatically mined with SLQA, which contains about 1.4M pairs on 6 PLs. Under statistical analysis and downstream evaluation, we demonstrate that Lang2Code is a large-scale high-quality data resource for further NL-PL research.
On Rangasamy's outsourcing algorithm for solving quadratic congruence equations
Venue: arXiv cs.CR 2022
Year: 2022

Outsourcing computation is a desired approach for IoT (Internet of Things) devices to transfer their burdens of heavy computations to those nearby, resource-abundant cloud servers. Recently, Rangasamy presented a passive attack against two outsourcing algorithms proposed by Zhang et al. for solving quadratic congruence equations, which is widely used in IoT applications. Furthermore, he also proposed a modified algorithm to fix these schemes and claimed that his algorithm was correct and enabled secure and verifiable delegation of solving quadratic congruence equations in IoTs. However, we show that Rangasamy's modified algorithm has a flaw which makes it incorrect and also propose some further attacks to break the security claim, even when the flaw has been corrected.
Multi-class Label Noise Learning via Loss Decomposition and Centroid Estimation
Venue: arXiv cs.LG 2022
Year: 2022

In real-world scenarios, many large-scale datasets often contain inaccurate labels, i.e., noisy labels, which may confuse model training and lead to performance degradation. To overcome this issue, Label Noise Learning (LNL) has recently attracted much attention, and various methods have been proposed to design an unbiased risk estimator to the noise-free dataset to combat such label noise. Among them, a trend of works based on Loss Decomposition and Centroid Estimation (LDCE) has shown very promising performance. However, existing LNL methods based on LDCE are only designed for binary classification, and they are not directly extendable to multi-class situations. In this paper, we propose a novel multi-class robust learning method for LDCE, which is termed "MC-LDCE". Specifically, we decompose the commonly adopted loss (e.g., mean squared loss) function into a label-dependent part and a label-independent part, in which only the former is influenced by label noise. Further, by defining a new form of data centroid, we transform the recovery problem of a label-dependent part to a centroid estimation problem. Finally, by critically examining the mathematical expectation of clean data centroid given the observed noisy set, the centroid can be estimated which helps to build an unbiased risk estimator for multi-class learning. The proposed MC-LDCE method is general and applicable to different types (i.e., linear and nonlinear) of classification models. The experimental results on five public datasets demonstrate the superiority of the proposed MC-LDCE against other representative LNL methods in tackling multi-class label noise problem.
Coresets for Weight-Constrained Anisotropic Assignment and Clustering
Venue: arXiv cs.DS 2022
Year: 2022

The present paper constructs coresets for weight-constrained anisotropic assignment and clustering. In contrast to the well-studied unconstrained least-squares clustering problem, approximating the centroids of the clusters no longer suffices in the weight-constrained anisotropic case, as even the assignment of the points to best sites is involved. This assignment step is often the limiting factor in materials science, a problem that partially motivates our work. We build on a paper by Har-Peled and Kushal, who constructed coresets of size $\mathcal{O}\bigl(\frac{k^3}{ε^{d+1}}\bigr)$ for unconstrained least-squares clustering. We generalize and improve on their results in various ways, leading to even smaller coresets with a size of only $\mathcal{O}\bigl(\frac{k^2}{ε^{d+1}}\bigr)$ for weight-constrained anisotropic clustering. Moreover, we answer an open question on coreset designs in the negative, by showing that the total sensitivity can become as large as the cardinality of the original data set in the constrained case. Consequently, many techniques based on importance sampling do not apply to weight-constrained clustering.
Follow the Leader: Technical and Inspirational Leadership in Open Source Software
Venue: arXiv cs.HC 2022
Year: 2022

We conduct the first comprehensive study of the behavioral factors which predict leader emergence within open source software (OSS) virtual teams. We leverage the full history of developers' interactions with their teammates and projects at github.com between January 2010 and April 2017 (representing about 133 million interactions) to establish that - contrary to a common narrative describing open source as a pure "technical meritocracy" - developers' communication abilities and community building skills are significant predictors of whether they emerge as team leaders. Inspirational communication therefore appears as central to the process of leader emergence in virtual teams, even in a setting like OSS, where technical contributions have often been conceptualized as the sole pathway to gaining community recognition. Those results should be of interest to researchers and practitioners theorizing about OSS in particular and, more generally, leadership in geographically dispersed virtual teams, as well as to online community managers.
Efficient Algorithms for Extreme Bandits
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we contribute to the Extreme Bandit problem, a variant of Multi-Armed Bandits in which the learner seeks to collect the largest possible reward. We first study the concentration of the maximum of i.i.d random variables under mild assumptions on the tail of the rewards distributions. This analysis motivates the introduction of Quantile of Maxima (QoMax). The properties of QoMax are sufficient to build an Explore-Then-Commit (ETC) strategy, QoMax-ETC, achieving strong asymptotic guarantees despite its simplicity. We then propose and analyze a more adaptive, anytime algorithm, QoMax-SDA, which combines QoMax with a subsampling method recently introduced by Baudry et al. (2021). Both algorithms are more efficient than existing approaches in two aspects (1) they lead to better empirical performance (2) they enjoy a significant reduction of the memory and time complexities.
Fully-probabilistic Terrain Modelling with Stochastic Variational Gaussian Process Maps
Venue: arXiv cs.RO 2022
Year: 2022

Gaussian processes (GPs) are becoming a standard tool to build terrain representations thanks to their capacity to model map uncertainty. This effectively yields a reliability measure of the areas of the map, which can be directly utilized by Bayes filtering algorithms in robot localization problems. A key insight is that this uncertainty can incorporate the noise intrinsic to the terrain surveying process through the GPs ability to train on uncertain inputs (UIs). However, existing techniques to build GP maps with UIs in a tractable manner are restricted in the form and degree of the input noise. In this letter, we propose a flexible and efficient framework to build large-scale GP maps with UIs based on Stochastic Variational GPs and Monte Carlo sampling of the UIs distributions. We validate our mapping approach on a large bathymetric survey collected with an AUV and analyze its performance against the use of deterministic inputs (DI). Finally, we show how using UI SVGP maps yields more accurate particle filter localization results than DI SVGP on a real AUV mission over an entirely predicted area.
TinyMLOps: Operational Challenges for Widespread Edge AI Adoption
Venue: arXiv cs.LG 2022
Year: 2022

Deploying machine learning applications on edge devices can bring clear benefits such as improved reliability, latency and privacy but it also introduces its own set of challenges. Most works focus on the limited computational resources of edge platforms but this is not the only bottleneck standing in the way of widespread adoption. In this paper we list several other challenges that a TinyML practitioner might need to consider when operationalizing an application on edge devices. We focus on tasks such as monitoring and managing the application, common functionality for a MLOps platform, and show how they are complicated by the distributed nature of edge deployment. We also discuss issues that are unique to edge applications such as protecting a model's intellectual property and verifying its integrity.
Spreadsheet computing with Finite Domain Constraint Enhancements
Venue: arXiv cs.AI 2022
Year: 2022

Spreadsheet computing is one of the more popular computing methodologies in today's modern society. The spreadsheet application's ease of use and usefulness has enabled non-programmers to perform programming-like tasks in a familiar setting modeled after the tabular "pen and paper" approach. However, spreadsheet applications are limited to bookkeeping-like tasks due to their single-direction data flow. This thesis demonstrates an extension of the spreadsheet computing paradigm in overcoming this limitation to solve constraint satisfaction problems. We present a framework seamlessly incorporating a finite constraint solver with the spreadsheet computing paradigm. This framework allows the individual cells in the spreadsheet to be attached to either a finite domain or a constraint specifying the relationship among the cells. The framework provides an interface for constraint solving and further enhances the spreadsheet computing paradigm by providing a set of spreadsheet-specific constraints that will aid in controlling the scalability of large spreadsheet applications implementations. Finally, we provide examples to demonstrate the usability and usefulness of the extended spreadsheet paradigm. Keywords: Spreadsheet computing, Constraint Logic Programming, Constraint satisfaction, Domain-Specific language, Excel, SWI Prolog, C#
Spoofing-Aware Speaker Verification with Unsupervised Domain Adaptation
Venue: arXiv cs.SD 2022
Year: 2022

In this paper, we initiate the concern of enhancing the spoofing robustness of the automatic speaker verification (ASV) system, without the primary presence of a separate countermeasure module. We start from the standard ASV framework of the ASVspoof 2019 baseline and approach the problem from the back-end classifier based on probabilistic linear discriminant analysis. We employ three unsupervised domain adaptation techniques to optimize the back-end using the audio data in the training partition of the ASVspoof 2019 dataset. We demonstrate notable improvements on both logical and physical access scenarios, especially on the latter where the system is attacked by replayed audios, with a maximum of 36.1% and 5.3% relative improvement on bonafide and spoofed cases, respectively. We perform additional studies such as per-attack breakdown analysis, data composition, and integration with a countermeasure system at score-level with Gaussian back-end.