OpenRaiser/Intern-Atlas huggingface dataset:
An Empirical Evaluation of Various Information Gain Criteria for Active Tactile Action Selection for Pose Estimation
Venue: arXiv cs.RO 2022
Year: 2022

Accurate object pose estimation using multi-modal perception such as visual and tactile sensing have been used for autonomous robotic manipulators in literature. Due to variation in density of visual and tactile data, we previously proposed a novel probabilistic Bayesian filter-based approach termed translation-invariant Quaternion filter (TIQF) for pose estimation. As tactile data collection is time consuming, active tactile data collection is preferred by reasoning over multiple potential actions for maximal expected information gain. In this paper, we empirically evaluate various information gain criteria for action selection in the context of object pose estimation. We demonstrate the adaptability and effectiveness of our proposed TIQF pose estimation approach with various information gain criteria. We find similar performance in terms of pose accuracy with sparse measurements across all the selected criteria.
Training Personalized Recommendation Systems from (GPU) Scratch: Look Forward not Backwards
Venue: arXiv cs.AR 2022
Year: 2022

Personalized recommendation models (RecSys) are one of the most popular machine learning workload serviced by hyperscalers. A critical challenge of training RecSys is its high memory capacity requirements, reaching hundreds of GBs to TBs of model size. In RecSys, the so-called embedding layers account for the majority of memory usage so current systems employ a hybrid CPU-GPU design to have the large CPU memory store the memory hungry embedding layers. Unfortunately, training embeddings involve several memory bandwidth intensive operations which is at odds with the slow CPU memory, causing performance overheads. Prior work proposed to cache frequently accessed embeddings inside GPU memory as means to filter down the embedding layer traffic to CPU memory, but this paper observes several limitations with such cache design. In this work, we present a fundamentally different approach in designing embedding caches for RecSys. Our proposed ScratchPipe architecture utilizes unique properties of RecSys training to develop an embedding cache that not only sees the past but also the "future" cache accesses. ScratchPipe exploits such property to guarantee that the active working set of embedding layers can "always" be captured inside our proposed cache design, enabling embedding layer training to be conducted at GPU memory speed.
Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures
Venue: arXiv cs.LG 2022
Year: 2022

With the advent of ubiquitous deployment of smart devices and the Internet of Things, data sources for machine learning inference have increasingly moved to the edge of the network. Existing machine learning inference platforms typically assume a homogeneous infrastructure and do not take into account the more complex and tiered computing infrastructure that includes edge devices, local hubs, edge datacenters, and cloud datacenters. On the other hand, recent AutoML efforts have provided viable solutions for model compression, pruning and quantization for heterogeneous environments; for a machine learning model, now we may easily find or even generate a series of models with different tradeoffs between accuracy and efficiency. We design and implement JellyBean, a system for serving and optimizing machine learning inference workflows on heterogeneous infrastructures. Given service-level objectives (e.g., throughput, accuracy), JellyBean picks the most cost-efficient models that meet the accuracy target and decides how to deploy them across different tiers of infrastructures. Evaluations show that JellyBean reduces the total serving cost of visual question answering by up to 58%, and vehicle tracking from the NVIDIA AI City Challenge by up to 36% compared with state-of-the-art model selection and worker assignment solutions. JellyBean also outperforms prior ML serving systems (e.g., Spark on the cloud) up to 5x in serving costs.
Integrating Parcel Deliveries into a Ride-Pooling Service -- An Agent-Based Simulation Study
Venue: arXiv cs.MA 2022
Year: 2022

This paper examines the integration of freight delivery into the passenger transport of an on-demand ride-pooling service. The goal of this research is to use existing passenger trips for logistics services and thus reduce additional vehicle kilometers for freight delivery and the total number of vehicles on the road network. This is achieved by merging the need for two separate fleets into a single one by combining the services. To evaluate the potential of such a mobility-on-demand service, this paper uses an agent-based simulation framework and integrates three heuristic parcel assignment strategies into a ride-pooling fleet control algorithm. Two integration scenarios (moderate and full) are set up. While in both scenarios passengers and parcels share rides in one vehicle, in the moderate scenario no stops for parcel pick-up and delivery are allowed during a passenger ride to decrease customer inconvenience. Using real-world demand data for a case study of Munich, Germany, the two integration scenarios together with the three assignment strategies are compared to the status quo, which uses two separate vehicle fleets for passenger and logistics transport. The results indicate that the integration of logistics services into a ride-pooling service is possible and can exploit unused system capacities without deteriorating passenger transport. Depending on the assignment strategies nearly all parcels can be served until a parcel to passenger demand ratio of 1:10 while the overall fleet kilometers can be deceased compared to the status quo.
Weakly-supervised segmentation of referring expressions
Venue: arXiv cs.CV 2022
Year: 2022

Visual grounding localizes regions (boxes or segments) in the image corresponding to given referring expressions. In this work we address image segmentation from referring expressions, a problem that has so far only been addressed in a fully-supervised setting. A fully-supervised setup, however, requires pixel-wise supervision and is hard to scale given the expense of manual annotation. We therefore introduce a new task of weakly-supervised image segmentation from referring expressions and propose Text grounded semantic SEGgmentation (TSEG) that learns segmentation masks directly from image-level referring expressions without pixel-level annotations. Our transformer-based method computes patch-text similarities and guides the classification objective during training with a new multi-label patch assignment mechanism. The resulting visual grounding model segments image regions corresponding to given natural language expressions. Our approach TSEG demonstrates promising results for weakly-supervised referring expression segmentation on the challenging PhraseCut and RefCOCO datasets. TSEG also shows competitive performance when evaluated in a zero-shot setting for semantic segmentation on Pascal VOC.
AI training resources for GLAM: a snapshot
Venue: arXiv cs.LG 2022
Year: 2022

We take a snapshot of current resources available for teaching and learning AI with a focus on the Galleries, Libraries, Archives and Museums (GLAM) community. The review was carried out in 2021 and 2022. The review provides an overview of material we identified as being relevant, offers a description of this material and makes recommendations for future work in this area.
Reliable Monte Carlo Localization for Mobile Robots
Venue: arXiv cs.RO 2022
Year: 2022

Reliability is a key factor for realizing safety guarantee of full autonomous robot systems. In this paper, we focus on reliability in mobile robot localization. Monte Carlo localization (MCL) is widely used for mobile robot localization. However, it is still difficult to guarantee its safety because there are no methods determining reliability for MCL estimate. This paper presents a novel localization framework that enables robust localization, reliability estimation, and quick re-localization, simultaneously. The presented method can be implemented using similar estimation manner to that of MCL. The method can increase localization robustness to environment changes by estimating known and unknown obstacles while performing localization; however, localization failure of course occurs by unanticipated errors. The method also includes a reliability estimation function that enables us to know whether localization has failed. Additionally, the method can seamlessly integrate a global localization method via importance sampling. Consequently, quick re-localization from failures can be realized while mitigating noisy influence of global localization. Through three types of experiments, we show that reliable MCL that performs robust localization, self-failure detection, and quick failure recovery can be realized.
Upper Bounds to Genome Rearrangement Problem using Prefix Transpositions
Venue: arXiv cs.DM 2022
Year: 2022

A Genome rearrangement problem studies large-scale mutations on a set of DNAs in living organisms. Various rearrangements like reversals, transpositions, translocations, fissions, fusions, and combinations and different variations have been studied extensively by computational biologists and computer scientists over the past four decades. From a mathematical point of view, a genome is represented by a permutation. The genome rearrangement problem is interpreted as a problem that transforms one permutation into another in a minimum number of moves under certain constraints depending on the chosen rearrangements. Finding the minimum number of moves is equivalent to sorting the permutation with the given rearrangement. A transposition is an operation on a permutation that moves a sublist of a permutation to a different position in the same permutation. A \emph{Prefix Transposition}, as the name suggests, is a transposition that moves a sublist which is a prefix of the permutation. In this thesis, we study prefix transpositions on permutations and present a better upper bound for sorting permutations with prefix transpositions. A greedy algorithm called the \emph{generalised sequence length algorithm} is defined as an extension of the sequence length algorithm where suitable alternate moves are also considered. This algorithm is used to sequentially improve the upper bound to $n-\log_{3.3} n$ and $n-\log_3 n$. In the latter part of the thesis, we defined the concept of a \emph{block}. We used it along with the greedy moves of the generalised sequence length algorithm to get an upper bound of $n-\log_2 n$ to sort permutations by prefix transpositions.
Fixed-Template Promise Model Checking Problems
Venue: arXiv cs.CC 2022
Year: 2022

The fixed-template constraint satisfaction problem (CSP) can be seen as the problem of deciding whether a given primitive positive first-order sentence is true in a fixed structure (also called model). We study a class of problems that generalizes the CSP simultaneously in two directions: we fix a set $\mathcal{L}$ of quantifiers and Boolean connectives, and we specify two versions of each constraint, one strong and one weak. Given a sentence which only uses symbols from $\mathcal{L}$, the task is to distinguish whether the sentence is true in the strong sense, or it is false even in the weak sense. We classify the computational complexity of these problems for the existential positive equality-free fragment of first-order logic, i.e., $\mathcal{L} = \{\exists,\land,\lor\}$, and we prove some upper and lower bounds for the positive equality-free fragment, $\mathcal{L} = \{\exists,\forall,\land,\lor\}$. The partial results are sufficient, e.g., for all extensions of the latter fragment.
A Specification Logic for Programs in the Probabilistic Guarded Command Language (Extended Version)
Venue: arXiv cs.LO 2022
Year: 2022

The semantics of probabilistic languages has been extensively studied, but specification languages for their properties have received little attention. This paper introduces the probabilistic dynamic logic pDL, a specification logic for programs in the probabilistic guarded command language (pGCL) of McIver and Morgan. The proposed logic pDL can express both first-order state properties and probabilistic reachability properties, addressing both the non-deterministic and probabilistic choice operators of pGCL. In order to precisely explain the meaning of specifications, we formally define the satisfaction relation for pDL. Since pDL embeds pGCL programs in its box-modality operator, pDL satisfiability builds on a formal MDP semantics for pGCL programs. The satisfaction relation is modeled after PCTL, but extended from propositional to first-order setting of dynamic logic, and also embedding program fragments. We study basic properties of pDL, such as weakening and distribution, that can support reasoning systems. Finally, we demonstrate the use of pDL to reason about program behavior.
Political Propagation of Social Botnets: Policy Consequences
Venue: arXiv cs.CY 2022
Year: 2022

The 2016 US election was a watershed event where an electoral intervention by an adversarial state made extensive use of networks of software robots and data driven communications which transformed the interference into a goal driven functionality of man-machine collaboration. Reviewing the debates post the debacle, we reflect upon the policy consequences of the use of Social Botnets and understand the impact of their adversarial operation in terms of catalysing institutional decay, growing infrastructural anxieties, increased industry regulations, more vulnerable Individuals and more distorted ideas, and most importantly, the emergence of an unintended constituency in form of the bot agency itself. The article first briefly introduces the nature and evolution of Social Botnets, and then moves over to discussing the policy consequences. For future work, it is important to understand the agency and collective properties of these software robots, in order to design the institutional and socio-technical mechanisms which mitigate the risk of adversarial social engineering using these bots from interfering into democratic processes.
Envelopes and Waves: Safe Multivehicle Collision Avoidance for Horizontal Non-deterministic Turns
Venue: arXiv cs.LO 2022
Year: 2022

We present an approach to analyzing the safety of asynchronous, independent, non-deterministic, turn-to-bearing horizontal maneuvers for two vehicles. Future turn rates, final bearings, and continuously varying ground speeds throughout the encounter are unknown but restricted to known ranges. We develop a library of formal proofs about turning kinematics, and apply the library to create a formally verified timing computation. Additionally, we create a technique that evaluates future collision possibilities that is based on waves of position possibilities and relies on the timing computation. The result either determines that the encounter will be collision-free, or computes a safe overapproximation for when and where collisions may occur.
Object Detection in Indian Food Platters using Transfer Learning with YOLOv4
Venue: arXiv cs.CV 2022
Year: 2022

Object detection is a well-known problem in computer vision. Despite this, its usage and pervasiveness in the traditional Indian food dishes has been limited. Particularly, recognizing Indian food dishes present in a single photo is challenging due to three reasons: 1. Lack of annotated Indian food datasets 2. Non-distinct boundaries between the dishes 3. High intra-class variation. We solve these issues by providing a comprehensively labelled Indian food dataset- IndianFood10, which contains 10 food classes that appear frequently in a staple Indian meal and using transfer learning with YOLOv4 object detector model. Our model is able to achieve an overall mAP score of 91.8% and f1-score of 0.90 for our 10 class dataset. We also provide an extension of our 10 class dataset- IndianFood20, which contains 10 more traditional Indian food classes.
Multi-Tree Guided Efficient Robot Motion Planning
Venue: arXiv cs.RO 2022
Year: 2022

Motion Planning is necessary for robots to complete different tasks. Rapidly-exploring Random Tree (RRT) and its variants have been widely used in robot motion planning due to their fast search in state space. However, they perform not well in many complex environments since the motion planning needs to simultaneously consider the geometry constraints and differential constraints. In this article, we propose a novel robot motion planning algorithm that utilizes multi-tree to guide the exploration and exploitation. The proposed algorithm maintains more than two trees to search the state space at first. Each tree will explore the local environment. The tree starts from the root will gradually collect information from other trees and grow towards the goal state. This simultaneous exploration and exploitation method can quickly find a feasible trajectory. We compare the proposed algorithm with other popular motion planning algorithms. The experiment results demonstrate that our algorithm achieves the best performance on different evaluation metrics.
Scaling-up Generalized Planning as Heuristic Search with Landmarks
Venue: arXiv cs.AI 2022
Year: 2022

Landmarks are one of the most effective search heuristics for classical planning, but largely ignored in generalized planning. Generalized planning (GP) is usually addressed as a combinatorial search in a given space of algorithmic solutions, where candidate solutions are evaluated w.r.t.~the instances they solve. This type of solution evaluation ignores any sub-goal information that is not explicit in the representation of the planning instances, causing plateaus in the space of candidate generalized plans. Furthermore, node expansion in GP is a run-time bottleneck since it requires evaluating every child node over the entire batch of classical planning instances in a GP problem. In this paper we define a landmark counting heuristic for GP (that considers sub-goal information that is not explicitly represented in the planning instances), and a novel heuristic search algorithm for GP (that we call PGP) and that progressively processes subsets of the planning instances of a GP problem. Our two orthogonal contributions are analyzed in an ablation study, showing that both improve the state-of-the-art in GP as heuristic search, and that both benefit from each other when used in combination.
From Trade-only to Zero-Value NFTs: The Asset Proxy NFT Paradigm in Web3
Venue: arXiv cs.SE 2022
Year: 2022

Many implementations of smart contracts available in NFT marketplaces today allow the modification of NFT token attributes, without any specific mechanism to control the consistency with off-chain metadata. We believe this is a weakness in overall design of NFTs today. We propose a computation model called the Asset Proxy NFT that guarantees the consistency between the NFT token (on-chain) and its corresponding asset metadata (off-chain). In general, the proposed model can be applied to any type of NFT that requires immutability or controlled mutability of metadata. A second contribution of this paper is the notion of the NFT design patterns which recognizes that a coherent framework for dealing with hybrid assets is required, and that for specific hybrid-asset deployments, suitable technological components must be utilized under the framework.
Iterative models for complex networks formed by extending cliques
Venue: arXiv cs.DM 2022
Year: 2022

We consider a new model for complex networks whose underlying mechanism is extending dense subgraphs. In the frustum model, we iteratively extend cliques over discrete-time steps. For many choices of the underlying parameters, graphs generated by the model densify over time. In the special case of the cone model, generated graphs provably satisfy properties observed in real-world complex networks such as the small world property and bad spectral expansion. We finish with a set of open problems and next steps for the frustum model.
The Future of Hybrid Meetings
Venue: arXiv cs.HC 2022
Year: 2022

Meetings are typically considered to be the fuel of an organization's productivity -- a place where employees discuss ideas and make collective decisions. However, it is no secret that meetings are also often perceived as wasteful vacuums, depleting employee morale and productivity, likely due to the fact that current technologies fall short in fully supporting physical or virtual meeting experience. In this position paper, we discuss the three key elements that make a meeting successful (i.e., execution, psychological safety, and physical comfort), and present new tools for hybrid meetings that incorporate those elements. As past research has focused on supporting meeting execution (the first element), we set the roadmap for future research on the two other elements: on psychological safety by articulating how new technologies could make meeting useful for all participants, ensure all participants give and receive appropriate levels of attention, and enable all participants to feel and make others feel comfortable; and on physical comfort by dwelling on how new technologies could make the meeting experience comfortable by integrating all human senses. We also discuss the potential danger of these technologies inadvertently becoming surveillance tools.
ALLSH: Active Learning Guided by Local Sensitivity and Hardness
Venue: arXiv cs.CL 2022
Year: 2022

Active learning, which effectively collects informative unlabeled data for annotation, reduces the demand for labeled data. In this work, we propose to retrieve unlabeled samples with a local sensitivity and hardness-aware acquisition function. The proposed method generates data copies through local perturbations and selects data points whose predictive likelihoods diverge the most from their copies. We further empower our acquisition function by injecting the select-worst case perturbation. Our method achieves consistent gains over the commonly used active learning strategies in various classification tasks. Furthermore, we observe consistent improvements over the baselines on the study of prompt selection in prompt-based few-shot learning. These experiments demonstrate that our acquisition guided by local sensitivity and hardness can be effective and beneficial for many NLP tasks.
Exploring Viable Algorithmic Options for Learning from Demonstration (LfD): A Parameterized Complexity Approach
Venue: arXiv cs.LG 2022
Year: 2022

The key to reconciling the polynomial-time intractability of many machine learning tasks in the worst case with the surprising solvability of these tasks by heuristic algorithms in practice seems to be exploiting restrictions on real-world data sets. One approach to investigating such restrictions is to analyze why heuristics perform well under restrictions. A complementary approach would be to systematically determine under which sets of restrictions efficient and reliable machine learning algorithms do and do not exist. In this paper, we show how such a systematic exploration of algorithmic options can be done using parameterized complexity analysis, As an illustrative example, we give the first parameterized complexity analysis of batch and incremental policy inference under Learning from Demonstration (LfD). Relative to a basic model of LfD, we show that none of our problems can be solved efficiently either in general or relative to a number of (often simultaneous) restrictions on environments, demonstrations, and policies. We also give the first known restrictions under which efficient solvability is possible and discuss the implications of our solvability and unsolvability results for both our basic model of LfD and more complex models of LfD used in practice.