OpenRaiser/Intern-Atlas huggingface dataset:
Architecting Intermediate Layers for Efficient Composition of Data Management and Machine Learning Systems
Venue: arXiv cs.PL 2023
Year: 2023

Modern data analytics workloads combine relational data processing with machine learning (ML). Most DBMS handle these workloads by offloading these ML operations to external specialized ML systems. While both DBMS and ML systems go to great lengths to optimize performance for their specific workloads, significant performance is lost when used in combination, due to data movement across system boundaries, conversions between incompatible internal data formats, and the lack of cross system optimizations. A key idea to remove these bottlenecks is to integrate existing data manipulation systems with ML systems by building a common intermediate layer (IR). Although this idea has been explored before (Weld, Delite), previous such attempts require significant re-engineering of prior systems and still fall short in achieving best-of-breed performance for individual tasks (e.g., SQL, Deep Learning). Specifically, they rely on re-implementing existing systems using a generic set of operators and fail to match best-of-breed individual performance due to the inability to recover high-level optimizations from this generic IR through compiler analysis. We present Flern, the first intermediate-layer integration between DB and ML systems that are best-of-breed individually, competitive with the best compiled query engines such as HyPer on comprehensive relational benchmarks (TPC-H) and competitive with TensorFlow and PyTorch in state-of-the-art ML models (e.g., DeepSpeech, SqueezeNet, Transformers) and also represents a new state-of-the-art for integration. A key realization is to architect intermediate layers based on generative programming capabilities, which preserves high-level contextual information for cross optimizations and enables the construction of a variety of complex structures and cross system optimizations with minimal effort.
Fast and Interpretable Face Identification for Out-Of-Distribution Data Using Vision Transformers
Venue: arXiv cs.CV 2023
Year: 2023

Most face identification approaches employ a Siamese neural network to compare two images at the image embedding level. Yet, this technique can be subject to occlusion (e.g. faces with masks or sunglasses) and out-of-distribution data. DeepFace-EMD (Phan et al. 2022) reaches state-of-the-art accuracy on out-of-distribution data by first comparing two images at the image level, and then at the patch level. Yet, its later patch-wise re-ranking stage admits a large $O(n^3 \log n)$ time complexity (for $n$ patches in an image) due to the optimal transport optimization. In this paper, we propose a novel, 2-image Vision Transformers (ViTs) that compares two images at the patch level using cross-attention. After training on 2M pairs of images on CASIA Webface (Yi et al. 2014), our model performs at a comparable accuracy as DeepFace-EMD on out-of-distribution data, yet at an inference speed more than twice as fast as DeepFace-EMD (Phan et al. 2022). In addition, via a human study, our model shows promising explainability through the visualization of cross-attention. We believe our work can inspire more explorations in using ViTs for face identification.
Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning
Venue: arXiv cs.CV 2023
Year: 2023

The goal of 2D human pose estimation (HPE) is to localize anatomical landmarks, given an image of a person in a pose. SOTA techniques make use of thousands of labeled figures (finetuning transformers or training deep CNNs), acquired using labor-intensive crowdsourcing. On the other hand, self-supervised methods re-frame the HPE task as a reconstruction problem, enabling them to leverage the vast amount of unlabeled visual data, though at the present cost of accuracy. In this work, we explore ways to improve self-supervised HPE. We (1) analyze the relationship between reconstruction quality and pose estimation accuracy, (2) develop a model pipeline that outperforms the baseline which inspired our work, using less than one-third the amount of training data, and (3) offer a new metric suitable for self-supervised settings that measures the consistency of predicted body part length proportions. We show that a combination of well-engineered reconstruction losses and inductive priors can help coordinate pose learning alongside reconstruction in a self-supervised paradigm.
SemanticTopoLoop: Semantic Loop Closure With 3D Topological Graph Based on Quadric-Level Object Map
Venue: arXiv cs.CV 2023
Year: 2023

Loop closure, as one of the crucial components in SLAM, plays an essential role in correcting the accumulated errors. Traditional appearance-based methods, such as bag-of-words models, are often limited by local 2D features and the volume of training data, making them less versatile and robust in real-world scenarios, leading to missed detections or false positives detections in loop closure. To address these issues, we first propose a object-level data association method based on multi-level verification, which can associate 2D semantic features of current frame with 3D objects landmarks of map. Next, taking advantage of these association relations, we introduce a semantic loop closure method based on quadric-level object map topology, which represents scenes through the topological graph of objects and achieves accurate loop closure at a wide field of view by comparing differences in the topological graphs. Finally, we integrate these two methods into a complete object-aware SLAM system. Qualitative experiments and ablation studies demonstrate the effectiveness and robustness of the proposed object-level data association algorithm. Quantitative experiments show that our semantic loop closure method outperforms existing state-of-the-art methods in terms of precision, recall and localization accuracy metrics.
Flexible Multi-Generator Model with Fused Spatiotemporal Graph for Trajectory Prediction
Venue: arXiv cs.CV 2023
Year: 2023

Trajectory prediction plays a vital role in automotive radar systems, facilitating precise tracking and decision-making in autonomous driving. Generative adversarial networks with the ability to learn a distribution over future trajectories tend to predict out-of-distribution samples, which typically occurs when the distribution of forthcoming paths comprises a blend of various manifolds that may be disconnected. To address this issue, we propose a trajectory prediction framework, which can capture the social interaction variations and model disconnected manifolds of pedestrian trajectories. Our framework is based on a fused spatiotemporal graph to better model the complex interactions of pedestrians in a scene, and a multi-generator architecture that incorporates a flexible generator selector network on generated trajectories to learn a distribution over multiple generators. We show that our framework achieves state-of-the-art performance compared with several baselines on different challenging datasets.
Cell-Probe Lower Bound for Accessible Interval Graphs
Venue: arXiv cs.DS 2023
Year: 2023

We spot a hole in the area of succinct data structures for graph classes from a universe of size at most $n^n$. Very often, the input graph is labeled by the user in an arbitrary and easy-to-use way, and the data structure for the graph relabels the input graph in some way. For any access, the user needs to store these labels or compute the new labels in an online manner. This might require more bits than the information-theoretic minimum of the original graph class, hence, defeating the purpose of succinctness. Given this, the data structure designer must allow the user to access the data structure with the original labels, i.e., relabeling is not allowed. We call such a graph data structure ``accessible''. In this paper, we study the complexity of such accessible data structures for interval graphs, a graph class with information-theoretic minimum less than $n\log n$ bits. - We formalize the concept of "accessibility" (which was implicitly assumed), and propose the "universal interval representation", for interval graphs. - Any data structure for interval graphs in universal interval representation, which supports both adjacency and degree query simultaneously with time cost $t_1$ and $t_2$ respectively, must consume at least $\log_2(n!)+n/(\log n)^{O(t_1+t_2)}$ bits of space. This is also the first lower bound for graph classes with information-theoretic minimum less than $n\log_2n$ bits. - We provide efficient succinct data structures for interval graphs in universal interval representation supporting adjacency query and degree query individually in constant time and space costs. Therefore, two upper bounds together with the lower bound show that the two elementary queries for interval graphs are incompatible with each other in the context of succinct data structure. To the best of our knowledge, this is the first proof of such incompatibility phenomenon.
Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, we present Consistent4D, a novel approach for generating 4D dynamic objects from uncalibrated monocular videos. Uniquely, we cast the 360-degree dynamic object reconstruction as a 4D generation problem, eliminating the need for tedious multi-view data collection and camera calibration. This is achieved by leveraging the object-level 3D-aware image diffusion model as the primary supervision signal for training Dynamic Neural Radiance Fields (DyNeRF). Specifically, we propose a Cascade DyNeRF to facilitate stable convergence and temporal continuity under the supervision signal which is discrete along the time axis. To achieve spatial and temporal consistency, we further introduce an Interpolation-driven Consistency Loss. It is optimized by minimizing the discrepancy between rendered frames from DyNeRF and interpolated frames from a pre-trained video interpolation model. Extensive experiments show that our Consistent4D can perform competitively to prior art alternatives, opening up new possibilities for 4D dynamic object generation from monocular videos, whilst also demonstrating advantage for conventional text-to-3D generation tasks. Our project page is https://consistent4d.github.io/.
AdaFlood: Adaptive Flood Regularization
Venue: arXiv cs.LG 2023
Year: 2023

Although neural networks are conventionally optimized towards zero training loss, it has been recently learned that targeting a non-zero training loss threshold, referred to as a flood level, often enables better test time generalization. Current approaches, however, apply the same constant flood level to all training samples, which inherently assumes all the samples have the same difficulty. We present AdaFlood, a novel flood regularization method that adapts the flood level of each training sample according to the difficulty of the sample. Intuitively, since training samples are not equal in difficulty, the target training loss should be conditioned on the instance. Experiments on datasets covering four diverse input modalities - text, images, asynchronous event sequences, and tabular - demonstrate the versatility of AdaFlood across data domains and noise levels.
Edge2Node: Reducing Edge Prediction to Node Classification
Venue: arXiv cs.LG 2023
Year: 2023

Despite the success of graph neural network models in node classification, edge prediction (the task of predicting missing or potential links between nodes in a graph) remains a challenging problem for these models. A common approach for edge prediction is to first obtain the embeddings of two nodes, and then a predefined scoring function is used to predict the existence of an edge between the two nodes. Here, we introduce a preliminary idea called Edge2Node which suggests to directly obtain an embedding for each edge, without the need for a scoring function. This idea wants to create a new graph H based on the graph G given for the edge prediction task, and then suggests reducing the edge prediction task on G to a node classification task on H. We anticipate that this introductory method could stimulate further investigations for edge prediction task.
Deep Image Semantic Communication Model for Artificial Intelligent Internet of Things
Venue: arXiv cs.CV 2023
Year: 2023

With the rapid development of Artificial Intelligent Internet of Things (AIoT), the image data from AIoT devices has been witnessing the explosive increasing. In this paper, a novel deep image semantic communication model is proposed for the efficient image communication in AIoT. Particularly, at the transmitter side, a high-precision image semantic segmentation algorithm is proposed to extract the semantic information of the image to achieve significant compression of the image data. At the receiver side, a semantic image restoration algorithm based on Generative Adversarial Network (GAN) is proposed to convert the semantic image to a real scene image with detailed information. Simulation results demonstrate that the proposed image semantic communication model can improve the image compression ratio and recovery accuracy by 71.93% and 25.07% on average in comparison with WebP and CycleGAN, respectively. More importantly, our demo experiment shows that the proposed model reduces the total delay by 95.26% in the image communication, when comparing with the original image transmission.
Marker-Based Localisation System Using an Active PTZ Camera and CNN-Based Ellipse Detection
Venue: arXiv cs.RO 2023
Year: 2023

Localisation in GPS-denied environments is challenging and many existing solutions have infrastructural and on-site calibration requirements. This paper tackles these challenges by proposing a localisation system that is infrastructure-free and does not require on-site calibration, using a single active PTZ camera to detect, track and localise a circular LED marker. We propose to use a CNN trained using only synthetic images to detect the LED marker as an ellipse and show that our approach is more robust than using traditional ellipse detection without requiring tuning of parameters for feature extraction. We also propose to leverage the predicted elliptical angle as a measure of uncertainty of the CNN's predictions and show how it can be used in a filter to improve marker range estimation and 3D localisation. We evaluate our system's performance through localisation of a UAV in real-world flight experiments and show that it can outperform alternative methods for localisation in GPS-denied environments. We also demonstrate our system's performance in indoor and outdoor environments.
Towards a Transformer-Based Reverse Dictionary Model for Quality Estimation of Definitions
Venue: arXiv cs.CL 2023
Year: 2023

In the last years, several variants of transformers have emerged. In this paper, we compare different transformer-based models for solving the reverse dictionary task and explore their use in the context of a serious game called The Dictionary Game.
Non Deterministic Pseudorandom Generator for Quantum Key Distribution
Venue: arXiv cs.CR 2023
Year: 2023

Quantum Key Distribution(QKD) thrives to achieve perfect secrecy of One time Pad (OTP) through quantum processes. One of the crucial components of QKD are Quantum Random Number Generators(QRNG) for generation of keys. Unfortunately, these QRNG does not immediately produce usable bits rather it produces raw bits with high entropy but low uniformity which can be hardly used by any cryptographic system. A lot of pre-processing is required before the random numbers generated by QRNG to be usable. This causes a bottle neck in random number generation rate as well as QKD system relying on it. To avoid this lacuna of post-processing methods employed as a central part of Quantum Random Number Generators alternative approaches that satisfy the entropy(non determinism) and quantum security is explored. Pseudorandom generators based on quantum secure primitives could be an alternative to the post-processing problem as PRNGs are way more faster than any random number generator employing physical randomness (quantum mechanical process in QRNG) as well as it can provide uniform bits required for cryptography application. In this work we propose a pseudorandom generator based on post quantum primitives. The central theme of this random number generator is designing PRNG with non deterministic entropy generated through hard lattice problem - Learning with errors. We leverage the non determinism by Gaussian errors of LWE to construct non-deterministic PRNG satisfying the entropy requirement of QKD. Further, the paper concludes by evaluating the PRNG through Die-Harder Test.
Reconfigurable, Transformable Soft Pneumatic Actuator with Tunable 3D Deformations for Dexterous Soft Robotics Applications
Venue: arXiv cs.RO 2023
Year: 2023

Numerous soft actuators based on PneuNet design have already been proposed and extensively employed across various soft robotics applications in recent years. Despite their widespread use, a common limitation of most existing designs is that their action is pre-determined during the fabrication process, thereby restricting the ability to modify or alter their function during operation. To address this shortcoming, in this article the design of a Reconfigurable, Transformable Soft Pneumatic Actuator (RT-SPA) is proposed. The working principle of the RT-SPA is analogous to the conventional PneuNet. The key distinction between the two lies in the ability of the RT-SPA to undergo controlled transformations, allowing for more versatile bending and twisting motions in various directions. Furthermore, the unique reconfigurable design of the RT-SPA enables the selection of actuation units with different sizes to achieve a diverse range of three-dimensional deformations. This versatility enhances the potential of the RT-SPA for adaptation to a multitude of tasks and environments, setting it apart from traditional PneuNet. The paper begins with a detailed description of the design and fabrication of the RT-SPA. Following this, a series of experiments are conducted to evaluate the performance of the RT-SPA. Finally, the abilities of the RT-SPA for locomotion, gripping, and object manipulation are demonstrated to illustrate the versatility of the RT-SPA across different aspects.
A multi-modal approach to continuous material identification through tactile sensing
Venue: arXiv cs.RO 2023
Year: 2023

Tactile sensing has recently been used in robotics for object identification, grasping, and material recognition. Most material recognition approaches use vibration information from a tactile exploration, typically above one second long, to identify the material. This work proposes a tactile multi-modal (vibration and thermal) material identification approach based on recursive Bayesian estimation. Through the frequency response of the vibration induced by the material and thermal features, like an estimate of the thermal power loss of the finger, we show that it is possible to identify materials in less than half a second. Moreover, a comparison between the use of vibration only and multi-modal identification shows that both recognition time and classification errors are reduced by adding thermal information.
TAMPAR: Visual Tampering Detection for Parcel Logistics in Postal Supply Chains
Venue: arXiv cs.CV 2023
Year: 2023

Due to the steadily rising amount of valuable goods in supply chains, tampering detection for parcels is becoming increasingly important. In this work, we focus on the use-case last-mile delivery, where only a single RGB image is taken and compared against a reference from an existing database to detect potential appearance changes that indicate tampering. We propose a tampering detection pipeline that utilizes keypoint detection to identify the eight corner points of a parcel. This permits applying a perspective transformation to create normalized fronto-parallel views for each visible parcel side surface. These viewpoint-invariant parcel side surface representations facilitate the identification of signs of tampering on parcels within the supply chain, since they reduce the problem to parcel side surface matching with pair-wise appearance change detection. Experiments with multiple classical and deep learning-based change detection approaches are performed on our newly collected TAMpering detection dataset for PARcels, called TAMPAR. We evaluate keypoint and change detection separately, as well as in a unified system for tampering detection. Our evaluation shows promising results for keypoint (Keypoint AP 75.76) and tampering detection (81% accuracy, F1-Score 0.83) on real images. Furthermore, a sensitivity analysis for tampering types, lens distortion and viewing angles is presented. Code and dataset are available at https://a-nau.github.io/tampar.
Enabling In-Situ Resources Utilisation by leveraging collaborative robotics and astronaut-robot interaction
Venue: arXiv cs.RO 2023
Year: 2023

Space exploration and establishing human presence on other planets demand advanced technology and effective collaboration between robots and astronauts. Efficient space resource utilization is also vital for extraterrestrial settlements. The Collaborative In-Situ Resources Utilisation (CISRU) project has developed a software suite comprising five key modules. The first module manages multi-agent autonomy, facilitating communication between agents and mission control. The second focuses on environment perception, employing AI algorithms for tasks like environment segmentation and object pose estimation. The third module ensures safe navigation, covering obstacle avoidance, social navigation with astronauts, and cooperation among robots. The fourth module addresses manipulation functions, including multi-tool capabilities and tool-changer design for diverse tasks in In-Situ Resources Utilization (ISRU) scenarios. Finally, the fifth module controls cooperative behaviour, incorporating astronaut commands, Mixed Reality interfaces, map fusion, task supervision, and error control. The suite was tested using an astronaut-rover interaction dataset in a planetary environment and GMV SPoT analogue environments. Results demonstrate the advantages of E4 autonomy and AI in space systems, benefiting astronaut-robot collaboration. This paper details CISRU's development, field test preparation, and analysis, highlighting its potential to revolutionize planetary exploration through AI-powered technology.
Discretizing Numerical Attributes: An Analysis of Human Perceptions
Venue: arXiv cs.LG 2023
Year: 2023

Machine learning (ML) has employed various discretization methods to partition numerical attributes into intervals. However, an effective discretization technique remains elusive in many ML applications, such as association rule mining. Moreover, the existing discretization techniques do not reflect best the impact of the independent numerical factor on the dependent numerical target factor. This research aims to establish a benchmark approach for numerical attribute partitioning. We conduct an extensive analysis of human perceptions of partitioning a numerical attribute and compare these perceptions with the results obtained from our two proposed measures. We also examine the perceptions of experts in data science, statistics, and engineering by employing numerical data visualization techniques. The analysis of collected responses reveals that $68.7\%$ of human responses approximately closely align with the values generated by our proposed measures. Based on these findings, our proposed measures may be used as one of the methods for discretizing the numerical attributes.