OpenRaiser/Intern-Atlas huggingface dataset:
Diagonal of Pseudoinverse of Graph Laplacian: Fast Estimation and Exact Results
Venue: arXiv cs.IT 2023
Year: 2023

The diagonal entries of pseudoinverse of the Laplacian matrix of a graph appear in many important practical applications, since they contain much information of the graph and many relevant quantities can be expressed in terms of them, such as Kirchhoff index and current flow centrality. However, a na\"ıve approach for computing the diagonal of a matrix inverse has cubic computational complexity in terms of the matrix dimension, which is not acceptable for large graphs with millions of nodes. Thus, rigorous solutions to the diagonal of the Laplacian matrices for general graphs, even for particluar graphs are much less. In this paper, we propose a theoretically guaranteed estimation algorithm, which approximates all diagonal entries of the pseudoinverse of a graph Laplacian in nearly linear time with respect to the number of edges in the graph. We execute extensive experiments on real-life networks, which indicate that our algorithm is both efficient and accurate. Also, we determine exact expressions for the diagonal elements of pseudoinverse of the Laplacian matrices for Koch networks and uniform recursive trees, and compare them with those obtained by our approximation algorithm. Finally, we use our algorithm to evaluate the Kirchhoff index of three deterministic model networks, for which the Kirchhoff index can be rigorously determined. These results further show the effectiveness and efficiency of our algorithm.
Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments
Venue: arXiv cs.RO 2023
Year: 2023

As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.
CLAID: Closing the Loop on AI & Data Collection -- A Cross-Platform Transparent Computing Middleware Framework for Smart Edge-Cloud and Digital Biomarker Applications
Venue: arXiv cs.DC 2023
Year: 2023

The increasing number of edge devices with enhanced sensing capabilities, such as smartphones, wearables, and IoT devices equipped with sensors, holds the potential for innovative smart-edge applications in healthcare. These devices generate vast amounts of multimodal data, enabling the implementation of digital biomarkers which can be leveraged by machine learning solutions to derive insights, predict health risks, and allow personalized interventions. Training these models requires collecting data from edge devices and aggregating it in the cloud. To validate and verify those models, it is essential to utilize them in real-world scenarios and subject them to testing using data from diverse cohorts. Since some models are too computationally expensive to be run on edge devices directly, a collaborative framework between the edge and cloud becomes necessary. In this paper, we present CLAID, an open-source cross-platform middleware framework based on transparent computing compatible with Android, iOS, WearOS, Linux, macOS, and Windows. CLAID enables logical integration of devices running different operating systems into an edge-cloud system, facilitating communication and offloading between them, with bindings available in different programming languages. We provide Modules for data collection from various sensors as well as for the deployment of machine-learning models. Furthermore, we propose a novel methodology, "ML-Model in the Loop" for verifying deployed machine learning models, which helps to analyze problems that may occur during the migration of models from cloud to edge devices. We verify our framework in three different experiments and achieve 100% sampling coverage for data collection across different sensors as well as an equal performance of a cough detection model deployed on both Android and iOS devices. We evaluate the memory and battery consumption of our framework.
FENCE: Fairplay Ensuring Network Chain Entity for Real-Time Multiple ID Detection at Scale In Fantasy Sports
Venue: arXiv cs.LG 2023
Year: 2023

Dream11 takes pride in being a unique platform that enables over 190 million fantasy sports users to demonstrate their skills and connect deeper with their favorite sports. While managing such a scale, one issue we are faced with is duplicate/multiple account creation in the system. This is done by some users with the intent of abusing the platform, typically for bonus offers. The challenge is to detect these multiple accounts before it is too late. We propose a graph-based solution to solve this problem in which we first predict edges/associations between users. Using the edge information we highlight clusters of colluding multiple accounts. In this paper, we talk about our distributed ML system which is deployed to serve and support the inferences from our detection models. The challenge is to do this in real-time in order to take corrective actions. A core part of this setup also involves human-in-the-loop components for validation, feedback, and ground-truth labeling.
No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
Venue: arXiv cs.CV 2023
Year: 2023

Vision Transformers (ViTs) have demonstrated outstanding performance in computer vision tasks, yet their high computational complexity prevents their deployment in computing resource-constrained environments. Various token pruning techniques have been introduced to alleviate the high computational burden of ViTs by dynamically dropping image tokens. However, some undesirable pruning at early stages may result in permanent loss of image information in subsequent layers, consequently hindering model performance. To address this problem, we propose IdleViT, a dynamic token-idle-based method that achieves an excellent trade-off between performance and efficiency. Specifically, in each layer, IdleViT selects a subset of the image tokens to participate in computations while keeping the rest of the tokens idle and directly passing them to this layer's output. By allowing the idle tokens to be re-selected in the following layers, IdleViT mitigates the negative impact of improper pruning in the early stages. Furthermore, inspired by the normalized graph cut, we devise a token cut loss on the attention map as regularization to improve IdleViT's token selection ability. Our method is simple yet effective and can be extended to pyramid ViTs since no token is completely dropped. Extensive experimental results on various ViT architectures have shown that IdleViT can diminish the complexity of pretrained ViTs by up to 33\% with no more than 0.2\% accuracy decrease on ImageNet, after finetuning for only 30 epochs. Notably, when the keep ratio is 0.5, IdleViT outperforms the state-of-the-art EViT on DeiT-S by 0.5\% higher accuracy and even faster inference speed. The source code is available in the supplementary material.
Abstractive Summarization of Large Document Collections Using GPT
Venue: arXiv cs.AI 2023
Year: 2023

This paper proposes a method of abstractive summarization designed to scale to document collections instead of individual documents. Our approach applies a combination of semantic clustering, document size reduction within topic clusters, semantic chunking of a cluster's documents, GPT-based summarization and concatenation, and a combined sentiment and text visualization of each topic to support exploratory data analysis. Statistical comparison of our results to existing state-of-the-art systems BART, BRIO, PEGASUS, and MoCa using ROGUE summary scores showed statistically equivalent performance with BART and PEGASUS on the CNN/Daily Mail test dataset, and with BART on the Gigaword test dataset. This finding is promising since we view document collection summarization as more challenging than individual document summarization. We conclude with a discussion of how issues of scale are
Based on What We Can Control Artificial Neural Networks
Venue: arXiv cs.AI 2023
Year: 2023

How can the stability and efficiency of Artificial Neural Networks (ANNs) be ensured through a systematic analysis method? This paper seeks to address that query. While numerous factors can influence the learning process of ANNs, utilizing knowledge from control systems allows us to analyze its system function and simulate system responses. Although the complexity of most ANNs is extremely high, we still can analyze each factor (e.g., optimiser, hyperparameters) by simulating their system response. This new method also can potentially benefit the development of new optimiser and learning system, especially when discerning which components adversely affect ANNs. Controlling ANNs can benefit from the design of optimiser and learning system, as (1) all optimisers act as controllers, (2) all learning systems operate as control systems with inputs and outputs, and (3) the optimiser should match the learning system. Please find codes: \url{https://github.com/RandomUserName2023/Control-ANNs}.
DiCE -- A Data Encryption Proxy for the Cloud
Venue: arXiv cs.CR 2023
Year: 2023

Outsourcing a relational database to the cloud offers several benefits, including scalability, availability, and cost-effectiveness. However, there are concerns about the confidentiality and security of the outsourced data. A general approach here would be to encrypt the data with a standardized encryption algorithm and then store the data only encrypted in the cloud. The problem with this approach, however, is that with encryption, important properties of the data such as sorting, format or comparability, which are essential for the functioning of database queries, are lost. One solution to this problem is the use of (e.g. order-preserving) encryption algorithms, which also preserve these properties in the encrypted data, thus enabling queries to encrypted data. These algorithms range from simple algorithms like Caesar encryption to secure algorithms like mOPE. In order to be able to use these algorithms as easy as possible, ``DiCE'' a JDBC driver was developed, that parses SQL queries as a proxy and transparently encrypts and decrypts these queries. This allows to execute many queries on an encrypted database in the cloud with (nearly) the performance as on unencrypted databases. The DiCE driver can be used with any other JDBC driver and therefore supports a variety of databases. The driver can be configured to support different encryption algorithms. To keep track of the operations, the ``Dice Information Client'' has been developed to track the encryption and decryption of the driver. Although the result of the performance analysis shows a certain overhead due to the parsing and encryption of the SQL queries in the Dice driver, this overhead is significantly reduced by other influencing factors such as the network and parallel queries.
Imitator Learning: Achieve Out-of-the-Box Imitation Ability in Variable Environments
Venue: arXiv cs.LG 2023
Year: 2023

Imitation learning (IL) enables agents to mimic expert behaviors. Most previous IL techniques focus on precisely imitating one policy through mass demonstrations. However, in many applications, what humans require is the ability to perform various tasks directly through a few demonstrations of corresponding tasks, where the agent would meet many unexpected changes when deployed. In this scenario, the agent is expected to not only imitate the demonstration but also adapt to unforeseen environmental changes. This motivates us to propose a new topic called imitator learning (ItorL), which aims to derive an imitator module that can on-the-fly reconstruct the imitation policies based on very limited expert demonstrations for different unseen tasks, without any extra adjustment. In this work, we focus on imitator learning based on only one expert demonstration. To solve ItorL, we propose Demo-Attention Actor-Critic (DAAC), which integrates IL into a reinforcement-learning paradigm that can regularize policies' behaviors in unexpected situations. Besides, for autonomous imitation policy building, we design a demonstration-based attention architecture for imitator policy that can effectively output imitated actions by adaptively tracing the suitable states in demonstrations. We develop a new navigation benchmark and a robot environment for \topic~and show that DAAC~outperforms previous imitation methods \textit{with large margins} both on seen and unseen tasks.
FeatSense -- A Feature-based Registration Algorithm with GPU-accelerated TSDF-Mapping Backend for NVIDIA Jetson Boards
Venue: arXiv cs.RO 2023
Year: 2023

This paper presents FeatSense, a feature-based GPU-accelerated SLAM system for high resolution LiDARs, combined with a map generation algorithm for real-time generation of large Truncated Signed Distance Fields (TSDFs) on embedded hardware. FeatSense uses LiDAR point cloud features for odometry estimation and point cloud registration. The registered point clouds are integrated into a global Truncated Signed Distance Field (TSDF) representation. FeatSense is intended to run on embedded systems with integrated GPU-accelerator like NVIDIA Jetson boards. In this paper, we present a real-time capable TSDF-SLAM system specially tailored for close coupled CPU/GPU systems. The implementation is evaluated in various structured and unstructured environments and benchmarked against existing reference datasets. The main contribution of this paper is the ability to register up to 128 scan lines of an Ouster OS1-128 LiDAR at 10Hz on a NVIDIA AGX Xavier while achieving a TSDF map generation speedup by a factor of 100 compared to previous work on the same power budget.
Sharing Information Between Machine Tools to Improve Surface Finish Forecasting
Venue: arXiv cs.LG 2023
Year: 2023

At present, most surface-quality prediction methods can only perform single-task prediction which results in under-utilised datasets, repetitive work and increased experimental costs. To counter this, the authors propose a Bayesian hierarchical model to predict surface-roughness measurements for a turning machining process. The hierarchical model is compared to multiple independent Bayesian linear regression models to showcase the benefits of partial pooling in a machining setting with respect to prediction accuracy and uncertainty quantification.
Write What You Want: Applying Text-to-video Retrieval to Audiovisual Archives
Venue: arXiv cs.MM 2023
Year: 2023

Audiovisual (AV) archives, as an essential reservoir of our cultural assets, are suffering from the issue of accessibility. The complex nature of the medium itself made processing and interaction an open challenge still in the field of computer vision, multimodal learning, and human-computer interaction, as well as in culture and heritage. In recent years, with the raising of video retrieval tasks, methods in retrieving video content with natural language (text-to-video retrieval) gained quite some attention and have reached a performance level where real-world application is on the horizon. Appealing as it may sound, such methods focus on retrieving videos using plain visual-focused descriptions of what has happened in the video and finding videos such as instructions. It is too early to say such methods would be the new paradigms for accessing and encoding complex video content into high-dimensional data, but they are indeed innovative attempts and foundations to build future exploratory interfaces for AV archives (e.g. allow users to write stories and retrieve related snippets in the archive, or encoding video content at high-level for visualisation). This work filled the application gap by examining such text-to-video retrieval methods from an implementation point of view and proposed and verified a classifier-enhanced workflow to allow better results when dealing with in-situ queries that might have been different from the training dataset. Such a workflow is then applied to the real-world archive from Télévision Suisse Romande (RTS) to create a demo. At last, a human-centred evaluation is conducted to understand whether the text-to-video retrieval methods improve the overall experience of accessing AV archives.
Domain-wise Invariant Learning for Panoptic Scene Graph Generation
Venue: arXiv cs.CV 2023
Year: 2023

Panoptic Scene Graph Generation (PSG) involves the detection of objects and the prediction of their corresponding relationships (predicates). However, the presence of biased predicate annotations poses a significant challenge for PSG models, as it hinders their ability to establish a clear decision boundary among different predicates. This issue substantially impedes the practical utility and real-world applicability of PSG models. To address the intrinsic bias above, we propose a novel framework to infer potentially biased annotations by measuring the predicate prediction risks within each subject-object pair (domain), and adaptively transfer the biased annotations to consistent ones by learning invariant predicate representation embeddings. Experiments show that our method significantly improves the performance of benchmark models, achieving a new state-of-the-art performance, and shows great generalization and effectiveness on PSG dataset.
Bio-inspired computational memory model of the Hippocampus: an approach to a neuromorphic spike-based Content-Addressable Memory
Venue: arXiv cs.NE 2023
Year: 2023

The brain has computational capabilities that surpass those of modern systems, being able to solve complex problems efficiently in a simple way. Neuromorphic engineering aims to mimic biology in order to develop new systems capable of incorporating such capabilities. Bio-inspired learning systems continue to be a challenge that must be solved, and much work needs to be done in this regard. Among all brain regions, the hippocampus stands out as an autoassociative short-term memory with the capacity to learn and recall memories from any fragment of them. These characteristics make the hippocampus an ideal candidate for developing bio-inspired learning systems that, in addition, resemble content-addressable memories. Therefore, in this work we propose a bio-inspired spiking content-addressable memory model based on the CA3 region of the hippocampus with the ability to learn, forget and recall memories, both orthogonal and non-orthogonal, from any fragment of them. The model was implemented on the SpiNNaker hardware platform using Spiking Neural Networks. A set of experiments based on functional, stress and applicability tests were performed to demonstrate its correct functioning. This work presents the first hardware implementation of a fully-functional bio-inspired spiking hippocampal content-addressable memory model, paving the way for the development of future more complex neuromorphic systems.
On Multi-Fidelity Impedance Tuning for Human-Robot Cooperative Manipulation
Venue: arXiv cs.RO 2023
Year: 2023

We examine how a human-robot interaction (HRI) system may be designed when input-output data from previous experiments are available. In particular, we consider how to select an optimal impedance in the assistance design for a cooperative manipulation task with a new operator. Due to the variability between individuals, the design parameters that best suit one operator of the robot may not be the best parameters for another one. However, by incorporating historical data using a linear auto-regressive (AR-1) Gaussian process, the search for a new operator's optimal parameters can be accelerated. We lay out a framework for optimizing the human-robot cooperative manipulation that only requires input-output data. We establish how the AR-1 model improves the bound on the regret and numerically simulate a human-robot cooperative manipulation task to show the regret improvement. Further, we show how our approach's input-output nature provides robustness against modeling error through an additional numerical study.
Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Input
Venue: arXiv cs.GR 2023
Year: 2023

Clothing is an important part of human appearance but challenging to model in photorealistic avatars. In this work we present avatars with dynamically moving loose clothing that can be faithfully driven by sparse RGB-D inputs as well as body and face motion. We propose a Neural Iterative Closest Point (N-ICP) algorithm that can efficiently track the coarse garment shape given sparse depth input. Given the coarse tracking results, the input RGB-D images are then remapped to texel-aligned features, which are fed into the drivable avatar models to faithfully reconstruct appearance details. We evaluate our method against recent image-driven synthesis baselines, and conduct a comprehensive analysis of the N-ICP algorithm. We demonstrate that our method can generalize to a novel testing environment, while preserving the ability to produce high-fidelity and faithful clothing dynamics and appearance.
Vulnerability Clustering and other Machine Learning Applications of Semantic Vulnerability Embeddings
Venue: arXiv cs.CR 2023
Year: 2023

Cyber-security vulnerabilities are usually published in form of short natural language descriptions (e.g., in form of MITRE's CVE list) that over time are further manually enriched with labels such as those defined by the Common Vulnerability Scoring System (CVSS). In the Vulnerability AI (Analytics and Intelligence) project, we investigated different types of semantic vulnerability embeddings based on natural language processing (NLP) techniques to obtain a concise representation of the vulnerability space. We also evaluated their use as a foundation for machine learning applications that can support cyber-security researchers and analysts in risk assessment and other related activities. The particular applications we explored and briefly summarize in this report are clustering, classification, and visualization, as well as a new logic-based approach to evaluate theories about the vulnerability space.
Technocracy, pseudoscience and performative compliance: the risks of privacy risk assessments. Lessons from NIST's Privacy Risk Assessment Methodology
Venue: arXiv cs.CR 2023
Year: 2023

Privacy risk assessments have been touted as an objective, principled way to encourage organizations to implement privacy-by-design. They are central to a new regulatory model of collaborative governance, as embodied by the GDPR. However, existing guidelines and methods remain vague, and there is little empirical evidence on privacy harms. In this paper we conduct a close analysis of US NIST's Privacy Risk Assessment Methodology, highlighting multiple sites of discretion that create countless opportunities for adversarial organizations to engage in performative compliance. Our analysis shows that the premises on which the success of privacy risk assessments depends do not hold, particularly in regard to organizations' incentives and regulators auditing capabilities. We highlight the limitations and pitfalls of what is essentially a utilitarian and technocratic approach, leading us to discuss alternatives and a realignment of our policy and research objectives.