OpenRaiser/Intern-Atlas huggingface dataset:
Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulation
Venue: arXiv cs.RO 2023
Year: 2023

Existing object-search approaches enable robots to search through free pathways, however, robots operating in unstructured human-centered environments frequently also have to manipulate the environment to their needs. In this work, we introduce a novel interactive multi-object search task in which a robot has to open doors to navigate rooms and search inside cabinets and drawers to find target objects. These new challenges require combining manipulation and navigation skills in unexplored environments. We present HIMOS, a hierarchical reinforcement learning approach that learns to compose exploration, navigation, and manipulation skills. To achieve this, we design an abstract high-level action space around a semantic map memory and leverage the explored environment as instance navigation points. We perform extensive experiments in simulation and the real world that demonstrate that, with accurate perception, the decision making of HIMOS effectively transfers to new environments in a zero-shot manner. It shows robustness to unseen subpolicies, failures in their execution, and different robot kinematics. These capabilities open the door to a wide range of downstream tasks across embodied AI and real-world use cases.
SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning
Venue: arXiv cs.RO 2023
Year: 2023

Large language models (LLMs) have demonstrated impressive results in developing generalist planning agents for diverse tasks. However, grounding these plans in expansive, multi-floor, and multi-room environments presents a significant challenge for robotics. We introduce SayPlan, a scalable approach to LLM-based, large-scale task planning for robotics using 3D scene graph (3DSG) representations. To ensure the scalability of our approach, we: (1) exploit the hierarchical nature of 3DSGs to allow LLMs to conduct a 'semantic search' for task-relevant subgraphs from a smaller, collapsed representation of the full graph; (2) reduce the planning horizon for the LLM by integrating a classical path planner and (3) introduce an 'iterative replanning' pipeline that refines the initial plan using feedback from a scene graph simulator, correcting infeasible actions and avoiding planning failures. We evaluate our approach on two large-scale environments spanning up to 3 floors and 36 rooms with 140 assets and objects and show that our approach is capable of grounding large-scale, long-horizon task plans from abstract, and natural language instruction for a mobile manipulator robot to execute. We provide real robot video demonstrations on our project page https://sayplan.github.io.
Reflective Hybrid Intelligence for Meaningful Human Control in Decision-Support Systems
Venue: arXiv cs.AI 2023
Year: 2023

With the growing capabilities and pervasiveness of AI systems, societies must collectively choose between reduced human autonomy, endangered democracies and limited human rights, and AI that is aligned to human and social values, nurturing collaboration, resilience, knowledge and ethical behaviour. In this chapter, we introduce the notion of self-reflective AI systems for meaningful human control over AI systems. Focusing on decision support systems, we propose a framework that integrates knowledge from psychology and philosophy with formal reasoning methods and machine learning approaches to create AI systems responsive to human values and social norms. We also propose a possible research approach to design and develop self-reflective capability in AI systems. Finally, we argue that self-reflective AI systems can lead to self-reflective hybrid systems (human + AI), thus increasing meaningful human control and empowering human moral reasoning by providing comprehensible information and insights on possible human moral blind spots.
CellGAN: Conditional Cervical Cell Synthesis for Augmenting Cytopathological Image Classification
Venue: arXiv cs.CV 2023
Year: 2023

Automatic examination of thin-prep cytologic test (TCT) slides can assist pathologists in finding cervical abnormality for accurate and efficient cancer screening. Current solutions mostly need to localize suspicious cells and classify abnormality based on local patches, concerning the fact that whole slide images of TCT are extremely large. It thus requires many annotations of normal and abnormal cervical cells, to supervise the training of the patch-level classifier for promising performance. In this paper, we propose CellGAN to synthesize cytopathological images of various cervical cell types for augmenting patch-level cell classification. Built upon a lightweight backbone, CellGAN is equipped with a non-linear class mapping network to effectively incorporate cell type information into image generation. We also propose the Skip-layer Global Context module to model the complex spatial relationship of the cells, and attain high fidelity of the synthesized images through adversarial learning. Our experiments demonstrate that CellGAN can produce visually plausible TCT cytopathological images for different cell types. We also validate the effectiveness of using CellGAN to greatly augment patch-level cell classification performance.
Self-Adaptive Large Language Model (LLM)-Based Multiagent Systems
Venue: arXiv cs.MA 2023
Year: 2023

In autonomic computing, self-adaptation has been proposed as a fundamental paradigm to manage the complexity of multiagent systems (MASs). This achieved by extending a system with support to monitor and adapt itself to achieve specific concerns of interest. Communication in these systems is key given that in scenarios involving agent interaction, it enhances cooperation and reduces coordination challenges by enabling direct, clear information exchange. However, improving the expressiveness of the interaction communication with MASs is not without challenges. In this sense, the interplay between self-adaptive systems and effective communication is crucial for future MAS advancements. In this paper, we propose the integration of large language models (LLMs) such as GPT-based technologies into multiagent systems. We anchor our methodology on the MAPE-K model, which is renowned for its robust support in monitoring, analyzing, planning, and executing system adaptations in response to dynamic environments. We also present a practical illustration of the proposed approach, in which we implement and assess a basic MAS-based application. The approach significantly advances the state-of-the-art of self-adaptive systems by proposing a new paradigm for MAS self-adaptation of autonomous systems based on LLM capabilities.
New Three and Four-Dimensional Toric and Burst-Error-Correcting Quantum Codes
Venue: arXiv cs.IT 2023
Year: 2023

Ongoing research and experiments have enabled quantum memory to realize the storage of qubits. On the other hand, interleaving techniques are used to deal with burst of errors. Effective interleaving techniques for combating burst of errors by using classical error-correcting codes have been proposed in several articles found in the literature, however, to the best of our knowledge, little is known regarding interleaving techniques for combating clusters of errors in topological quantum error-correcting codes. Motivated by that, in this work, we present new three and four-dimensional toric quantum codes which are featured by lattice codes and apply a quantum interleaving method to such new three and four-dimensional toric quantum codes. By applying such a method to these new codes we provide new three and four-dimensional quantum burst-error-correcting codes. As a consequence, new three and four-dimensional toric and burst-error-correcting quantum codes are obtained which have better information rates than those three and four-dimensional toric quantum codes from the literature. In addition to these proposed three and four-dimensional quantum burst-error-correcting codes improve such information rates, they can be used for burst-error-correction in errors which are located, quantum data stored and quantum channels with memory.
Exposing the Fake: Effective Diffusion-Generated Images Detection
Venue: arXiv cs.CV 2023
Year: 2023

Image synthesis has seen significant advancements with the advent of diffusion-based generative models like Denoising Diffusion Probabilistic Models (DDPM) and text-to-image diffusion models. Despite their efficacy, there is a dearth of research dedicated to detecting diffusion-generated images, which could pose potential security and privacy risks. This paper addresses this gap by proposing a novel detection method called Stepwise Error for Diffusion-generated Image Detection (SeDID). Comprising statistical-based $\text{SeDID}_{\text{Stat}}$ and neural network-based $\text{SeDID}_{\text{NNs}}$, SeDID exploits the unique attributes of diffusion models, namely deterministic reverse and deterministic denoising computation errors. Our evaluations demonstrate SeDID's superior performance over existing methods when applied to diffusion models. Thus, our work makes a pivotal contribution to distinguishing diffusion model-generated images, marking a significant step in the domain of artificial intelligence security.
SAGE -- A Tool for Optimal Deployments in Kubernetes Clusters
Venue: arXiv cs.DC 2023
Year: 2023

Cloud computing has brought a fundamental transformation in how organizations operate their applications, enabling them to achieve affordable high availability of services. Kubernetes has emerged as the preferred choice for container orchestration and service management across many Cloud computing platforms. The scheduler in Kubernetes plays a crucial role in determining the placement of newly deployed service containers. However, the default scheduler, while fast, often lacks optimization, leading to inefficient service placement or even deployment failures. This paper introduces SAGE, a tool for optimal solutions in Kubernetes clusters that can also assist the Kubernetes default scheduler and any other custom scheduler in application deployment. SAGE computes an optimal deployment plan based on the constraints of the application to be deployed and the available Cloud resources. We show the potential benefits of using SAGE by considering test cases with various characteristics. It turns out that SAGE surpasses other schedulers by comprehensively analyzing the application demand and cluster image. This ability allows it to better understand the needs of the pods, resulting in consistently optimal solutions across all scenarios. The accompanying material of this paper is publicly available at https://github.com/SAGE-Project/SAGE-Predeployer.
Personalized Anomaly Detection in PPG Data using Representation Learning and Biometric Identification
Venue: arXiv cs.LG 2023
Year: 2023

Photoplethysmography (PPG) signals, typically acquired from wearable devices, hold significant potential for continuous fitness-health monitoring. In particular, heart conditions that manifest in rare and subtle deviating heart patterns may be interesting. However, robust and reliable anomaly detection within these data remains a challenge due to the scarcity of labeled data and high inter-subject variability. This paper introduces a two-stage framework leveraging representation learning and personalization to improve anomaly detection performance in PPG data. The proposed framework first employs representation learning to transform the original PPG signals into a more discriminative and compact representation. We then apply three different unsupervised anomaly detection methods for movement detection and biometric identification. We validate our approach using two different datasets in both generalized and personalized scenarios. The results show that representation learning significantly improves anomaly detection performance while reducing the high inter-subject variability. Personalized models further enhance anomaly detection performance, underscoring the role of personalization in PPG-based fitness-health monitoring systems. The results from biometric identification show that it's easier to distinguish a new user from one intended authorized user than from a group of users. Overall, this study provides evidence of the effectiveness of representation learning and personalization for anomaly detection in PPG data.
Rethinking Answer Set Programming Templates
Venue: arXiv cs.AI 2023
Year: 2023

In imperative programming, the Domain-Driven Design methodology helps in coping with the complexity of software development by materializing in code the invariants of a domain of interest. Code is cleaner and more secure because any implicit assumption is removed in favor of invariants, thus enabling a fail fast mindset and the immediate reporting of unexpected conditions. This article introduces a notion of template for Answer Set Programming that, in addition to the don't repeat yourself principle, enforces locality of some predicates by means of a simple naming convention. Local predicates are mapped to the usual global namespace adopted by mainstream engines, using universally unique identifiers to avoid name clashes. This way, local predicates can be used to enforce invariants on the expected outcome of a template in a possibly empty context of application, independently by other rules that can be added to such a context. Template applications transpiled this way can be processed by mainstream engines and safely shared with other knowledge designers, even when they have zero knowledge of templates.
RaBiT: An Efficient Transformer using Bidirectional Feature Pyramid Network with Reverse Attention for Colon Polyp Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Automatic and accurate segmentation of colon polyps is essential for early diagnosis of colorectal cancer. Advanced deep learning models have shown promising results in polyp segmentation. However, they still have limitations in representing multi-scale features and generalization capability. To address these issues, this paper introduces RaBiT, an encoder-decoder model that incorporates a lightweight Transformer-based architecture in the encoder to model multiple-level global semantic relationships. The decoder consists of several bidirectional feature pyramid layers with reverse attention modules to better fuse feature maps at various levels and incrementally refine polyp boundaries. We also propose ideas to lighten the reverse attention module and make it more suitable for multi-class segmentation. Extensive experiments on several benchmark datasets show that our method outperforms existing methods across all datasets while maintaining low computational complexity. Moreover, our method demonstrates high generalization capability in cross-dataset experiments, even when the training and test sets have different characteristics.
Scientific mobility, prestige and skill alignment in academic institutions
Venue: arXiv cs.DL 2023
Year: 2023

Scientific institutions play a crucial role in driving intellectual, social, and technological progress. Their capacity to innovate depends mainly on their ability to attract, retain, and nurture scientific talent and ultimately make it available to other organizations, industries, or the economy. As researchers change institutions during their careers, their skills are also transferred. The extent and mechanisms by which academic institutions manage their internal portfolio of scientific skills by attracting and sending researchers are far from being understood. We examine 25 million publication histories of 9.2 million scientists extracted from a large-scale bibliographic database covering thousands of research institutions worldwide to understand how the skills of mobile scientists align with those present in-house. We find a clear association between top-ranked institutions and greater skill alignment, i.e., the degree to which skills of incoming academics match those of their colleagues at the institution. We uncover similar high-alignment for scientists leaving top-ranked institutions. This type of academic alignment is more pronounced in engineering and life, health, earth, and physical sciences than in mathematics, computer science, social sciences, and the humanities. We show that over the past two decades, institutions generally have become more closely aligned in their overall skill profiles. We interpret these results in terms of levels of proactive management of the composition of the scientific workforce, diversity, and internal collaboration strategies at the institutional level.
Efficient Convolution and Transformer-Based Network for Video Frame Interpolation
Venue: arXiv cs.CV 2023
Year: 2023

Video frame interpolation is an increasingly important research task with several key industrial applications in the video coding, broadcast and production sectors. Recently, transformers have been introduced to the field resulting in substantial performance gains. However, this comes at a cost of greatly increased memory usage, training and inference time. In this paper, a novel method integrating a transformer encoder and convolutional features is proposed. This network reduces the memory burden by close to 50% and runs up to four times faster during inference time compared to existing transformer-based interpolation methods. A dual-encoder architecture is introduced which combines the strength of convolutions in modelling local correlations with those of the transformer for long-range dependencies. Quantitative evaluations are conducted on various benchmarks with complex motion to showcase the robustness of the proposed method, achieving competitive performance compared to state-of-the-art interpolation networks.
Market Driven Multi-domain Network Service Orchestration in 5G Networks
Venue: arXiv cs.NI 2023
Year: 2023

The advent of a new breed of enhanced multimedia services has put network operators into a position where they must support innovative services while ensuring both end-to-end Quality of Service requirements and profitability. Recently, Network Function Virtualization (NFV) has been touted as a cost-effective underlying technology in 5G networks to efficiently provision novel services. These NFV-based services have been increasingly associated with multi-domain networks. However, several orchestration issues, linked to cross-domain interactions and emphasized by the heterogeneity of underlying technologies and administrative authorities, present an important challenge. In this paper, we tackle the cross-domain interaction issue by proposing an intelligent and profitable auction-based approach to allow inter-domains resource allocation.
Free-Form Composition Networks for Egocentric Action Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Egocentric action recognition is gaining significant attention in the field of human action recognition. In this paper, we address data scarcity issue in egocentric action recognition from a compositional generalization perspective. To tackle this problem, we propose a free-form composition network (FFCN) that can simultaneously learn disentangled verb, preposition, and noun representations, and then use them to compose new samples in the feature space for rare classes of action videos. First, we use a graph to capture the spatial-temporal relations among different hand/object instances in each action video. We thus decompose each action into a set of verb and preposition spatial-temporal representations using the edge features in the graph. The temporal decomposition extracts verb and preposition representations from different video frames, while the spatial decomposition adaptively learns verb and preposition representations from action-related instances in each frame. With these spatial-temporal representations of verbs and prepositions, we can compose new samples for those rare classes in a free-form manner, which is not restricted to a rigid form of a verb and a noun. The proposed FFCN can directly generate new training data samples for rare classes, hence significantly improve action recognition performance. We evaluated our method on three popular egocentric action recognition datasets, Something-Something V2, H2O, and EPIC-KITCHENS-100, and the experimental results demonstrate the effectiveness of the proposed method for handling data scarcity problems, including long-tailed and few-shot egocentric action recognition.
Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study
Venue: arXiv cs.CL 2023
Year: 2023

This paper explores the integration of Large Language Models (LLMs) into Automatic Speech Recognition (ASR) systems to improve transcription accuracy. The increasing sophistication of LLMs, with their in-context learning capabilities and instruction-following behavior, has drawn significant attention in the field of Natural Language Processing (NLP). Our primary focus is to investigate the potential of using an LLM's in-context learning capabilities to enhance the performance of ASR systems, which currently face challenges such as ambient noise, speaker accents, and complex linguistic contexts. We designed a study using the Aishell-1 and LibriSpeech datasets, with ChatGPT and GPT-4 serving as benchmarks for LLM capabilities. Unfortunately, our initial experiments did not yield promising results, indicating the complexity of leveraging LLM's in-context learning for ASR applications. Despite further exploration with varied settings and models, the corrected sentences from the LLMs frequently resulted in higher Word Error Rates (WER), demonstrating the limitations of LLMs in speech applications. This paper provides a detailed overview of these experiments, their results, and implications, establishing that using LLMs' in-context learning capabilities to correct potential errors in speech recognition transcriptions is still a challenging task at the current stage.
Faster Rectangular Matrix Multiplication by Combination Loss Analysis
Venue: arXiv cs.DS 2023
Year: 2023

Duan, Wu and Zhou (FOCS 2023) recently obtained the improved upper bound on the exponent of square matrix multiplication $ω<2.3719$ by introducing a new approach to quantify and compensate the ``combination loss" in prior analyses of powers of the Coppersmith-Winograd tensor. In this paper we show how to use this new approach to improve the exponent of rectangular matrix multiplication as well. Our main technical contribution is showing how to combine this analysis of the combination loss and the analysis of the fourth power of the Coppersmith-Winograd tensor in the context of rectangular matrix multiplication developed by Le Gall and Urrutia (SODA 2018).
Money: Who Has a Stake in the Most Value-Centric Common Design Material?
Venue: arXiv cs.CY 2023
Year: 2023

Money is more than just a numeric value. It embodies trust and moral gravity, and it offers flexible ways to transact. However, the emergence of Central Bank Digital Currency (CBDC) is set to bring about a drastic change in the future of money. This paper invites designers to reflect on their role in shaping material and immaterial monetary change. In this rapidly changing landscape, design could be instrumental in uncovering and showcasing the diverse values that money holds for different stakeholders. Understanding these diversities could promote a more equitable and inclusive financial, social, and global landscape within emergent forms of cash-like digital currency. Without such consideration, certain forms of money we have come to know could disappear, along with the values people hold upon them. We report on semi-structured interviews with stakeholders who have current knowledge or involvement in the emerging field of Central Bank Digital Currency (CBDC). Our research indicates that this new form of money presents both challenges and opportunities for designers. Specifically, we emphasise the potential for Central Bank Digital Currency (CBDC) to either positively or negatively reform values through its design. By considering time, reflecting present values, and promoting inclusion in its deployment, we can strive to ensure that Central Bank Digital Currency (CBDC) represents the diverse needs and perspectives of its users.