OpenRaiser/Intern-Atlas huggingface dataset:
Level Assembly as a Markov Decision Process
Venue: arXiv cs.AI 2023
Year: 2023

Many games feature a progression of levels that doesn't adapt to the player. This can be problematic because some players may get stuck if the progression is too difficult, while others may find it boring if the progression is too slow to get to more challenging levels. This can be addressed by building levels based on the player's performance and preferences. In this work, we formulate the problem of generating levels for a player as a Markov Decision Process (MDP) and use adaptive dynamic programming (ADP) to solve the MDP before assembling a level. We tested with two case studies and found that using an ADP outperforms two baselines. Furthermore, we experimented with player proxies and switched them in the middle of play, and we show that a simple modification prior to running ADP results in quick adaptation. By using ADP, which searches the entire MDP, we produce a dynamic progression of levels that adapts to the player.
Open Metaverse: Issues, Evolution, and Future
Venue: arXiv cs.DB 2023
Year: 2023

With the evolution of content on the web and the Internet, there is a need for cyberspace that can be used to work, live, and play in digital worlds regardless of geography. The Metaverse provides the possibility of future Internet and represents a future trend. In the future, the Metaverse will be a space where the real and the virtual are combined. In this article, we have a comprehensive survey of the compelling Metaverse. We introduce computer technology, the history of the Internet, and the promise of the Metaverse as the next generation of the Internet. In addition, we briefly introduce the related concepts of the Metaverse, including novel terms like trusted Metaverse, human-intelligence Metaverse, personalized Metaverse, AI-enabled Metaverse, Metaverse-as-a-service, etc. Moreover, we present the challenges of the Metaverse such as limited resources and ethical issues. We also present Metaverse's promising directions, including lightweight Metaverse and autonomous Metaverse. We hope this survey will provide some helpful prospects and insightful directions about the Metaverse to related developments.
You Can't Always Check What You Wanted: Selective Checking and Trusted Execution to Prevent False Actuations in Cyber-Physical Systems
Venue: arXiv cs.CR 2023
Year: 2023

Cyber-physical systems (CPS) are vulnerable to attacks targeting outgoing actuation commands that modify their physical behaviors. The limited resources in such systems, coupled with their stringent timing constraints, often prevents the checking of every outgoing command. We present a "selective checking" mechanism that uses game-theoretic modeling to identify the right subset of commands to be checked in order to deter an adversary. This mechanism is coupled with a "delay-aware" trusted execution environment (TEE) to ensure that only verified actuation commands are ever sent to the physical system, thus maintaining their safety and integrity. The selective checking and trusted execution (SCATE) framework is implemented on an off-the-shelf ARM platform running standard embedded Linux. We demonstrate the effectiveness of SCATE using four realistic cyber-physical systems (a ground rover, a flight controller, a robotic arm and an automated syringe pump) and study design trade-offs. Not only does SCATE provide a high level of security and high performance, it also suffers from significantly lower overheads (30.48%-47.32% less) in the process. In fact, SCATE can work with more systems without negatively affecting the safety of the system. Considering that most CPS do not have any such checking mechanisms, and SCATE is guaranteed to meet all the timing requirements (i.e., ensure the safety/integrity of the system), our methods can significantly improve the security (and, hence, safety) of the system.
Human-machine knowledge hybrid augmentation method for surface defect detection based few-data learning
Venue: arXiv cs.CV 2023
Year: 2023

Visual-based defect detection is a crucial but challenging task in industrial quality control. Most mainstream methods rely on large amounts of existing or related domain data as auxiliary information. However, in actual industrial production, there are often multi-batch, low-volume manufacturing scenarios with rapidly changing task demands, making it difficult to obtain sufficient and diverse defect data. This paper proposes a parallel solution that uses a human-machine knowledge hybrid augmentation method to help the model extract unknown important features. Specifically, by incorporating experts' knowledge of abnormality to create data with rich features, positions, sizes, and backgrounds, we can quickly accumulate an amount of data from scratch and provide it to the model as prior knowledge for few-data learning. The proposed method was evaluated on the magnetic tile dataset and achieved F1-scores of 60.73%, 70.82%, 77.09%, and 82.81% when using 2, 5, 10, and 15 training images, respectively. Compared to the traditional augmentation method's F1-score of 64.59%, the proposed method achieved an 18.22% increase in the best result, demonstrating its feasibility and effectiveness in few-data industrial defect detection.
A Review of Panoptic Segmentation for Mobile Mapping Point Clouds
Venue: arXiv cs.CV 2023
Year: 2023

3D point cloud panoptic segmentation is the combined task to (i) assign each point to a semantic class and (ii) separate the points in each class into object instances. Recently there has been an increased interest in such comprehensive 3D scene understanding, building on the rapid advances of semantic segmentation due to the advent of deep 3D neural networks. Yet, to date there is very little work about panoptic segmentation of outdoor mobile-mapping data, and no systematic comparisons. The present paper tries to close that gap. It reviews the building blocks needed to assemble a panoptic segmentation pipeline and the related literature. Moreover, a modular pipeline is set up to perform comprehensive, systematic experiments to assess the state of panoptic segmentation in the context of street mapping. As a byproduct, we also provide the first public dataset for that task, by extending the NPM3D dataset to include instance labels. That dataset and our source code are publicly available. We discuss which adaptations are need to adapt current panoptic segmentation methods to outdoor scenes and large objects. Our study finds that for mobile mapping data, KPConv performs best but is slower, while PointNet++ is fastest but performs significantly worse. Sparse CNNs are in between. Regardless of the backbone, Instance segmentation by clustering embedding features is better than using shifted coordinates.
An FPTAS for Budgeted Laminar Matroid Independent Set
Venue: arXiv cs.DS 2023
Year: 2023

We study the budgeted laminar matroid independent set problem. The input is a ground set, where each element has a cost and a non-negative profit, along with a laminar matroid over the elements and a budget. The goal is to select a maximum profit independent set of the matroid whose total cost is bounded by the budget. Several well known special cases, where we have, e.g., no matroid constraint (the classic knapsack problem) or a uniform matroid constraint (knapsack with a cardinality constraint), admit a fully polynomial-time approximation scheme (FPTAS). In contrast, the budgeted matroid independent set (BMI) problem with a general matroid has an efficient polynomial-time approximation scheme (EPTAS) but does not admit an FPTAS. This implies an EPTAS for our problem, which is the best known result prior to this work. We present an FPTAS for budgeted laminar matroid independent set, improving the previous EPTAS for this matroid family and generalizing the FPTAS known for knapsack with a cardinality constraint and multiple-choice knapsack. Our scheme is based on a simple dynamic program which utilizes the tree-like structure of laminar matroids.
SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish
Venue: arXiv cs.CL 2023
Year: 2023

We present SweCTRL-Mini, a large Swedish language model that can be used for inference and fine-tuning on a single consumer-grade GPU. The model is based on the CTRL architecture by Keskar, McCann, Varshney, Xiong, and Socher (2019), which means that users of the SweCTRL-Mini model can control the genre of the generated text by inserting special tokens in the generation prompts. SweCTRL-Mini is trained on a subset of the Swedish part of the mC4 corpus and a set of Swedish novels. In this article, we provide (1) a detailed account of the utilized training data and text pre-processing steps, to the extent that it is possible to check whether a specific phrase/source was a part of the training data, and (2) an evaluation of the model on both discriminative tasks, using automatic evaluation methods, and generative tasks, using human referees. We also compare the generative capabilities of the model with those of GPT-3. SweCTRL-Mini is fully open and available for download.
Direct Visual Servoing Based on Discrete Orthogonal Moments
Venue: arXiv cs.RO 2023
Year: 2023

This paper proposes a new approach to achieve direct visual servoing (DVS) based on discrete orthogonal moments (DOMs). DVS is performed in such a way that the extraction of geometric primitives, matching, and tracking steps in the conventional feature-based visual servoing pipeline can be bypassed. Although DVS enables highly precise positioning, it suffers from a limited convergence domain and poor robustness due to the extreme nonlinearity of the cost function to be minimized and the presence of redundant data between visual features. To tackle these issues, we propose a generic and augmented framework that considers DOMs as visual features. By using the Tchebichef, Krawtchouk, and Hahn moments as examples, we not only present the strategies for adaptively tuning the parameters and order of the visual features but also exhibit an analytical formulation of the associated interaction matrix. Simulations demonstrate the robustness and accuracy of our approach, as well as its advantages over the state-of-the-art. Real-world experiments have also been performed to validate the effectiveness of our approach.
Current Safety Legislation of Food Processing Smart Robot Systems The Red Meat Sector
Venue: arXiv cs.RO 2023
Year: 2023

Ensuring the safety of the equipment, its environment and most importantly, the operator during robot operations is of paramount importance. Robots and complex robotic systems are appearing in more and more industrial and professional service applications. However, while mechanical components and control systems are advancing rapidly, the legislation background and standards framework for such systems and machinery are lagging behind. As part of a fundamental research work targeting industrial robots and industry 4.0 solutions for completely automated slaughtering, it was revealed that there are no particular standards addressing robotics systems applied to the agrifood domain. More specifically, within the agrifood sector, the only standards existing for the meat industry and the red meat sector are hygienic standards related to machinery. None of the identified standards or regulations consider the safety of autonomous robot operations or human robot collaborations in the abattoirs. The goal of this paper is to provide a general overview of the regulations and standards (and similar guiding documents) relevant for such applications, that could possibly be used as guidelines during the development of inherently safe robotic systems for abattoirs. Reviewing and summarizing the relevant standard and legislation landscape should also offer some instrumental help regarding the foreseen certification procedure of meat processing robots and robot cells for slaughterhouses in the near future.
Scaling up Action Through Collective Engagement with Environmental Data
Venue: arXiv cs.HC 2023
Year: 2023

Sustainability has over the past two decades emerged as a key concern in human-computer interaction, with a much critiqued focus on quantification and eco-feedback. This approach fits within a modernist framing of sustainability, treating the environment (and our impact on it) as an externality, reducing it to a set of simple metrics. While data about the climate impact of our actions provide an important indication of harm, such data is fragmented and incomplete, capturing only a partial picture of a very wicked and entangled problem. My doctoral research departs from this notion of "information will solve the problem" and through design-oriented explorations of environmental data such as CO2 emissions from academic flying, I investigate alternative ways to engage people with environmental data in order to unsettle relations to the climate impact of our actions and foster care. So far, I have studied this through design-oriented case studies of data in action, with a specific focus on interventions aimed at engaging people in social contexts with the carbon emissions of everyday practices.
Large Scale Genealogical Information Extraction From Handwritten Quebec Parish Records
Venue: arXiv cs.CV 2023
Year: 2023

This paper presents a complete workflow designed for extracting information from Quebec handwritten parish registers. The acts in these documents contain individual and family information highly valuable for genetic, demographic and social studies of the Quebec population. From an image of parish records, our workflow is able to identify the acts and extract personal information. The workflow is divided into successive steps: page classification, text line detection, handwritten text recognition, named entity recognition and act detection and classification. For all these steps, different machine learning models are compared. Once the information is extracted, validation rules designed by experts are then applied to standardize the extracted information and ensure its consistency with the type of act (birth, marriage, and death). This validation step is able to reject records that are considered invalid or merged. The full workflow has been used to process over two million pages of Quebec parish registers from the 19-20th centuries. On a sample comprising 65% of registers, 3.2 million acts were recognized. Verification of the birth and death acts from this sample shows that 74% of them are considered complete and valid. These records will be integrated into the BALSAC database and linked together to recreate family and genealogical relations at large scale.
Origin Tracing and Detecting of LLMs
Venue: arXiv cs.CL 2023
Year: 2023

The extraordinary performance of large language models (LLMs) heightens the importance of detecting whether the context is generated by an AI system. More importantly, while more and more companies and institutions release their LLMs, the origin can be hard to trace. Since LLMs are heading towards the time of AGI, similar to the origin tracing in anthropology, it is of great importance to trace the origin of LLMs. In this paper, we first raise the concern of the origin tracing of LLMs and propose an effective method to trace and detect AI-generated contexts. We introduce a novel algorithm that leverages the contrastive features between LLMs and extracts model-wise features to trace the text origins. Our proposed method works under both white-box and black-box settings therefore can be widely generalized to detect various LLMs.(e.g. can be generalized to detect GPT-3 models without the GPT-3 models). Also, our proposed method requires only limited data compared with the supervised learning methods and can be extended to trace new-coming model origins. We construct extensive experiments to examine whether we can trace the origins of given texts. We provide valuable observations based on the experimental results, such as the difficulty level of AI origin tracing, and the AI origin similarities, and call for ethical concerns of LLM providers. We are releasing all codes and data as a toolkit and benchmark for future AI origin tracing and detecting studies. \footnote{We are releasing all available resource at \url{https://github.com/OpenLMLab/}.}
Exploiting Inductive Bias in Transformer for Point Cloud Classification and Segmentation
Venue: arXiv cs.CV 2023
Year: 2023

Discovering inter-point connection for efficient high-dimensional feature extraction from point coordinate is a key challenge in processing point cloud. Most existing methods focus on designing efficient local feature extractors while ignoring global connection, or vice versa. In this paper, we design a new Inductive Bias-aided Transformer (IBT) method to learn 3D inter-point relations, which considers both local and global attentions. Specifically, considering local spatial coherence, local feature learning is performed through Relative Position Encoding and Attentive Feature Pooling. We incorporate the learned locality into the Transformer module. The local feature affects value component in Transformer to modulate the relationship between channels of each point, which can enhance self-attention mechanism with locality based channel interaction. We demonstrate its superiority experimentally on classification and segmentation tasks. The code is available at: https://github.com/jiamang/IBT
Improved Online Scheduling of Moldable Task Graphs under Common Speedup Models
Venue: arXiv cs.DS 2023
Year: 2023

We consider the online scheduling problem of moldable task graphs on multiprocessor systems for minimizing the overall completion time (or makespan). Moldable job scheduling has been widely studied in the literature, in particular when tasks have dependencies (i.e., task graphs) or when tasks are released on-the-fly (i.e., online). However, few studies have focused on both (i.e., online scheduling of moldable task graphs). In this paper, we design a new online scheduling algorithm for this problem and derive constant competitive ratios under several common yet realistic speedup models (i.e., roofline, communication, Amdahl, and a general combination). These results improve the ones we have shown in the preliminary version of this paper. We also prove, for each speedup model, a lower bound on the competitiveness of any online list scheduling algorithm that allocates processors to a task based only on the task's parameters and not on its position in the graph. This lower bound matches exactly the competitive ratio of our algorithm for the roofline, communication and Amdahl's model, and is close to the ratio for the general model. Finally, we provide a lower bound on the competitive ratio of any deterministic online algorithm for the arbitrary speedup model, which is not constant but depends on the number of tasks in the longest path of the graph.
Categorification of Group Equivariant Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

We present a novel application of category theory for deep learning. We show how category theory can be used to understand and work with the linear layer functions of group equivariant neural networks whose layers are some tensor power space of $\mathbb{R}^{n}$ for the groups $S_n$, $O(n)$, $Sp(n)$, and $SO(n)$. By using category theoretic constructions, we build a richer structure that is not seen in the original formulation of these neural networks, leading to new insights. In particular, we outline the development of an algorithm for quickly computing the result of a vector that is passed through an equivariant, linear layer for each group in question. The success of our approach suggests that category theory could be beneficial for other areas of deep learning.
ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task
Venue: arXiv cs.CL 2023
Year: 2023

Transformer-based language models, including ChatGPT, have demonstrated exceptional performance in various natural language generation tasks. However, there has been limited research evaluating ChatGPT's keyphrase generation ability, which involves identifying informative phrases that accurately reflect a document's content. This study seeks to address this gap by comparing ChatGPT's keyphrase generation performance with state-of-the-art models, while also testing its potential as a solution for two significant challenges in the field: domain adaptation and keyphrase generation from long documents. We conducted experiments on six publicly available datasets from scientific articles and news domains, analyzing performance on both short and long documents. Our results show that ChatGPT outperforms current state-of-the-art models in all tested datasets and environments, generating high-quality keyphrases that adapt well to diverse domains and document lengths.
Simultaneously Transmitting And Reflecting (STAR) RIS for 6G: Fundamentals, Recent Advances, and Future Directions
Venue: arXiv cs.IT 2023
Year: 2023

Simultaneously transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) have been attracting significant attention in both academia and industry for their advantages of achieving 360° coverage and enhanced degrees of freedom. This article first identifies the fundamentals of STAR-RIS, by discussing the hardware models, channel models, and signal models. Then, three representative categorizing approaches for STAR-RIS are introduced from phase-shift, directional, and energy consumption perspectives. Furthermore, the beamforming design of STAR-RIS is investigated for both independent and coupled phase-shift cases. A general optimization framework is proposed as the recent advances, which has high compatibility and provable optimality regardless of the application scenarios. As a further advance, several promising applications are discussed to demonstrate the potential benefits of applying STAR-RIS in the sixth-generation wireless network. Lastly, a few future directions and research opportunities are highlighted for motivating future work.
Quadric Representations for LiDAR Odometry, Mapping and Localization
Venue: arXiv cs.CV 2023
Year: 2023

Current LiDAR odometry, mapping and localization methods leverage point-wise representations of 3D scenes and achieve high accuracy in autonomous driving tasks. However, the space-inefficiency of methods that use point-wise representations limits their development and usage in practical applications. In particular, scan-submap matching and global map representation methods are restricted by the inefficiency of nearest neighbor searching (NNS) for large-volume point clouds. To improve space-time efficiency, we propose a novel method of describing scenes using quadric surfaces, which are far more compact representations of 3D objects than conventional point clouds. In contrast to point cloud-based methods, our quadric representation-based method decomposes a 3D scene into a collection of sparse quadric patches, which improves storage efficiency and avoids the slow point-wise NNS process. Our method first segments a given point cloud into patches and fits each of them to a quadric implicit function. Each function is then coupled with other geometric descriptors of the patch, such as its center position and covariance matrix. Collectively, these patch representations fully describe a 3D scene, which can be used in place of the original point cloud and employed in LiDAR odometry, mapping and localization algorithms. We further design a novel incremental growing method for quadric representations, which eliminates the need to repeatedly re-fit quadric surfaces from the original point cloud. Extensive odometry, mapping and localization experiments on large-volume point clouds in the KITTI and UrbanLoco datasets demonstrate that our method maintains low latency and memory utility while achieving competitive, and even superior, accuracy.