OpenRaiser/Intern-Atlas huggingface dataset:
Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language
Venue: arXiv cs.AI 2023
Year: 2023

Reinforcement learning is a powerful technique for learning from trial and error, but it often requires a large number of interactions to achieve good performance. In some domains, such as sparse-reward tasks, an oracle that can provide useful feedback or guidance to the agent during the learning process is really of great importance. However, querying the oracle too frequently may be costly or impractical, and the oracle may not always have a clear answer for every situation. Therefore, we propose a novel method for interacting with the oracle in a selective and efficient way, using a retrieval-based approach. We assume that the interaction can be modeled as a sequence of templated questions and answers, and that there is a large corpus of previous interactions available. We use a neural network to encode the current state of the agent and the oracle, and retrieve the most relevant question from the corpus to ask the oracle. We then use the oracle's answer to update the agent's policy and value function. We evaluate our method on an object manipulation task. We show that our method can significantly improve the efficiency of RL by reducing the number of interactions needed to reach a certain level of performance, compared to baselines that do not use the oracle or use it in a naive way.
2DDATA: 2D Detection Annotations Transmittable Aggregation for Semantic Segmentation on Point Cloud
Venue: arXiv cs.CV 2023
Year: 2023

Recently, multi-modality models have been introduced because of the complementary information from different sensors such as LiDAR and cameras. It requires paired data along with precise calibrations for all modalities, the complicated calibration among modalities hugely increases the cost of collecting such high-quality datasets, and hinder it from being applied to practical scenarios. Inherit from the previous works, we not only fuse the information from multi-modality without above issues, and also exhaust the information in the RGB modality. We introduced the 2D Detection Annotations Transmittable Aggregation(\textbf{2DDATA}), designing a data-specific branch, called \textbf{Local Object Branch}, which aims to deal with points in a certain bounding box, because of its easiness of acquiring 2D bounding box annotations. We demonstrate that our simple design can transmit bounding box prior information to the 3D encoder model, proving the feasibility of large multi-modality models fused with modality-specific data.
FGFusion: Fine-Grained Lidar-Camera Fusion for 3D Object Detection
Venue: arXiv cs.CV 2023
Year: 2023

Lidars and cameras are critical sensors that provide complementary information for 3D detection in autonomous driving. While most prevalent methods progressively downscale the 3D point clouds and camera images and then fuse the high-level features, the downscaled features inevitably lose low-level detailed information. In this paper, we propose Fine-Grained Lidar-Camera Fusion (FGFusion) that make full use of multi-scale features of image and point cloud and fuse them in a fine-grained way. First, we design a dual pathway hierarchy structure to extract both high-level semantic and low-level detailed features of the image. Second, an auxiliary network is introduced to guide point cloud features to better learn the fine-grained spatial information. Finally, we propose multi-scale fusion (MSF) to fuse the last N feature maps of image and point cloud. Extensive experiments on two popular autonomous driving benchmarks, i.e. KITTI and Waymo, demonstrate the effectiveness of our method.
The Undecidability of Third Order Pattern Matching in Calculi with Dependent Types or Type Constructors
Venue: arXiv cs.LO 2023
Year: 2023

We prove the undecidability of the third order pattern matching problem in typed lambda-calculi with dependent types and in those with type constructors by reducing the second order unification problem to them.
Evaluating Large Language Models for Document-grounded Response Generation in Information-Seeking Dialogues
Venue: arXiv cs.CL 2023
Year: 2023

In this paper, we investigate the use of large language models (LLMs) like ChatGPT for document-grounded response generation in the context of information-seeking dialogues. For evaluation, we use the MultiDoc2Dial corpus of task-oriented dialogues in four social service domains previously used in the DialDoc 2022 Shared Task. Information-seeking dialogue turns are grounded in multiple documents providing relevant information. We generate dialogue completion responses by prompting a ChatGPT model, using two methods: Chat-Completion and LlamaIndex. ChatCompletion uses knowledge from ChatGPT model pretraining while LlamaIndex also extracts relevant information from documents. Observing that document-grounded response generation via LLMs cannot be adequately assessed by automatic evaluation metrics as they are significantly more verbose, we perform a human evaluation where annotators rate the output of the shared task winning system, the two Chat-GPT variants outputs, and human responses. While both ChatGPT variants are more likely to include information not present in the relevant segments, possibly including a presence of hallucinations, they are rated higher than both the shared task winning system and human responses.
Joint Beamforming for RIS Aided Full-Duplex Integrated Sensing and Uplink Communication
Venue: arXiv cs.IT 2023
Year: 2023

This paper studies integrated sensing and communication (ISAC) technology in a full-duplex (FD) uplink communication system. As opposed to the half-duplex system, where sensing is conducted in a first-emit-then-listen manner, FD ISAC system emits and listens simultaneously and hence conducts uninterrupted target sensing. Besides, impressed by the recently emerging reconfigurable intelligent surface (RIS) technology, we also employ RIS to improve the self-interference (SI) suppression and signal processing gain. As will be seen, the joint beamforming, RIS configuration and mobile users' power allocation is a difficult optimization problem. To resolve this challenge, via leveraging the cutting-the-edge majorization-minimization (MM) and penalty-dual-decomposition (PDD) methods, we develop an iterative solution that optimizes all variables via using convex optimization techniques. Numerical results demonstrate the effectiveness of our proposed solution and the great benefit of employing RIS in the FD ISAC system.
DEYOv3: DETR with YOLO for Real-time Object Detection
Venue: arXiv cs.CV 2023
Year: 2023

Recently, end-to-end object detectors have gained significant attention from the research community due to their outstanding performance. However, DETR typically relies on supervised pretraining of the backbone on ImageNet, which limits the practical application of DETR and the design of the backbone, affecting the model's potential generalization ability. In this paper, we propose a new training method called step-by-step training. Specifically, in the first stage, the one-to-many pre-trained YOLO detector is used to initialize the end-to-end detector. In the second stage, the backbone and encoder are consistent with the DETR-like model, but only the detector needs to be trained from scratch. Due to this training method, the object detector does not need the additional dataset (ImageNet) to train the backbone, which makes the design of the backbone more flexible and dramatically reduces the training cost of the detector, which is helpful for the practical application of the object detector. At the same time, compared with the DETR-like model, the step-by-step training method can achieve higher accuracy than the traditional training method of the DETR-like model. With the aid of this novel training method, we propose a brand-new end-to-end real-time object detection model called DEYOv3. DEYOv3-N achieves 41.1% on COCO val2017 and 270 FPS on T4 GPU, while DEYOv3-L achieves 51.3% AP and 102 FPS. Without the use of additional training data, DEYOv3 surpasses all existing real-time object detectors in terms of both speed and accuracy. It is worth noting that for models of N, S, and M scales, the training on the COCO dataset can be completed using a single 24GB RTX3090 GPU. Code will be released at https://github.com/ouyanghaodong/DEYOv3.
BitCoin: Bidirectional Tagging and Supervised Contrastive Learning based Joint Relational Triple Extraction Framework
Venue: arXiv cs.CL 2023
Year: 2023

Relation triple extraction (RTE) is an essential task in information extraction and knowledge graph construction. Despite recent advancements, existing methods still exhibit certain limitations. They just employ generalized pre-trained models and do not consider the specificity of RTE tasks. Moreover, existing tagging-based approaches typically decompose the RTE task into two subtasks, initially identifying subjects and subsequently identifying objects and relations. They solely focus on extracting relational triples from subject to object, neglecting that once the extraction of a subject fails, it fails in extracting all triples associated with that subject. To address these issues, we propose BitCoin, an innovative Bidirectional tagging and supervised Contrastive learning based joint relational triple extraction framework. Specifically, we design a supervised contrastive learning method that considers multiple positives per anchor rather than restricting it to just one positive. Furthermore, a penalty term is introduced to prevent excessive similarity between the subject and object. Our framework implements taggers in two directions, enabling triples extraction from subject to object and object to subject. Experimental results show that BitCoin achieves state-of-the-art results on the benchmark datasets and significantly improves the F1 score on Normal, SEO, EPO, and multiple relation extraction tasks.
Syntactic Variation Across the Grammar: Modelling a Complex Adaptive System
Venue: arXiv cs.CL 2023
Year: 2023

While language is a complex adaptive system, most work on syntactic variation observes a few individual constructions in isolation from the rest of the grammar. This means that the grammar, a network which connects thousands of structures at different levels of abstraction, is reduced to a few disconnected variables. This paper quantifies the impact of such reductions by systematically modelling dialectal variation across 49 local populations of English speakers in 16 countries. We perform dialect classification with both an entire grammar as well as with isolated nodes within the grammar in order to characterize the syntactic differences between these dialects. The results show, first, that many individual nodes within the grammar are subject to variation but, in isolation, none perform as well as the grammar as a whole. This indicates that an important part of syntactic variation consists of interactions between different parts of the grammar. Second, the results show that the similarity between dialects depends heavily on the sub-set of the grammar being observed: for example, New Zealand English could be more similar to Australian English in phrasal verbs but at the same time more similar to UK English in dative phrases.
A critical review of mobile device-to-device communication
Venue: arXiv cs.HC 2023
Year: 2023

Since the advent of mobile devices, both end-users and the IT industry have been longing for direct device-to-device (D2D) communication capabilities, expecting new kinds of interactive, personalized, and collaborative services. Fifteen years later, many D2D solutions have been implemented and deployed, but their availability and functionality are underwhelming. Arguably, the most widely-adopted D2D use case covers the pairing of accessories with smartphones; however, many other use cases-such as mobile media sharing-did not progress. Pervasive computing and cyber-physical convergence need local communication paradigms to scale. For inherently local use cases, they are even more appealing than ever: eschewing third-parties simultaneously fosters environmental sustainability, privacy and network resiliency. This paper proposes a survey on D2D communication, investigates its deployment and adoption, with the objective of easing the creation and adoption of modern D2D frameworks. We present the results of an online poll that estimates end-users' utilisation of D2D processes, and review enabling technologies and security models.
Using Saliency and Cropping to Improve Video Memorability
Venue: arXiv cs.CV 2023
Year: 2023

Video memorability is a measure of how likely a particular video is to be remembered by a viewer when that viewer has no emotional connection with the video content. It is an important characteristic as videos that are more memorable are more likely to be shared, viewed, and discussed. This paper presents results of a series of experiments where we improved the memorability of a video by selectively cropping frames based on image saliency. We present results of a basic fixed cropping as well as the results from dynamic cropping where both the size of the crop and the position of the crop within the frame, move as the video is played and saliency is tracked. Our results indicate that especially for videos of low initial memorability, the memorability score can be improved.
The Spatiotemporal Scaling Laws of Bitcoin Transactions
Venue: arXiv cs.SI 2023
Year: 2023

This study, to the best of our knowledge for the first time, delves into the spatiotemporal dynamics of Bitcoin transactions, shedding light on the scaling laws governing its geographic usage. Leveraging a dataset of IP addresses and Bitcoin addresses spanning from October 2013 to December 2013, we explore the geospatial patterns unique to Bitcoin. Motivated by the needs of cryptocurrency businesses, regulatory clarity, and network science inquiries, we make several contributions. Firstly, we empirically characterize Bitcoin transactions' spatiotemporal scaling laws, providing insights into its spending behaviours. Secondly, we introduce a Markovian model that effectively approximates Bitcoin's observed spatiotemporal patterns, revealing economic connections among user groups in the Bitcoin ecosystem. Our measurements and model shed light on the inhomogeneous structure of the network: although Bitcoin is designed to be decentralized, there are significant geographical differences in the distribution of user activity, which has consequences for all participants and possible (regulatory) control over the system.
Towards In-Cabin Monitoring: A Preliminary Study on Sensors Data Collection and Analysis
Venue: arXiv cs.ET 2023
Year: 2023

The last decade's market has been characterized by wearable devices, mainly smartwatches, edge, and cloud computing. A possible application of these technologies is to improve the safety of dangerous activities, especially driving motor vehicles. Common enabling technologies, such as system-on-chip, ultra-low-power computational platforms, and wide-band wireless connectivity, push all these trends. On the one hand, wearable devices, thanks to the continuous contact with the user's body, can measure physiological parameters. On the other hand, edge computing and machine learning techniques, alongside cameras, allow the implementation of contactless computer vision systems capable of providing information about the user's current behavior. Another trend is the usage of RADARs in automotive applications, both for collision avoidance and monitoring driver behavior. These technologies can be combined to develop systems designed to aid the driver. For the sake of this paper, we are focusing on warning drivers, allowing them to know whenever they are drowsy and hence risking a sleep onset or are not paying attention to the road. Developing such systems poses many challenges, such as automatic classification of physiological signal patterns, facial expression recognition, head movements and eye gaze detection. These challenges have been individually addressed in the literature. Anyway, we noticed a need for more description on implementing data fusion. Two main reasons for adopting the fusion approach are to improve the quality of the overall representation (increasing accuracy and specificity against drowsy) and make a more reliable system due to redundancy.
REM-U-net: Deep Learning Based Agile REM Prediction with Energy-Efficient Cell-Free Use Case
Venue: arXiv cs.NI 2023
Year: 2023

Radio environment maps (REMs) hold a central role in optimizing wireless network deployment, enhancing network performance, and ensuring effective spectrum management. Conventional REM prediction methods are either excessively time-consuming, e.g., ray tracing, or inaccurate, e.g., statistical models, limiting their adoption in modern inherently dynamic wireless networks. Deep-learning-based REM prediction has recently attracted considerable attention as an appealing, accurate, and time-efficient alternative. However, existing works on REM prediction using deep learning are either confined to 2D maps or use a limited dataset. In this paper, we introduce a runtime-efficient REM prediction framework based on u-nets, trained on a large-scale 3D maps dataset. In addition, data preprocessing steps are investigated to further refine the REM prediction accuracy. The proposed u-net framework, along with preprocessing steps, are evaluated in the context of the 2023 IEEE ICASSP Signal Processing Grand Challenge, namely, the First Pathloss Radio Map Prediction Challenge. The evaluation results demonstrate that the proposed method achieves an average normalized root-mean-square error (RMSE) of 0.045 with an average of 14 milliseconds (ms) runtime. Finally, we position our achieved REM prediction accuracy in the context of a relevant cell-free massive multiple-input multiple-output (CF-mMIMO) use case. We demonstrate that one can obviate consuming energy on large-scale fading measurements and rely on predicted REM instead to decide on which sleep access points (APs) to switch on in a CF-mMIMO network that adopts a minimum propagation loss AP switch ON/OFF strategy.
Quantum Microservices Development and Deployment
Venue: arXiv cs.SE 2023
Year: 2023

Early advances in the field of quantum computing have provided new opportunities to tackle intricate problems in areas as diverse as mathematics, physics, or healthcare. However, the technology required to construct such systems where different pieces of quantum and classical software collaborate is currently lacking. For this reason, significant advancements in quantum service-oriented computing are necessary to enable developers to create and operate quantum services and microservices comparable to their classical counterparts. Therefore, the core objective of this work is to establish the necessary technological infrastructure that enables the application of the benefits and lessons learned from service-oriented computing to the domain of quantum software engineering. To this end, we propose a pipeline for the continuous deployment of services. Additionally, we have validated the proposal by making use of a modification of the OpenAPI specification, the GitHub Actions, and AWS.
Video Scene Location Recognition with Neural Networks
Venue: arXiv cs.CV 2023
Year: 2023

This paper provides an insight into the possibility of scene recognition from a video sequence with a small set of repeated shooting locations (such as in television series) using artificial neural networks. The basic idea of the presented approach is to select a set of frames from each scene, transform them by a pre-trained singleimage pre-processing convolutional network, and classify the scene location with subsequent layers of the neural network. The considered networks have been tested and compared on a dataset obtained from The Big Bang Theory television series. We have investigated different neural network layers to combine individual frames, particularly AveragePooling, MaxPooling, Product, Flatten, LSTM, and Bidirectional LSTM layers. We have observed that only some of the approaches are suitable for the task at hand.
Generating Hierarchical Structures for Improved Time Series Classification Using Stochastic Splitting Functions
Venue: arXiv cs.LG 2023
Year: 2023

This study introduces a novel hierarchical divisive clustering approach with stochastic splitting functions (SSFs) to enhance classification performance in multi-class datasets through hierarchical classification (HC). The method has the unique capability of generating hierarchy without requiring explicit information, making it suitable for datasets lacking prior knowledge of hierarchy. By systematically dividing classes into two subsets based on their discriminability according to the classifier, the proposed approach constructs a binary tree representation of hierarchical classes. The approach is evaluated on 46 multi-class time series datasets using popular classifiers (svm and rocket) and SSFs (potr, srtr, and lsoo). The results reveal that the approach significantly improves classification performance in approximately half and a third of the datasets when using rocket and svm as the classifier, respectively. The study also explores the relationship between dataset features and HC performance. While the number of classes and flat classification (FC) score show consistent significance, variations are observed with different splitting functions. Overall, the proposed approach presents a promising strategy for enhancing classification by generating hierarchical structure in multi-class time series datasets. Future research directions involve exploring different splitting functions, classifiers, and hierarchy structures, as well as applying the approach to diverse domains beyond time series data. The source code is made openly available to facilitate reproducibility and further exploration of the method.
Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems: Language Acquisition as a Core for Future Metrics
Venue: arXiv cs.CL 2023
Year: 2023

In the burgeoning field of artificial intelligence (AI), the unprecedented progress of large language models (LLMs) in natural language processing (NLP) offers an opportunity to revisit the entire approach of traditional metrics of machine intelligence, both in form and content. As the realm of machine cognitive evaluation has already reached Imitation, the next step is an efficient Language Acquisition and Understanding. Our paper proposes a paradigm shift from the established Turing Test towards an all-embracing framework that hinges on language acquisition, taking inspiration from the recent advancements in LLMs. The present contribution is deeply tributary of the excellent work from various disciplines, point out the need to keep interdisciplinary bridges open, and delineates a more robust and sustainable approach.
Quantifying Feature Importance of Games and Strategies via Shapley Values
Venue: arXiv cs.MA 2023
Year: 2023

Recent advances in game informatics have enabled us to find strong strategies across a diverse range of games. However, these strategies are usually difficult for humans to interpret. On the other hand, research in Explainable Artificial Intelligence (XAI) has seen a notable surge in scholarly activity. Interpreting strong or near-optimal strategies or the game itself can provide valuable insights. In this paper, we propose two methods to quantify the feature importance using Shapley values: one for the game itself and another for individual AIs. We empirically show that our proposed methods yield intuitive explanations that resonate with and augment human understanding.