OpenRaiser/Intern-Atlas huggingface dataset:
Automatic Diary Generation System including Information on Joint Experiences between Humans and Robots
Venue: arXiv cs.RO 2023
Year: 2023

In this study, we propose an automatic diary generation system that uses information from past joint experiences with the aim of increasing the favorability for robots through shared experiences between humans and robots. For the verbalization of the robot's memory, the system applies a large-scale language model, which is a rapidly developing field. Since this model does not have memories of experiences, it generates a diary by receiving information from joint experiences. As an experiment, a robot and a human went for a walk and generated a diary with interaction and dialogue history. The proposed diary achieved high scores in comfort and performance in the evaluation of the robot's impression. In the survey of diaries giving more favorable impressions, diaries with information on joint experiences were selected higher than diaries without such information, because diaries with information on joint experiences showed more cooperation between the robot and the human and more intimacy from the robot.
Electro-Chemo-Mechanical Modeling of Multiscale Active Materials for Next-Generation Energy Storage: Opportunities and Challenges
Venue: arXiv cs.CE 2023
Year: 2023

The recent geopolitical crisis resulted in a gas price surge. Although lithium-ion batteries represent the best available rechargeable battery technology, a significant energy and power density gap exists between LIBs and petrol/gasoline. The battery electrodes comprise a mixture of active materials particles, conductive carbon, and binder additives deposited onto a current collector. Although this basic design has persisted for decades, the active material particle's desired size scale is debated. Traditionally, microparticles have been used in batteries. Advances in nanotechnology have spurred interest in deploying nanoparticles as active materials. However, despite many efforts in nano, industries still primarily use 'old' microparticles. Most importantly, the battery industry is unlikely to replace microstructures with nanometer-sized analogs. This poses an important question: Is there a place for nanostructure in battery design due to irreplaceable microstructure? The way forward lies in multiscale active materials, microscale structures with built-in nanoscale features, such as microparticles assembled from nanoscale building blocks or patterned with engineered or natural nanopores. Although experimental strides have been made in developing such materials, computational progress in this domain remains limited and, in some cases, negligible. However, the fields hold immense computational potential, presenting a multitude of opportunities. This perspective highlights the existing gaps in modeling multiscale active materials and delineates various open challenges in the realm of electro-chemo-mechanical modeling. By doing so, it aims to inspire computational research within this field and promote synergistic collaborative efforts between computational and experimental researchers.
A Survey on Interpretable Cross-modal Reasoning
Venue: arXiv cs.AI 2023
Year: 2023

In recent years, cross-modal reasoning (CMR), the process of understanding and reasoning across different modalities, has emerged as a pivotal area with applications spanning from multimedia analysis to healthcare diagnostics. As the deployment of AI systems becomes more ubiquitous, the demand for transparency and comprehensibility in these systems' decision-making processes has intensified. This survey delves into the realm of interpretable cross-modal reasoning (I-CMR), where the objective is not only to achieve high predictive performance but also to provide human-understandable explanations for the results. This survey presents a comprehensive overview of the typical methods with a three-level taxonomy for I-CMR. Furthermore, this survey reviews the existing CMR datasets with annotations for explanations. Finally, this survey summarizes the challenges for I-CMR and discusses potential future directions. In conclusion, this survey aims to catalyze the progress of this emerging research area by providing researchers with a panoramic and comprehensive perspective, illuminating the state of the art and discerning the opportunities. The summarized methods, datasets, and other resources are available at https://github.com/ZuyiZhou/Awesome-Interpretable-Cross-modal-Reasoning.
The $\ell$-intersection Pairs of Constacyclic and Conjucyclic Codes
Venue: arXiv cs.IT 2023
Year: 2023

A pair of linear codes whose intersection is of dimension $\ell$, where $\ell$ is a non-negetive integer, is called an $\ell$-intersection pair of codes. This paper focuses on studying $\ell$-intersection pairs of $λ_i$-constacyclic, $i=1,2,$ and conjucyclic codes. We first characterize an $\ell$-intersection pair of $λ_i$-constacyclic codes. A formula for $\ell$ has been established in terms of the degrees of the generator polynomials of $λ_i$-constacyclic codes. This allows obtaining a condition for $\ell$-linear complementary pairs (LPC) of constacyclic codes. Later, we introduce and characterize the $\ell$-intersection pair of conjucyclic codes over $\mathbb{F}_{q^2}$. The first observation in the process is that there are no non-trivial linear conjucyclic codes over finite fields. So focus on the characterization of additive conjucyclic (ACC) codes. We show that the largest $\mathbb{F}_q$-subcode of an ACC code over $\mathbb{F}_{q^2}$ is cyclic and obtain its generating polynomial. This enables us to find the size of an ACC code. Furthermore, we discuss the trace code of an ACC code and show that it is cyclic. Finally, we determine $\ell$-intersection pairs of trace codes of ACC codes over $\mathbb{F}_4$.
Topologies for Error-Detecting Variable-Length Codes
Venue: arXiv cs.DM 2023
Year: 2023

Given a finite alphabet $A$, a quasi-metric $d$ over $A^*$, and a non-negative integer $k$, we introduce the relation $τ_{d,k}\subseteq A^*\times A^*$ such that $(x,y)\inτ_{d,k}$ holds whenever $d(x,y)\le k$. The error detection capability of variable-length codes is expressed in term of conditions over $τ_{d,k}$. With respect to the prefix metric, the factor one, and any quasi-metric associated with some free monoid (anti-)automorphism, we prove that one can decide whether a given regular variable-length code satisfies any of those error detection constraints.
On the peer review reports: It's not the size that matters ... really?
Venue: arXiv cs.DL 2023
Year: 2023

Scientometers and sociologists of science have spilled much ink on the topic of peer review over the past twenty years, given its primordial role in a context marked by the exponential growth of scientific production and the proliferation of predatory journals. Although the topic is addressed under different prisms, few studies have empirically analyzed to what extent it can affect the quality of publications. Here we study the link between the length of reviewers' reports and the citations received by publications. To do this, we used data from the Publons database (58,093 peer review reports). We have adjusted this sample to match the WoS database structure. Our regression results show that peer review positively affects the quality of publications. In other words, the more indepth (longer) the referees' reports are, the greater the publication improvements will be, resulting in an increase in citations received. This result is important from both the point of view of reviewers and that of journal's chiefseditors. Even if it is not a remunerated activity, it is important that it be more valued at least within the framework of research evaluation exercises, given its positive impact on science.
Non-Parametric Representation Learning with Kernels
Venue: arXiv cs.LG 2023
Year: 2023

Unsupervised and self-supervised representation learning has become popular in recent years for learning useful features from unlabelled data. Representation learning has been mostly developed in the neural network literature, and other models for representation learning are surprisingly unexplored. In this work, we introduce and analyze several kernel-based representation learning approaches: Firstly, we define two kernel Self-Supervised Learning (SSL) models using contrastive loss functions and secondly, a Kernel Autoencoder (AE) model based on the idea of embedding and reconstructing data. We argue that the classical representer theorems for supervised kernel machines are not always applicable for (self-supervised) representation learning, and present new representer theorems, which show that the representations learned by our kernel models can be expressed in terms of kernel matrices. We further derive generalisation error bounds for representation learning with kernel SSL and AE, and empirically evaluate the performance of these methods in both small data regimes as well as in comparison with neural network based models.
Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion
Venue: arXiv cs.CV 2023
Year: 2023

RGB-guided depth completion aims at predicting dense depth maps from sparse depth measurements and corresponding RGB images, where how to effectively and efficiently exploit the multi-modal information is a key issue. Guided dynamic filters, which generate spatially-variant depth-wise separable convolutional filters from RGB features to guide depth features, have been proven to be effective in this task. However, the dynamically generated filters require massive model parameters, computational costs and memory footprints when the number of feature channels is large. In this paper, we propose to decompose the guided dynamic filters into a spatially-shared component multiplied by content-adaptive adaptors at each spatial location. Based on the proposed idea, we introduce two decomposition schemes A and B, which decompose the filters by splitting the filter structure and using spatial-wise attention, respectively. The decomposed filters not only maintain the favorable properties of guided dynamic filters as being content-dependent and spatially-variant, but also reduce model parameters and hardware costs, as the learned adaptors are decoupled with the number of feature channels. Extensive experimental results demonstrate that the methods using our schemes outperform state-of-the-art methods on the KITTI dataset, and rank 1st and 2nd on the KITTI benchmark at the time of submission. Meanwhile, they also achieve comparable performance on the NYUv2 dataset. In addition, our proposed methods are general and could be employed as plug-and-play feature fusion blocks in other multi-modal fusion tasks such as RGB-D salient object detection.
On the Age of Information of Processor Sharing Systems
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we examine the Age of Information (AoI) of a source sending status updates to a monitor through a queue operating under the Processor Sharing (PS) discipline. In the PS queueing discipline, all the updates are served simultaneously and, therefore, none of of the jobs wait in the queue to get service. While AoI has been well studied for various queuing models and policies, less attention has been given so far to the PS discipline. We first consider the M/M/1/2 queue with and without preemption and provide closed-form expressions for the average AoI in this case. We overcome the challenges of deriving the AoI expression by employing the Stochastic Hybrid Systems (SHS) tool. We then extend the analysis to the M/M/1 queue with one and two sources and provide numerical results for these cases. Our results show that PS can outperform the M/M/1/1* queue in some cases.
Dual Adversarial Alignment for Realistic Support-Query Shift Few-shot Learning
Venue: arXiv cs.CV 2023
Year: 2023

Support-query shift few-shot learning aims to classify unseen examples (query set) to labeled data (support set) based on the learned embedding in a low-dimensional space under a distribution shift between the support set and the query set. However, in real-world scenarios the shifts are usually unknown and varied, making it difficult to estimate in advance. Therefore, in this paper, we propose a novel but more difficult challenge, RSQS, focusing on Realistic Support-Query Shift few-shot learning. The key feature of RSQS is that the individual samples in a meta-task are subjected to multiple distribution shifts in each meta-task. In addition, we propose a unified adversarial feature alignment method called DUal adversarial ALignment framework (DuaL) to relieve RSQS from two aspects, i.e., inter-domain bias and intra-domain variance. On the one hand, for the inter-domain bias, we corrupt the original data in advance and use the synthesized perturbed inputs to train the repairer network by minimizing distance in the feature level. On the other hand, for intra-domain variance, we proposed a generator network to synthesize hard, i.e., less similar, examples from the support set in a self-supervised manner and introduce regularized optimal transportation to derive a smooth optimal transportation plan. Lastly, a benchmark of RSQS is built with several state-of-the-art baselines among three datasets (CIFAR100, mini-ImageNet, and Tiered-Imagenet). Experiment results show that DuaL significantly outperforms the state-of-the-art methods in our benchmark.
Exploiting Spatial-temporal Data for Sleep Stage Classification via Hypergraph Learning
Venue: arXiv cs.LG 2023
Year: 2023

Sleep stage classification is crucial for detecting patients' health conditions. Existing models, which mainly use Convolutional Neural Networks (CNN) for modelling Euclidean data and Graph Convolution Networks (GNN) for modelling non-Euclidean data, are unable to consider the heterogeneity and interactivity of multimodal data as well as the spatial-temporal correlation simultaneously, which hinders a further improvement of classification performance. In this paper, we propose a dynamic learning framework STHL, which introduces hypergraph to encode spatial-temporal data for sleep stage classification. Hypergraphs can construct multi-modal/multi-type data instead of using simple pairwise between two subjects. STHL creates spatial and temporal hyperedges separately to build node correlations, then it conducts type-specific hypergraph learning process to encode the attributes into the embedding space. Extensive experiments show that our proposed STHL outperforms the state-of-the-art models in sleep stage classification tasks.
Exploring the Intersection of Complex Aesthetics and Generative AI for Promoting Cultural Creativity in Rural China after the Post-Pandemic Era
Venue: arXiv cs.CY 2023
Year: 2023

This paper explores using generative AI and aesthetics to promote cultural creativity in rural China amidst COVID-19's impact. Through literature reviews, case studies, surveys, and text analysis, it examines art and technology applications in rural contexts and identifies key challenges. The study finds artworks often fail to resonate locally, while reliance on external artists limits sustainability. Hence, nurturing grassroots "artist villagers" through AI is proposed. Our approach involves training machine learning on subjective aesthetics to generate culturally relevant content. Interactive AI media can also boost tourism while preserving heritage. This pioneering research puts forth original perspectives on the intersection of AI and aesthetics to invigorate rural culture. It advocates holistic integration of technology and emphasizes AI's potential as a creative enabler versus replacement. Ultimately, it lays the groundwork for further exploration of leveraging AI innovations to empower rural communities. This timely study contributes to growing interest in emerging technologies to address critical issues facing rural China.
Model-based Offline Policy Optimization with Adversarial Network
Venue: arXiv cs.LG 2023
Year: 2023

Model-based offline reinforcement learning (RL), which builds a supervised transition model with logging dataset to avoid costly interactions with the online environment, has been a promising approach for offline policy optimization. As the discrepancy between the logging data and online environment may result in a distributional shift problem, many prior works have studied how to build robust transition models conservatively and estimate the model uncertainty accurately. However, the over-conservatism can limit the exploration of the agent, and the uncertainty estimates may be unreliable. In this work, we propose a novel Model-based Offline policy optimization framework with Adversarial Network (MOAN). The key idea is to use adversarial learning to build a transition model with better generalization, where an adversary is introduced to distinguish between in-distribution and out-of-distribution samples. Moreover, the adversary can naturally provide a quantification of the model's uncertainty with theoretical guarantees. Extensive experiments showed that our approach outperforms existing state-of-the-art baselines on widely studied offline RL benchmarks. It can also generate diverse in-distribution samples, and quantify the uncertainty more accurately.
The Adversarial Implications of Variable-Time Inference
Venue: arXiv cs.CR 2023
Year: 2023

Machine learning (ML) models are known to be vulnerable to a number of attacks that target the integrity of their predictions or the privacy of their training data. To carry out these attacks, a black-box adversary must typically possess the ability to query the model and observe its outputs (e.g., labels). In this work, we demonstrate, for the first time, the ability to enhance such decision-based attacks. To accomplish this, we present an approach that exploits a novel side channel in which the adversary simply measures the execution time of the algorithm used to post-process the predictions of the ML model under attack. The leakage of inference-state elements into algorithmic timing side channels has never been studied before, and we have found that it can contain rich information that facilitates superior timing attacks that significantly outperform attacks based solely on label outputs. In a case study, we investigate leakage from the non-maximum suppression (NMS) algorithm, which plays a crucial role in the operation of object detectors. In our examination of the timing side-channel vulnerabilities associated with this algorithm, we identified the potential to enhance decision-based attacks. We demonstrate attacks against the YOLOv3 detector, leveraging the timing leakage to successfully evade object detection using adversarial examples, and perform dataset inference. Our experiments show that our adversarial examples exhibit superior perturbation quality compared to a decision-based attack. In addition, we present a new threat model in which dataset inference based solely on timing leakage is performed. To address the timing leakage vulnerability inherent in the NMS algorithm, we explore the potential and limitations of implementing constant-time inference passes as a mitigation strategy.
PCFGaze: Physics-Consistent Feature for Appearance-based Gaze Estimation
Venue: arXiv cs.CV 2023
Year: 2023

Although recent deep learning based gaze estimation approaches have achieved much improvement, we still know little about how gaze features are connected to the physics of gaze. In this paper, we try to answer this question by analyzing the gaze feature manifold. Our analysis revealed the insight that the geodesic distance between gaze features is consistent with the gaze differences between samples. According to this finding, we construct the Physics- Consistent Feature (PCF) in an analytical way, which connects gaze feature to the physical definition of gaze. We further propose the PCFGaze framework that directly optimizes gaze feature space by the guidance of PCF. Experimental results demonstrate that the proposed framework alleviates the overfitting problem and significantly improves cross-domain gaze estimation accuracy without extra training data. The insight of gaze feature has the potential to benefit other regression tasks with physical meanings.
Blockchain-assisted Twin Migration for Vehicular Metaverses: A Game Theory Approach
Venue: arXiv cs.GT 2023
Year: 2023

As the fusion of automotive industry and metaverse, vehicular metaverses establish a bridge between the physical space and virtual space, providing intelligent transportation services through the integration of various technologies, such as extended reality and real-time rendering technologies, to offer immersive metaverse services for Vehicular Metaverse Users (VMUs). In vehicular metaverses, VMUs update vehicle twins (VTs) deployed in RoadSide Units (RSUs) to obtain metaverse services. However, due to the mobility of vehicles and the limited service coverage of RSUs, VT migration is necessary to ensure continuous immersive experiences for VMUs. This process requires RSUs to contribute resources for enabling efficient migration, which leads to a resource trading problem between RSUs and VMUs. Moreover, a single RSU cannot support large-scale VT migration. To this end, we propose a blockchain-assisted game approach framework for reliable VT migration in vehicular metaverses. Based on the subject logic model, we first calculate the reputation values of RSUs considering the freshness of interaction between RSUs and VMUs. Then, a coalition game based on the reputation values of RSUs is formulated, and RSU coalitions are formed to jointly provide bandwidth resources for reliable and large-scale VT migration. Subsequently, the RSU coalition with the highest utility is selected. Finally, to incentivize VMUs to participate in VT migration, we propose a Stackelberg model between the selected coalition and VMUs. Numerical results demonstrate the reliability and effectiveness of the proposed schemes.
A Quantitative Method to Determine What Collisions Are Reasonably Foreseeable and Preventable
Venue: arXiv cs.RO 2023
Year: 2023

The development of Automated Driving Systems (ADSs) has made significant progress in the last years. To enable the deployment of Automated Vehicles (AVs) equipped with such ADSs, regulations concerning the approval of these systems need to be established. In 2021, the World Forum for Harmonization of Vehicle Regulations has approved a new United Nations regulation concerning the approval of Automated Lane Keeping Systems (ALKSs). An important aspect of this regulation is that "the activated system shall not cause any collisions that are reasonably foreseeable and preventable." The phrasing of "reasonably foreseeable and preventable" might be subjected to different interpretations and, therefore, this might result in disagreements among AV developers and the authorities that are requested to approve AVs. The objective of this work is to propose a method for quantifying what is "reasonably foreseeable and preventable". The proposed method considers the Operational Design Domain (ODD) of the system and can be applied to any ODD. Having a quantitative method for determining what is reasonably foreseeable and preventable provides developers, authorities, and the users of ADSs a better understanding of the residual risks to be expected when deploying these systems in real traffic. Using our proposed method, we can estimate what collisions are reasonably foreseeable and preventable. This will help in setting requirements regarding the safety of ADSs and can lead to stronger justification for design decisions and test coverage for developing ADSs.