OpenRaiser/Intern-Atlas huggingface dataset:
Tools and methods for Human-Autonomy Teaming: Contributions to cognitive state monitoring and system adaptation
Venue: arXiv cs.HC 2023
Year: 2023

The Human-Autonomy Teaming paradigm (HAT) has recently emerged to model and design hybrid teams, where a human operator must cooperate with an artificial agent, able to independently evolve in dynamic and uncertain situations. An important challenge in HAT is to transform autonomous systems into better teammates, capable of joining humans in highly interdependent activities. The presented works explore two main avenues, supported by industrial collaborations (in the domain of transportation and industrial systems), academic partnerships (especially with South Australian universities), and with the supervision PhD students. The first axis deals with the monitoring of cognitive states, to equip the machine with an ability to detect when human face difficulties. To address this question, a global approach is proposed to classify operators mental workload from the fusion of multisourced physiological and behavioral data. The second axis focused on the mechanisms for adapting human-autonomy teaming, making machine more compatible with human. Two kinds of solution are explored. One focused on the offline enhancement of the know-how-to-cooperate of machines, with the aid of CWA method and MDE techniques. The other deals with online adaptation of human-machine cooperation, where autonomous system can be considered inside the team - as a teammate - or above the team-as a coach. Finally, new research directions are opened, supported by ongoing initiatives in France and abroad. These perspectives relate to the consolidation of a multilevel approach for cognitive state monitoring, the building of a transparent dialogue between human and autonomy, a deeper consideration of transitional and longitudinal situations in HAT, and the scale-up challenge of studying HAT with human teams.
Policy Learning for Active Target Tracking over Continuous SE(3) Trajectories
Venue: arXiv cs.RO 2023
Year: 2023

This paper proposes a novel model-based policy gradient algorithm for tracking dynamic targets using a mobile robot, equipped with an onboard sensor with limited field of view. The task is to obtain a continuous control policy for the mobile robot to collect sensor measurements that reduce uncertainty in the target states, measured by the target distribution entropy. We design a neural network control policy with the robot $SE(3)$ pose and the mean vector and information matrix of the joint target distribution as inputs and attention layers to handle variable numbers of targets. We also derive the gradient of the target entropy with respect to the network parameters explicitly, allowing efficient model-based policy gradient optimization.
Online Estimation of the Koopman Operator Using Fourier Features
Venue: arXiv cs.RO 2023
Year: 2023

Transfer operators offer linear representations and global, physically meaningful features of nonlinear dynamical systems. Discovering transfer operators, such as the Koopman operator, require careful crafted dictionaries of observables, acting on states of the dynamical system. This is ad hoc and requires the full dataset for evaluation. In this paper, we offer an optimization scheme to allow joint learning of the observables and Koopman operator with online data. Our results show we are able to reconstruct the evolution and represent the global features of complex dynamical systems.
XTENTH-CAR: A Proportionally Scaled Experimental Vehicle Platform for Connected Autonomy and All-Terrain Research
Venue: arXiv cs.RO 2023
Year: 2023

Connected Autonomous Vehicles (CAVs) are key components of the Intelligent Transportation System (ITS), and all-terrain Autonomous Ground Vehicles (AGVs) are indispensable tools for a wide range of applications such as disaster response, automated mining, agriculture, military operations, search and rescue missions, and planetary exploration. Experimental validation is a requisite for CAV and AGV research, but requires a large, safe experimental environment when using full-size vehicles which is time-consuming and expensive. To address these challenges, we developed XTENTH-CAR (eXperimental one-TENTH scaled vehicle platform for Connected autonomy and All-terrain Research), an open-source, cost-effective proportionally one-tenth scaled experimental vehicle platform governed by the same physics as a full-size on-road vehicle. XTENTH-CAR is equipped with the best-in-class NVIDIA Jetson AGX Orin System on Module (SOM), stereo camera, 2D LiDAR and open-source Electronic Speed Controller (ESC) with drivers written for both versions of the Robot Operating System (ROS 1 & ROS 2) to facilitate experimental CAV and AGV perception, motion planning and control research, that incorporate state-of-the-art computationally expensive algorithms such as Deep Reinforcement Learning (DRL). XTENTH-CAR is designed for compact experimental environments, and aims to increase the accessibility of experimental CAV and AGV research with low upfront costs, and complete Autonomous Vehicle (AV) hardware and software architectures similar to the full-sized X-CAR experimental vehicle platform, enabling efficient cross-platform development between small-scale and full-scale vehicles.
Self-supervised AutoFlow
Venue: arXiv cs.CV 2023
Year: 2023

Recently, AutoFlow has shown promising results on learning a training set for optical flow, but requires ground truth labels in the target domain to compute its search metric. Observing a strong correlation between the ground truth search metric and self-supervised losses, we introduce self-supervised AutoFlow to handle real-world videos without ground truth labels. Using self-supervised loss as the search metric, our self-supervised AutoFlow performs on par with AutoFlow on Sintel and KITTI where ground truth is available, and performs better on the real-world DAVIS dataset. We further explore using self-supervised AutoFlow in the (semi-)supervised setting and obtain competitive results against the state of the art.
Can Evolutionary Clustering Have Theoretical Guarantees?
Venue: arXiv cs.NE 2023
Year: 2023

Clustering is a fundamental problem in many areas, which aims to partition a given data set into groups based on some distance measure, such that the data points in the same group are similar while that in different groups are dissimilar. Due to its importance and NP-hardness, a lot of methods have been proposed, among which evolutionary algorithms are a class of popular ones. Evolutionary clustering has found many successful applications, but all the results are empirical, lacking theoretical support. This paper fills this gap by proving that the approximation performance of the GSEMO (a simple multi-objective evolutionary algorithm) for solving four formulations of clustering, i.e., $k$-tMM, $k$-center, discrete $k$-median and $k$-means, can be theoretically guaranteed. Furthermore, we consider clustering under fairness, which tries to avoid algorithmic bias, and has recently been an important research topic in machine learning. We prove that for discrete $k$-median clustering under individual fairness, the approximation performance of the GSEMO can be theoretically guaranteed with respect to both the objective function and the fairness constraint.
Generative Models for Improved Naturalness, Intelligibility, and Voicing of Whispered Speech
Venue: arXiv cs.SD 2023
Year: 2023

This work adapts two recent architectures of generative models and evaluates their effectiveness for the conversion of whispered speech to normal speech. We incorporate the normal target speech into the training criterion of vector-quantized variational autoencoders (VQ-VAEs) and MelGANs, thereby conditioning the systems to recover voiced speech from whispered inputs. Objective and subjective quality measures indicate that both VQ-VAEs and MelGANs can be modified to perform the conversion task. We find that the proposed approaches significantly improve the Mel cepstral distortion (MCD) metric by at least 25% relative to a DiscoGAN baseline. Subjective listening tests suggest that the MelGAN-based system significantly improves naturalness, intelligibility, and voicing compared to the whispered input speech. A novel evaluation measure based on differences between latent speech representations also indicates that our MelGAN-based approach yields improvements relative to the baseline.
Differential uniformity properties of some classes of permutation polynomials
Venue: arXiv cs.IT 2023
Year: 2023

The notion of $c$-differential uniformity has recently received a lot of attention since its proposal~\cite{Ellingsen}, and recently a characterization of perfect $c$-nonlinear functions in terms of difference sets in some quasigroups was obtained in~\cite{AMS22}. Independent of their applications as a measure for certain statistical biases, the construction of functions, especially permutations, with low $c$-differential uniformity is an interesting mathematical problem in this area, and recent work has focused heavily in this direction. We provide a few classes of permutation polynomials with low $c$-differential uniformity. The used technique involves handling various Weil sums, as well as analyzing some equations in finite fields, and we believe these can be of independent interest.
Integrating Intelligent Reflecting Surface into Base Station: Architecture, Channel Model, and Passive Reflection Design
Venue: arXiv cs.IT 2023
Year: 2023

Existing works on IRS have mainly considered IRS being deployed in the environment to dynamically control the wireless channels between the BS and its served users. In contrast, we propose in this paper a new integrated IRS BS architecture by deploying IRSs inside the BS antenna radome. Since the distance between the integrated IRSs and BS antenna array is practically small, the path loss among them is significantly reduced and the real time control of the IRS reflection by the BS becomes easier to implement. However, the resultant near field channel model also becomes drastically different. Thus, we propose an element wise channel model for IRS to characterize the channel vector between each single antenna user and the antenna array of the BS, which includes the direct (without any IRS reflection) as well as the single and double IRS-reflection channel components. Then, we formulate a problem to optimize the reflection coefficients of all IRS reflecting elements for maximizing the uplink sum rate of the users. By considering two typical cases with/without perfect CSI at the BS, the formulated problem is solved efficiently by adopting the successive refinement method and iterative random phase algorithm (IRPA), respectively. Numerical results validate the substantial capacity gain of the integrated IRS BS architecture over the conventional multi antenna BS without integrated IRS. Moreover, the proposed algorithms significantly outperform other benchmark schemes in terms of sum rate, and the IRPA without CSI can approach the performance upper bound with perfect CSI as the training overhead increases.
Dataset vs Reality: Understanding Model Performance from the Perspective of Information Need
Venue: arXiv cs.IR 2023
Year: 2023

Deep learning technologies have brought us many models that outperform human beings on a few benchmarks. An interesting question is: can these models well solve real-world problems with similar settings (e.g., identical input/output) to the benchmark datasets? We argue that a model is trained to answer the same information need for which the training dataset is created. Although some datasets may share high structural similarities, e.g., question-answer pairs for the question answering (QA) task and image-caption pairs for the image captioning (IC) task, they may represent different research tasks aiming for answering different information needs. To support our argument, we use the QA task and IC task as two case studies and compare their widely used benchmark datasets. From the perspective of information need in the context of information retrieval, we show the differences in the dataset creation processes, and the differences in morphosyntactic properties between datasets. The differences in these datasets can be attributed to the different information needs of the specific research tasks. We encourage all researchers to consider the information need the perspective of a research task before utilizing a dataset to train a model. Likewise, while creating a dataset, researchers may also incorporate the information need perspective as a factor to determine the degree to which the dataset accurately reflects the research task they intend to tackle.
An Index Policy for Minimizing the Uncertainty-of-Information of Markov Sources
Venue: arXiv cs.IT 2023
Year: 2023

This paper focuses on the information freshness of finite-state Markov sources, using the uncertainty of information (UoI) as the performance metric. Measured by Shannon's entropy, UoI can capture not only the transition dynamics of the Markov source but also the different evolutions of information quality caused by the different values of the last observation. We consider an information update system with M finite-state Markov sources transmitting information to a remote monitor via m communication channels. Our goal is to explore the optimal scheduling policy to minimize the sum-UoI of the Markov sources. The problem is formulated as a restless multi-armed bandit (RMAB). We relax the RMAB and then decouple the relaxed problem into M single bandit problems. Analyzing the single bandit problem provides useful properties with which the relaxed problem reduces to maximizing a concave and piecewise linear function, allowing us to develop a gradient method to solve the relaxed problem and obtain its optimal policy. By rounding up the optimal policy for the relaxed problem, we obtain an index policy for the original RMAB problem. Notably, the proposed index policy is universal in the sense that it applies to general RMABs with bounded cost functions.
Political Honeymoon Effect on Social Media: Characterizing Social Media Reaction to the Changes of Prime Minister in Japan
Venue: arXiv cs.SI 2023
Year: 2023

New leaders in democratic countries typically enjoy high approval ratings immediately after taking office. This phenomenon is called the honeymoon effect and is regarded as a significant political phenomenon; however, its mechanism remains underexplored. Therefore, this study examines how social media users respond to changes in political leadership in order to better understand the honeymoon effect in politics. In particular, we constructed a 15-year Twitter dataset on eight change timings of Japanese prime ministers consisting of 6.6M tweets and analyzed them in terms of sentiments, topics, and users. We found that, while not always, social media tend to show a honeymoon effect at the change timings of prime minister. The study also revealed that sentiment about prime ministers differed by topic, indicating that public expectations vary from one prime minister to another. Furthermore, the user base was largely replaced before and after the change in the prime minister, and their sentiment was also significantly different. The implications of this study would be beneficial for administrative management.
Space and Move-optimal Arbitrary Pattern Formation on a Rectangular Grid by Robot Swarms
Venue: arXiv cs.DC 2023
Year: 2023

Arbitrary pattern formation (\textsc{Apf}) is a well-studied problem in swarm robotics. To the best of our knowledge, the problem has been considered in two different settings: one in a euclidean plane and another in an infinite grid. This work deals with the problem in an infinite rectangular grid setting. The previous works in literature dealing with the \textsc{Apf} problem in an infinite grid had a fundamental issue. These deterministic algorithms use a lot of space in the grid to solve the problem, mainly to maintain the asymmetry of the configuration or to avoid a collision. These solution techniques cannot be useful if there is a space constraint in the application field. In this work, we consider luminous robots (with one light that can take three colors) to avoid symmetry, but we carefully designed a deterministic algorithm that solves the \textsc{Apf} problem using the minimal required space in the grid. The robots are autonomous, identical, and anonymous, and they operate in Look-Compute-Move cycles under a fully asynchronous scheduler. The \textsc{Apf} algorithm proposed in \cite{BOSE2020} by Bose et al. can be modified using luminous robots so that it uses minimal space, but that algorithm is not move-optimal. The algorithm proposed in this paper not only uses minimal space but is also asymptotically move-optimal. The algorithm proposed in this work is designed for an infinite rectangular grid, but it can be easily modified to work on a finite grid as well.
State Space Closure: Revisiting Endless Online Level Generation via Reinforcement Learning
Venue: arXiv cs.AI 2023
Year: 2023

In this paper, we revisit endless online level generation with the recently proposed experience-driven procedural content generation via reinforcement learning (EDRL) framework. Inspired by an observation that EDRL tends to generate recurrent patterns, we formulate a notion of state space closure which makes any stochastic state appeared possibly in an infinite-horizon online generation process can be found within a finite-horizon. Through theoretical analysis, we find that even though state space closure arises a concern about diversity, it generalises EDRL trained with a finite-horizon to the infinite-horizon scenario without deterioration of content quality. Moreover, we verify the quality and the diversity of contents generated by EDRL via empirical studies, on the widely used Super Mario Bros. benchmark. Experimental results reveal that the diversity of levels generated by EDRL is limited due to the state space closure, whereas their quality does not deteriorate in a horizon which is longer than the one specified in the training. Concluding our outcomes and analysis, future work on endless online level generation via reinforcement learning should address the issue of diversity while assuring the occurrence of state space closure and quality.
Denoising diffusion probabilistic models for probabilistic energy forecasting
Venue: arXiv cs.LG 2023
Year: 2023

Scenario-based probabilistic forecasts have become vital for decision-makers in handling intermittent renewable energies. This paper presents a recent promising deep learning generative approach called denoising diffusion probabilistic models. It is a class of latent variable models which have recently demonstrated impressive results in the computer vision community. However, to our knowledge, there has yet to be a demonstration that they can generate high-quality samples of load, PV, or wind power time series, crucial elements to face the new challenges in power systems applications. Thus, we propose the first implementation of this model for energy forecasting using the open data of the Global Energy Forecasting Competition 2014. The results demonstrate this approach is competitive with other state-of-the-art deep learning generative models, including generative adversarial networks, variational autoencoders, and normalizing flows.
SODA: A Natural Language Processing Package to Extract Social Determinants of Health for Cancer Studies
Venue: arXiv cs.CL 2023
Year: 2023

Objective: We aim to develop an open-source natural language processing (NLP) package, SODA (i.e., SOcial DeterminAnts), with pre-trained transformer models to extract social determinants of health (SDoH) for cancer patients, examine the generalizability of SODA to a new disease domain (i.e., opioid use), and evaluate the extraction rate of SDoH using cancer populations. Methods: We identified SDoH categories and attributes and developed an SDoH corpus using clinical notes from a general cancer cohort. We compared four transformer-based NLP models to extract SDoH, examined the generalizability of NLP models to a cohort of patients prescribed with opioids, and explored customization strategies to improve performance. We applied the best NLP model to extract 19 categories of SDoH from the breast (n=7,971), lung (n=11,804), and colorectal cancer (n=6,240) cohorts. Results and Conclusion: We developed a corpus of 629 cancer patients notes with annotations of 13,193 SDoH concepts/attributes from 19 categories of SDoH. The Bidirectional Encoder Representations from Transformers (BERT) model achieved the best strict/lenient F1 scores of 0.9216 and 0.9441 for SDoH concept extraction, 0.9617 and 0.9626 for linking attributes to SDoH concepts. Fine-tuning the NLP models using new annotations from opioid use patients improved the strict/lenient F1 scores from 0.8172/0.8502 to 0.8312/0.8679. The extraction rates among 19 categories of SDoH varied greatly, where 10 SDoH could be extracted from >70% of cancer patients, but 9 SDoH had a low extraction rate (<70% of cancer patients). The SODA package with pre-trained transformer models is publicly available at https://github.com/uf-hobiinformatics-lab/SDoH_SODA.
Causal Inference via Style Transfer for Out-of-distribution Generalisation
Venue: arXiv cs.CV 2023
Year: 2023

Out-of-distribution (OOD) generalisation aims to build a model that can generalise well on an unseen target domain using knowledge from multiple source domains. To this end, the model should seek the causal dependence between inputs and labels, which may be determined by the semantics of inputs and remain invariant across domains. However, statistical or non-causal methods often cannot capture this dependence and perform poorly due to not considering spurious correlations learnt from model training via unobserved confounders. A well-known existing causal inference method like back-door adjustment cannot be applied to remove spurious correlations as it requires the observation of confounders. In this paper, we propose a novel method that effectively deals with hidden confounders by successfully implementing front-door adjustment (FA). FA requires the choice of a mediator, which we regard as the semantic information of images that helps access the causal mechanism without the need for observing confounders. Further, we propose to estimate the combination of the mediator with other observed images in the front-door formula via style transfer algorithms. Our use of style transfer to estimate FA is novel and sensible for OOD generalisation, which we justify by extensive experimental results on widely used benchmark datasets.
Univalent Monoidal Categories
Venue: arXiv cs.LO 2023
Year: 2023

Univalent categories constitute a well-behaved and useful notion of category in univalent foundations. The notion of univalence has subsequently been generalized to bicategories and other structures in (higher) category theory. Here, we zoom in on monoidal categories and study them in a univalent setting. Specifically, we show that the bicategory of univalent monoidal categories is univalent. Furthermore, we construct a Rezk completion for monoidal categories: we show how any monoidal category is weakly equivalent to a univalent monoidal category, universally. We have fully formalized these results in UniMath, a library of univalent mathematics in the Coq proof assistant.