Miracle-dz/UniFAR-FTUs huggingface dataset:
Multi-Agent Reinforcement Learning for Cooperative Air Transportation Services in City-Wide Autonomous Urban Air Mobility

The development of urban-air-mobility (UAM) is rapidly progressing with spurs, and the demand for efficient transportation management systems is a rising need due to the multifaceted environmental uncertainties. Thus, this article proposes a novel air transportation service management algorithm based on multi-agent deep reinforcement learning (MADRL) to address the challenges of multi-UAM cooperation. Specifically, the proposed algorithm in this article is based on communication network (CommNet) method utilizing centralized training and distributed execution (CTDE) in multiple UAMs for providing efficient air transportation services to passengers collaboratively. Furthermore, this article adopts actual vertiport maps and UAM specifications for constructing realistic air transportation networks. By evaluating the performance of the proposed algorithm in data-intensive simulations, the results show that the proposed algorithm outperforms existing approaches in terms of air transportation service quality. Furthermore, there are no inferior UAMs by utilizing parameter sharing in CommNet and a centralized critic network in CTDE. Therefore, it can be confirmed that the research results in this article can provide a promising solution for autonomous air transportation management systems in city-wide urban areas.
Bounded cohomology property on a smooth projective surface with Picard number two

Abstract We say a smooth projective surface X satisfies the bounded cohomology property if there exists a positive constant such that for every prime divisor C on X. Let the closed Mori cone such that and with are some curves on X. If either (i) the Kodaira dimension or (ii) , the irregularity and the Iitaka dimension , then we prove that X satisfies the bounded cohomology property.
ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models

Humor is a central aspect of human communication that has not been solved for artificial agents so far. Large language models (LLMs) are increasingly able to capture implicit and contextual information. Especially, OpenAI’s ChatGPT recently gained immense public attention. The GPT3-based model almost seems to communicate on a human level and can even tell jokes. Humor is an essential component of human communication. But is ChatGPT really funny? We put ChatGPT’s sense of humor to the test. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT’s capability to grasp and reproduce human humor. Since the model itself is not accessible, we applied prompt-based experiments. Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model. Over 90% of 1008 generated jokes were the same 25 Jokes. The system accurately explains valid jokes but also comes up with fictional explanations for invalid jokes. Joke-typical characteristics can mislead ChatGPT in the classification of jokes. ChatGPT has not solved computational humor yet but it can be a big leap toward “funny” machines.
Can current NLI systems handle German word order? Investigating language model performance on a new German challenge set of minimal pairs

Compared to English, German word order is freer and therefore poses additional challenges for natural language inference (NLI). We create WOGLI (Word Order in German Language Inference), the first adversarial NLI dataset for German word order that has the following properties: (i) each premise has an entailed and a non-entailed hypothesis; (ii) premise and hypotheses differ only in word order and necessary morphological changes to mark case and number. In particular, each premise and its two hypotheses contain exactly the same lemmata. Our adversarial examples require the model to use morphological markers in order to recognise or reject entailment. We show that current German autoencoding models fine-tuned on translated NLI data can struggle on this challenge set, reflecting the fact that translated NLI datasets will not mirror all necessary language phenomena in the target language. We also examine performance after data augmentation as well as on related word order phenomena derived from WOGLI. Our datasets are publically available at https://github.com/ireinig/wogli.
Point Cloud Video Anomaly Detection Based on Point Spatio-Temporal Auto-Encoder

Video anomaly detection has great potential in enhancing safety in the production and monitoring of crucial areas. Currently, most video anomaly detection methods are based on RGB modality, but its redundant semantic information may breach the privacy of residents or patients. The 3D data obtained by depth camera and LiDAR can accurately locate anomalous events in 3D space while preserving human posture and motion information. Identifying individuals through the point cloud is difficult due to its sparsity, which protects personal privacy. In this study, we propose Point Spatio-Temporal Auto-Encoder (PSTAE), an autoencoder framework that uses point cloud videos as input to detect anomalies in point cloud videos. We introduce PSTOp and PSTTransOp to maintain spatial geometric and temporal motion information in point cloud videos. To measure the reconstruction loss of the proposed autoencoder framework, we propose a reconstruction loss measurement strategy based on a shallow feature extractor. Experimental results on the TIMo dataset show that our method outperforms currently representative depth modality-based methods in terms of AUROC and has superior performance in detecting Medical Issue anomalies. These results suggest the potential of point cloud modality in video anomaly detection. Our method sets a new state-of-the-art (SOTA) on the TIMo dataset.
Cosmological significance of the early bright galaxies observed with JWST

The recent discovery of objects with redshift $z>10$ with the help of James Webb Space Telescope (JWST) poses serious challenges to the $\Lambda$CDM cosmological model, which has been in vogue for some time now. The new data indicate that galaxy formation must have taken place much earlier than expected in this model. Another viable class of cosmological models is that of the so-called coasting models, in which the scale factor of the universe varies proportionately with time. In these models, the universe at redshift $z=12$ has ample time ($\sim 1070$ Myrs) for galaxy formation. The earliest such model is the one proposed by E.A. Milne, based on his `kinematic relativity', but it is considered unrealistic for not treating gravity as relevant at cosmological scales. A closed version of an eternal coasting FLRW model was proposed by the present authors even before SNe Ia data began to pour in. Subsequently we developed a more general model of the same class, which is valid for all the three possible geometries, with open, closed or flat spatial sections. In the nonrelativistic era, this model makes the falsifiable prediction that the ratio of matter density to dark energy density is 2. This avoids the cosmic coincidence problem. Moreover, this eternal coasting model allows room for creation of matter from dark energy, that may speed up galaxy and structure formation at the early epochs, as implied by the JWST data. The paper also attempts to review some similar coasting models, but emphasizes the eternal coasting cosmology as the most probable candidate model capable of explaining the presence of high redshift galaxies discovered by JWST.
Professional Basketball Player Behavior Synthesis via Planning with Diffusion

Dynamically planning in multi-agent systems has been explored to improve decision-making in various domains. Professional basketball serves as a compelling example of a dynamic spatio-temporal game, encompassing both concealed strategic policies and decision-making. However, processing the diverse on-court signals and navigating the vast space of potential actions and outcomes makes it difficult for existing approaches to swiftly identify optimal strategies in response to evolving circumstances. In this study, we first formulate the sequential decision-making process as a conditional trajectory generation process. We further introduce PLAYBEST (PLAYer BEhavior SynThesis), a method for enhancing player decision-making. We extend the state-of-the-art generative model, diffusion probabilistic model, to learn challenging multi-agent environmental dynamics from historical National Basketball Association (NBA) player motion tracking data. To incorporate data-driven strategies, an auxiliary value function is trained using the play-by-play data with corresponding rewards acting as the plan guidance. To accomplish reward-guided trajectory generation, conditional sampling is introduced to condition the diffusion model on the value function and conduct classifier-guided sampling. We validate the effectiveness of PLAYBEST via comprehensive simulation studies from real-world data, contrasting the generated trajectories and play strategies with those employed by professional basketball teams. Our results reveal that the model excels at generating high-quality basketball trajectories that yield efficient plays, surpassing conventional planning techniques in terms of adaptability, flexibility, and overall performance. Moreover, the synthesized play strategies exhibit a remarkable alignment with professional tactics, highlighting the model's capacity to capture the intricate dynamics of basketball games.
Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks

In this work, we reveal a strong implicit bias of stochastic gradient descent (SGD) that drives overly expressive networks to much simpler subnetworks, thereby dramatically reducing the number of independent parameters, and improving generalization. To reveal this bias, we identify invariant sets, or subsets of parameter space that remain unmodified by SGD. We focus on two classes of invariant sets that correspond to simpler (sparse or low-rank) subnetworks and commonly appear in modern architectures. Our analysis uncovers that SGD exhibits a property of stochastic attractivity towards these simpler invariant sets. We establish a sufficient condition for stochastic attractivity based on a competition between the loss landscape's curvature around the invariant set and the noise introduced by stochastic gradients. Remarkably, we find that an increased level of noise strengthens attractivity, leading to the emergence of attractive invariant sets associated with saddle-points or local maxima of the train loss. We observe empirically the existence of attractive invariant sets in trained deep neural networks, implying that SGD dynamics often collapses to simple subnetworks with either vanishing or redundant neurons. We further demonstrate how this simplifying process of stochastic collapse benefits generalization in a linear teacher-student framework. Finally, through this analysis, we mechanistically explain why early training with large learning rates for extended periods benefits subsequent generalization.
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning

Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding is limited. In this work, we provide evidence that disentangled representations coupled with sparse base-predictors improve generalization. In the context of multi-task learning, we prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem. Finally, we explore a meta-learning version of this algorithm based on group Lasso multiclass SVM base-predictors, for which we derive a tractable dual formulation. It obtains competitive results on standard few-shot classification benchmarks, while each task is using only a fraction of the learned representations.
Dual policy as self-model for planning

Planning is a data efficient decision-making strategy where an agent selects candidate actions by exploring possible future states. To simulate future states when there is a high-dimensional action space, the knowledge of one's decision making strategy must be used to limit the number of actions to be explored. We refer to the model used to simulate one's decisions as the agent's self-model. While self-models are implicitly used widely in conjunction with world models to plan actions, it remains unclear how self-models should be designed. Inspired by current reinforcement learning approaches and neuroscience, we explore the benefits and limitations of using a distilled policy network as the self-model. In such dual-policy agents, a model-free policy and a distilled policy are used for model-free actions and planned actions, respectively. Our results on a ecologically relevant, parametric environment indicate that distilled policy network for self-model stabilizes training, has faster inference than using model-free policy, promotes better exploration, and could learn a comprehensive understanding of its own behaviors, at the cost of distilling a new network apart from the model-free policy.
ECQED: Emotion-Cause Quadruple Extraction in Dialogs

The existing emotion-cause pair extraction (ECPE) task, unfortunately, ignores extracting the emotion type and cause type, while these fine-grained meta-information can be practically useful in real-world applications, i.e., chat robots and empathic dialog generation. Also the current ECPE is limited to the scenario of single text piece, while neglecting the studies at dialog level that should have more realistic values. In this paper, we extend the ECPE task with a broader definition and scenario, presenting a new task, Emotion-Cause Quadruple Extraction in Dialogs (ECQED), which requires detecting emotion-cause utterance pairs and emotion and cause types. We present an ECQED model based on a structural and semantic heterogeneous graph as well as a parallel grid tagging scheme, which advances in effectively incorporating the dialog context structure, meanwhile solving the challenging overlapped quadruple issue. Via experiments we show that introducing the fine-grained emotion and cause features evidently helps better dialog generation. Also our proposed ECQED system shows exceptional superiority over baselines on both the emotion-cause quadruple or pair extraction tasks, meanwhile being highly efficient.
Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications

In many machine learning systems that jointly learn from multiple modalities, a core research question is to understand the nature of multimodal interactions: the emergence of new task-relevant information during learning from both modalities that was not present in either alone. We study this challenge of interaction quantification in a semi-supervised setting with only labeled unimodal data and naturally co-occurring multimodal data (e.g., unlabeled images and captions, video and corresponding audio) but when labeling them is time-consuming. Using a precise information-theoretic definition of interactions, our key contributions are the derivations of lower and upper bounds to quantify the amount of multimodal interactions in this semi-supervised setting. We propose two lower bounds based on the amount of shared information between modalities and the disagreement between separately trained unimodal classifiers, and derive an upper bound through connections to approximate algorithms for min-entropy couplings. We validate these estimated bounds and show how they accurately track true interactions. Finally, two semi-supervised multimodal applications are explored based on these theoretical results: (1) analyzing the relationship between multimodal performance and estimated interactions, and (2) self-supervised learning that embraces disagreement between modalities beyond agreement as is typically done.
The lifted functional approach to mean field games with common noise

We introduce a new path-by-path approach to mean field games with common noise that recovers duality at the pathwise level. We verify this perspective by explicitly solving some difficult examples with linear-quadratic data, including control in the volatility coefficient of the common noise as well as the constraint of partial information. As an application, we establish the celebrated separation principle in the latter context. In pursuing this program, we believe we have made a crucial contribution to clarifying the notion of regular solution in the path dependent PDE literature.
Label Shift Quantification with Robustness Guarantees via Distribution Feature Matching

Quantification learning deals with the task of estimating the target label distribution under label shift. In this paper, we first present a unifying framework, distribution feature matching (DFM), that recovers as particular instances various estimators introduced in previous literature. We derive a general performance bound for DFM procedures, improving in several key aspects upon previous bounds derived in particular cases. We then extend this analysis to study robustness of DFM procedures in the misspecified setting under departure from the exact label shift hypothesis, in particular in the case of contamination of the target by an unknown distribution. These theoretical findings are confirmed by a detailed numerical study on simulated and real-world datasets. We also introduce an efficient, scalable and robust version of kernel-based DFM using the Random Fourier Feature principle.
Contrastive Bootstrapping for Label Refinement

Traditional text classification typically categorizes texts into pre-defined coarse-grained classes, from which the produced models cannot handle the real-world scenario where finer categories emerge periodically for accurate services. In this work, we investigate the setting where fine-grained classification is done only using the annotation of coarse-grained categories and the coarse-to-fine mapping. We propose a lightweight contrastive clustering-based bootstrapping method to iteratively refine the labels of passages. During clustering, it pulls away negative passage-prototype pairs under the guidance of the mapping from both global and local perspectives. Experiments on NYT and 20News show that our method outperforms the state-of-the-art methods by a large margin.
Experimental demonstration of scalable quantum key distribution over a thousand kilometers

Secure communication over long distances is one of the major problems of modern informatics. Classical transmissions are recognized to be vulnerable to quantum computer attacks. Remarkably, the same quantum mechanics that engenders quantum computers offers guaranteed protection against such attacks via quantum key distribution (QKD). Yet, long-distance transmission is problematic since the essential signal decay in optical channels occurs at a distance of about a hundred kilometers. We propose to resolve this problem by a QKD protocol, further referred to as the Terra Quantum QKD protocol (TQ-QKD protocol). In our protocol, we use semiclassical pulses containing enough photons for random bit encoding and exploiting erbium amplifiers to retranslate photon pulses and, at the same time, ensuring that at the chosen pulse intensity only a few photons could go outside the channel even at distances of about a hundred meters. As a result, an eavesdropper will not be able to efficiently utilize the lost part of the signal. The central component of the TQ-QKD protocol is the end-to-end loss control of the fiber-optic communication line since optical losses can in principle be used by the eavesdropper to obtain the transmitted information. However, our control precision is such that if the degree of the leak is below the detectable level, then the leaking states are quantum since they contain only a few photons. Therefore, available to the eavesdropper parts of the bit encoding states representing `0' and `1' are nearly indistinguishable. Our work presents the experimental demonstration of the TQ-QKD protocol allowing quantum key distribution over 1079 kilometers. Further refining the quality of the scheme's components will expand the attainable transmission distances. This paves the way for creating a secure global QKD network in the upcoming years.
Competing relaxation channels in continuously polydisperse fluids: A mode-coupling study

Systems with a high degree of size polydispersity are becoming standard in the computational study of deeply supercooled liquids. In this work we perform a systematic analysis of continuously polydisperse fluids as a function of the degree of polydispersity within the framework of the Mode-Coupling Theory of the glass transition (MCT). Our results show that a high degree of polydispersity tends to stabilize the liquid phase against vitrification, the magnitude of which depends on the shape of the polydispersity distribution. Further, we report on a separation between the localization lengths of the smallest and largest particles. A diameter-resolved analysis of the intermediate scattering functions reveals that this separation significantly stretches the relaxation patterns, which we quantitatively study by an analysis of the dynamical exponents predicted by the theory. Our observations have strong implications for our understanding of the nature of dynamical heterogeneities and localization lengths in continuously polydisperse systems. These results suggest that the dynamics of the smallest particles is of central importance to understand structural relaxation of continuously size polydisperse fluids, already in the mildly supercooled regime where MCT is usually applicable.
Asymptotic Structure of Higher Dimensional Yang-Mills Theory

Using the covariant phase space formalism, we construct the phase space for non-Abelian gauge theories in $(d+2)$-dimensional Minkowski spacetime for any $d \geq 2$, including the edge modes that symplectically pair to the low energy degrees of freedom of the gauge field. Despite the fact that the symplectic form in odd and even-dimensional spacetimes appear ostensibly different, we demonstrate that both cases can be treated in a unified manner by utilizing the shadow transform. Upon quantization, we recover the algebra of the vacuum sector of the Hilbert space and derive a Ward identity that implies the leading soft gluon theorem in $(d+2)$-dimensional spacetime.