Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards autonomous system: flexible modular production system enhanced with large language model agents

In this paper, we present a novel framework that combines large language models (LLMs), digital twins and industrial automation system to enable intelligent planning and control of production processes. We retrofit the automation system for a modular production facility and create executable control interfaces of fine-granular functionalities and coarse-granular skills. Low-level functionalities are executed by automation components, and high-level skills are performed by automation modules. Subsequently, a digital twin system is developed, registering these interfaces and containing additional descriptive information about the production system. Based on the retrofitted automation system and the created digital twins, LLM-agents are designed to interpret descriptive information in the digital twins and control the physical system through service interfaces. These LLM-agents serve as intelligent agents on different levels within an automation system, enabling autonomous planning and control of flexible production. Given a task instruction as input, the LLM-agents orchestrate a sequence of atomic functionalities and skills to accomplish the task. We demonstrate how our implemented prototype can handle un-predefined tasks, plan a production process, and execute the operations. This research highlights the potential of integrating LLMs into industrial automation systems in the context of smart factory for more agile, flexible, and adaptive production processes, while it also underscores the critical insights and limitations for future work. Demos at: https://github.com/YuchenXia/GPT4IndustrialAutomation
Improve Video Representation with Temporal Adversarial Augmentation

Recent works reveal that adversarial augmentation benefits the generalization of neural networks (NNs) if used in an appropriate manner. In this paper, we introduce Temporal Adversarial Augmentation (TA), a novel video augmentation technique that utilizes temporal attention. Unlike conventional adversarial augmentation, TA is specifically designed to shift the attention distributions of neural networks with respect to video clips by maximizing a temporal-related loss function. We demonstrate that TA will obtain diverse temporal views, which significantly affect the focus of neural networks. Training with these examples remedies the flaw of unbalanced temporal information perception and enhances the ability to defend against temporal shifts, ultimately leading to better generalization. To leverage TA, we propose Temporal Video Adversarial Fine-tuning (TAF) framework for improving video representations. TAF is a model-agnostic, generic, and interpretability-friendly training strategy. We evaluate TAF with four powerful models (TSM, GST, TAM, and TPN) over three challenging temporal-related benchmarks (Something-something V1&V2 and diving48). Experimental results demonstrate that TAF effectively improves the test accuracy of these models with notable margins without introducing additional parameters or computational costs. As a byproduct, TAF also improves the robustness under out-of-distribution (OOD) settings. Code is available at https://github.com/jinhaoduan/TAF.
The ACM Multimedia 2023 Computational Paralinguistics Challenge: Emotion Share & Requests

The ACM Multimedia 2023 Computational Paralinguistics Challenge addresses two different problems for the first time in a research competition under well-defined conditions: In the Emotion Share Sub-Challenge, a regression on speech has to be made; and in the Requests Sub-Challenges, requests and complaints need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' ComPaRE features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectRum toolkit; in addition, wav2vec2 models are used.
Octonions and Quantum Gravity through the Central Charge Anomaly in the Clifford Algebra

We derive a theory of quantum gravity containing an AdS$_3$ isometry/qubit duality. The theory is based on a superalgebra generalization of the enveloping algebra of the homogeneous AdS$_3$ spacetime isometry group and is isomorphic to the complexified octonion algebra through canonical quantization. Its first three quaternion generators correspond to an $\hbar$-quantized AdS$_3$ embedded spacetime and its remaining four non-quaternion generators to a $G$-quantized embedding $2+2$ Minkowski spacetime. The quaternion algebra's expression after a monomorphism into the complexified Clifford algebra produces a two-dimensional conformal operator product expansion with a central charge anomaly, which results in an area-law $\hbar G$ scaling satisfying the holographic principle and defines an"arrow of time". This relationship allows us to extend the theory through supersymmetry- and conformal-breaking $\mathcal O(G)$ transformations of the embedding to produce perturbed AdS$_3$ spacetimes and derive a resolution to the black hole information paradox with an explicit mechanism.
Understanding accountability in algorithmic supply chains

Academic and policy proposals on algorithmic accountability often seek to understand algorithmic systems in their socio-technical context, recognising that they are produced by ‘many hands’. Increasingly, however, algorithmic systems are also produced, deployed, and used within a supply chain comprising multiple actors tied together by flows of data between them. In such cases, it is the working together of an algorithmic supply chain of different actors who contribute to the production, deployment, use, and functionality that drives systems and produces particular outcomes. We argue that algorithmic accountability discussions must consider supply chains and the difficult implications they raise for the governance and accountability of algorithmic systems. In doing so, we explore algorithmic supply chains, locating them in their broader technical and political economic context and identifying some key features that should be understood in future work on algorithmic governance and accountability (particularly regarding general purpose AI services). To highlight ways forward and areas warranting attention, we further discuss some implications raised by supply chains: challenges for allocating accountability stemming from distributed responsibility for systems between actors, limited visibility due to the accountability horizon, service models of use and liability, and cross-border supply chains and regulatory arbitrage.
Excitons guided by polaritons

We show that an exciton on a discrete chain of sites can be guided by effective measurements induced by an ambient, non-equilibrium medium that is synchronised to the exciton transport. For experimental verification, we propose a hybrid cold atom platform, carrying the exciton as electronic excitation on a chain of atoms, which are surrounded by a slow light medium supporting polaritons. The chain is coupled to the medium through long-range Rydberg interactions. Despite the guiding mechanism being incoherent, the exciton pulse can be coherently transported with high fidelity, relying on effective quantum non-demolition (QND) measurements. These are enacted by the medium, measuring those parts of the Hilbertspace that are instantaneously not helpful for transport, which are thus effectively removed through a quantum Zeno effect. The implementation requires careful alignment of chain and medium but then no further time-dependent control. In addition to guidance, the same mechanism can also be utilized to incoherently switch coherent transport. Our concept can be ported to other exciton and polariton carrying media or devices, and will enable switches and waveguides operating with the two quasi particles involved, as we demonstrate.
IMP: Iterative Matching and Pose Estimation with Adaptive Pooling

Previous methods solve feature matching and pose estimation using a two-stage process by first finding matches and then estimating the pose. As they ignore the geometric relationships between the two tasks, they focus on either improving the quality of matches or filtering potential outliers, leading to limited efficiency or accuracy. In contrast, we propose an iterative matching and pose estimation framework (IMP) leveraging the geometric connections between the two tasks: a few good matches are enough for a roughly accurate pose estimation; a roughly accurate pose can be used to guide the matching by providing geometric constraints. To this end, we implement a geometry-aware recurrent attention-based module which jointly outputs sparse matches and camera poses. Specifically, for each iteration, we first implicitly embed geometric information into the module via a pose-consistency loss, allowing it to predict geometry-aware matches progressively. Second, we introduce an efficient IMP, called EIMP, to dynamically discard keypoints without potential matches, avoiding redundant updating and significantly reducing the quadratic time complexity of attention computation in transformers. Experiments on YFCC100m, Scannet, and Aachen Day-Night datasets demonstrate that the proposed method outperforms previous approaches in terms of accuracy and efficiency. Code is available at https://github.com/feixue94/imp-release
Identifying Minimal Changes in the Zone Abstract Domain

Verification techniques express program states as logical formulas over program variables. For example, symbolic execution and abstract interpretation encode program states as a set of integer inequalities. However, for real-world programs these formulas tend to become large, which affects scalability of analyses. To address this problem, researchers developed complementary approaches which either remove redundant inequalities or extract a subset of inequalities sufficient for specific reasoning. For arbitrary integer inequalities, such reduction approaches either have high complexities or over-approximate. However, efficiency and precision of these approaches can be improved for a restricted type of logical formulas used in relational numerical abstract domains. While previous work investigated custom efficient redundant inequality elimination for Zones states, our work examines custom semantic slicing algorithms that identify a minimal set of changed inequalities in Zones states. The client application of the minimal changes in Zones is an empirical study on comparison between invariants computed by data-flow analysis using Zones, Intervals and Predicates numerical domains. In particular, evaluations compare how our proposed algorithms affect the precision of comparing Zones vs. Intervals and Zones vs. Predicates abstract domains. The results show our techniques reduce the number of variables by more than 70% and the number of inequalities by 30%, compared to full states. The approach refines the granularity of comparison between domains, reducing incomparable invariants between Zones and Predicates from 52% to 4%, and increases equality of Intervals and Zones, invariants from 27% to 71%. The techniques improve the comparison efficiency by reducing total runtime for all subject comparisons for Zones and Predicates from over 4 minutes to a few seconds.
Information Redundancy and Biases in Public Document Information Extraction Benchmarks

Advances in the Visually-rich Document Understanding (VrDU) field and particularly the Key-Information Extraction (KIE) task are marked with the emergence of efficient Transformer-based approaches such as the LayoutLM models. Despite the good performance of KIE models when fine-tuned on public benchmarks, they still struggle to generalize on complex real-life use-cases lacking sufficient document annotations. Our research highlighted that KIE standard benchmarks such as SROIE and FUNSD contain significant similarity between training and testing documents and can be adjusted to better evaluate the generalization of models. In this work, we designed experiments to quantify the information redundancy in public benchmarks, revealing a 75% template replication in SROIE official test set and 16% in FUNSD. We also proposed resampling strategies to provide benchmarks more representative of the generalization ability of models. We showed that models not suited for document analysis struggle on the adjusted splits dropping on average 10,5% F1 score on SROIE and 3.5% on FUNSD compared to multi-modal models dropping only 7,5% F1 on SROIE and 0.5% F1 on FUNSD.
Restoring Original Signal From Pile-up Signal using Deep Learning

Pile-up signals are frequently produced in experimental physics. They create inaccurate physics data with high uncertainty and cause various problems. Therefore, the correction to pile-up signals is crucially required. In this study, we implemented a deep learning method to restore the original signals from the pile-up signals. We showed that a deep learning model could accurately reconstruct the original signal waveforms from the pile-up waveforms. By substituting the pile-up signals with the original signals predicted by the model, the energy and timing resolutions of the data are notably enhanced. The model implementation significantly improved the quality of the particle identification plot and particle tracks. This method is applicable to similar problems, such as separating multiple signals or correcting pile-up signals with other types of noises and backgrounds.
Approximate Nearest Neighbor for Polygonal Curves under Fréchet Distance

We propose $\kappa$-approximate nearest neighbor (ANN) data structures for $n$ polygonal curves under the Fr\'{e}chet distance in $\mathbb{R}^d$, where $\kappa \in \{1+\varepsilon,3+\varepsilon\}$ and $d \geq 2$. We assume that every input curve has at most $m$ vertices, every query curve has at most $k$ vertices, $k \ll m$, and $k$ is given for preprocessing. The query times are $\tilde{O}(k(mn)^{0.5+\varepsilon}/\varepsilon^d+ k(d/\varepsilon)^{O(dk)})$ for $(1+\varepsilon)$-ANN and $\tilde{O}(k(mn)^{0.5+\varepsilon}/\varepsilon^d)$ for $(3+\varepsilon)$-ANN. The space and expected preprocessing time are $\tilde{O}(k(mnd^d/\varepsilon^d)^{O(k+1/\varepsilon^2)})$ in both cases. In two and three dimensions, we improve the query times to $O(1/\varepsilon)^{O(k)} \cdot \tilde{O}(k)$ for $(1+\varepsilon)$-ANN and $\tilde{O}(k)$ for $(3+\varepsilon)$-ANN. The space and expected preprocessing time improve to $O(mn/\varepsilon)^{O(k)} \cdot \tilde{O}(k)$ in both cases. For ease of presentation, we treat factors in our bounds that depend purely on $d$ as~$O(1)$. The hidden polylog factors in the big-$\tilde{O}$ notation have powers dependent on $d$.
A Class of Dependent Random Distributions Based on Atom Skipping

We propose the Plaid Atoms Model (PAM), a novel Bayesian nonparametric model for grouped data. Founded on an idea of `atom skipping', PAM is part of a well-established category of models that generate dependent random distributions and clusters across multiple groups. Atom skipping referrs to stochastically assigning 0 weights to atoms in an infinite mixture. Deploying atom skipping across groups, PAM produces a dependent clustering pattern with overlapping and non-overlapping clusters across groups. As a result, interpretable posterior inference is possible such as reporting the posterior probability of a cluster being exclusive to a single group or shared among a subset of groups. We discuss the theoretical properties of the proposed and related models. Minor extensions of the proposed model for multivariate or count data are presented. Simulation studies and applications using real-world datasets illustrate the performance of the new models with comparison to existing models.
Majorana transformation of the Thomas-Fermi equation demystified

The Majorana transformation makes it possible to reduce the Thomas-Fermi equation to a first-order differential equation. This reduction is possible due to the special scaling property of the Thomas-Fermi equation under homology transformations. Such reductions are well known in the context of stellar astrophysics, where the use of homology-invariant variables has long proved useful. We use homology-invariant variables in the context of the Thomas-Fermi equation to demystify the origin of the otherwise mysterious Majorana transformation.
Quantum Cross Subspace Alignment Codes via the N-sum Box Abstraction

Cross-subspace alignment (CSA) codes are used in various private information retrieval (PIR) schemes (e.g., with secure storage) and in secure distributed batch matrix multiplication (SDBMM). Using a recently developed $N$-sum box abstraction of a quantum multiple-access channel (QMAC), we translate CSA schemes over classical multiple-access channels into efficient quantum CSA schemes over a QMAC, achieving maximal superdense coding gain. Because of the $N$-sum box abstraction, the underlying problem of coding to exploit quantum entanglements for CSA schemes, becomes conceptually equivalent to that of designing a channel matrix for a MIMO MAC subject to given structural constraints imposed by the $N$-sum box abstraction, such that the resulting MIMO MAC is able to implement the functionality of a CSA scheme (encoding/decoding) over-the-air. Applications include Quantum PIR with secure and MDS-coded storage, as well as Quantum SDBMM.
2D dilaton gravity and the Weil-Petersson volumes with conical defects

We derive the Weil-Petersson measure on the moduli space of hyperbolic surfaces with defects of arbitrary opening angles and use this to compute its volume. We conjecture a matrix integral computing the corresponding volumes and confirm agreement in simple cases. We combine this mathematical result with the equivariant localization approach to Jackiw-Teitelboim gravity to justify a proposed exact solution of pure 2d dilaton gravity for a large class of dilaton potentials.
Imbalanced Node Classification Beyond Homophilic Assumption

Imbalanced node classification widely exists in real-world networks where graph neural networks (GNNs) are usually highly inclined to majority classes and suffer from severe performance degradation on classifying minority class nodes. Various imbalanced node classification methods have been proposed recently which construct synthetic nodes and edges w.r. t. minority classes to balance the label and topology distribution. However, they are all based on the homophilic assumption that nodes of the same label tend to connect despite the wide existence of heterophilic edges in real-world graphs. Thus, they uniformly aggregate features from both homophilic and heterophilic neighbors and rely on feature similarity to generate synthetic edges, which cannot be applied to imbalanced graphs in high heterophily. To address this problem, we propose a novel GraphSANN for imbalanced node classification on both homophilic and heterophilic graphs. Firstly, we propose a unified feature mixer to generate synthetic nodes with both homophilic and heterophilic interpolation in a unified way. Next, by randomly sampling edges between synthetic nodes and existing nodes as candidate edges, we design an adaptive subgraph extractor to adaptively extract the contextual subgraphs of candidate edges with flexible ranges. Finally, we develop a multi-filter subgraph encoder that constructs different filter channels to discriminatively aggregate neighbor's information along the homophilic and heterophilic edges. Extensive experiments on eight datasets demonstrate the superiority of our model for imbalanced node classification on both homophilic and heterophilic graphs.
Non-Commutative classifying spaces of groups via quasi-topologies and pro-$C^*$-algebras

For a completely Hausdorff quasi-topological group $G$, we construct a universal pro-$C^*$-algebra $C(E^+G)$ as the non-commutative geometer's analogue of the total space $EG$ of the classifying principal $G$-bundle $EG\to BG$. The pro-$C^*$-algebra $C(EG)$ of (possibly unbounded) continuous functions on $EG$ is then recoverable as the abelianization of $C(E^+G)$. Along the way, we develop various aspects of the theory of quasi-topological $G$-spaces and $G$-pro-$C^*$-algebras.
Visual Referential Games Further the Emergence of Disentangled Representations

Natural languages are powerful tools wielded by human beings to communicate information. Among their desirable properties, compositionality has been the main focus in the context of referential games and variants, as it promises to enable greater systematicity to the agents which would wield it. The concept of disentanglement has been shown to be of paramount importance to learned representations that generalise well in deep learning, and is thought to be a necessary condition to enable systematicity. Thus, this paper investigates how do compositionality at the level of the emerging languages, disentanglement at the level of the learned representations, and systematicity relate to each other in the context of visual referential games. Firstly, we find that visual referential games that are based on the Obverter architecture outperforms state-of-the-art unsupervised learning approach in terms of many major disentanglement metrics. Secondly, we expand the previously proposed Positional Disentanglement (PosDis) metric for compositionality to (re-)incorporate some concerns pertaining to informativeness and completeness features found in the Mutual Information Gap (MIG) disentanglement metric it stems from. This extension allows for further discrimination between the different kind of compositional languages that emerge in the context of Obverter-based referential games, in a way that neither the referential game accuracy nor previous metrics were able to capture. Finally we investigate whether the resulting (emergent) systematicity, as measured by zero-shot compositional learning tests, correlates with any of the disentanglement and compositionality metrics proposed so far. Throughout the training process, statically significant correlation coefficients can be found both positive and negative depending on the moment of the measure.
Giant-atom effects on population and entanglement dynamics of Rydberg atoms in the optical regime

Giant atoms are attracting interest as an emerging paradigm in the quantum optics of engineered waveguides. Here we propose to realize a synthetic giant atom working in the optical regime starting from a pair of interacting Rydberg atoms driven by a coherent field and coupled to a photonic crystal waveguide. Giant-atom effects can be observed as a phase-dependent decay of the double Rydberg excitation during the initial evolution of this atomic pair while (internal) atomic entanglement is exhibited at later times. Such an intriguing entanglement onset occurs in the presence of intrinsic atomic decay toward non-guided vacuum modes and is accompanied by an anti-bunching correlation of the emitted photons. Our findings may be relevant to quantum information processing, besides broadening the giant-atom waveguide physics with optically driven natural atoms.