OpenRaiser/Intern-Atlas huggingface dataset:
Towards Complex Dynamic Physics System Simulation with Graph Neural ODEs
Venue: arXiv cs.LG 2023
Year: 2023

The great learning ability of deep learning models facilitates us to comprehend the real physical world, making learning to simulate complicated particle systems a promising endeavour. However, the complex laws of the physical world pose significant challenges to the learning based simulations, such as the varying spatial dependencies between interacting particles and varying temporal dependencies between particle system states in different time stamps, which dominate particles' interacting behaviour and the physical systems' evolution patterns. Existing learning based simulation methods fail to fully account for the complexities, making them unable to yield satisfactory simulations. To better comprehend the complex physical laws, this paper proposes a novel learning based simulation model- Graph Networks with Spatial-Temporal neural Ordinary Equations (GNSTODE)- that characterizes the varying spatial and temporal dependencies in particle systems using a united end-to-end framework. Through training with real-world particle-particle interaction observations, GNSTODE is able to simulate any possible particle systems with high precisions. We empirically evaluate GNSTODE's simulation performance on two real-world particle systems, Gravity and Coulomb, with varying levels of spatial and temporal dependencies. The results show that the proposed GNSTODE yields significantly better simulations than state-of-the-art learning based simulation methods, which proves that GNSTODE can serve as an effective solution to particle simulations in real-world application.
Temporal Fusion Transformers for Streamflow Prediction: Value of Combining Attention with Recurrence
Venue: arXiv cs.LG 2023
Year: 2023

Over the past few decades, the hydrology community has witnessed notable advancements in streamflow prediction, particularly with the introduction of cutting-edge machine-learning algorithms. Recurrent neural networks, especially Long Short-Term Memory (LSTM) networks, have become popular due to their capacity to create precise forecasts and realistically mimic the system dynamics. Attention-based models, such as Transformers, can learn from the entire data sequence concurrently, a feature that LSTM does not have. This work tests the hypothesis that combining recurrence with attention can improve streamflow prediction. We set up the Temporal Fusion Transformer (TFT) architecture, a model that combines both of these aspects and has never been applied in hydrology before. We compare the performance of LSTM, Transformers, and TFT over 2,610 globally distributed catchments from the recently available Caravan dataset. Our results demonstrate that TFT indeed exceeds the performance benchmark set by the LSTM and Transformers for streamflow prediction. Additionally, being an explainable AI method, TFT helps in gaining insights into the streamflow generation processes.
Measuring Intersectional Biases in Historical Documents
Venue: arXiv cs.CL 2023
Year: 2023

Data-driven analyses of biases in historical texts can help illuminate the origin and development of biases prevailing in modern society. However, digitised historical documents pose a challenge for NLP practitioners as these corpora suffer from errors introduced by optical character recognition (OCR) and are written in an archaic language. In this paper, we investigate the continuities and transformations of bias in historical newspapers published in the Caribbean during the colonial era (18th to 19th centuries). Our analyses are performed along the axes of gender, race, and their intersection. We examine these biases by conducting a temporal study in which we measure the development of lexical associations using distributional semantics models and word embeddings. Further, we evaluate the effectiveness of techniques designed to process OCR-generated data and assess their stability when trained on and applied to the noisy historical newspapers. We find that there is a trade-off between the stability of the word embeddings and their compatibility with the historical dataset. We provide evidence that gender and racial biases are interdependent, and their intersection triggers distinct effects. These findings align with the theory of intersectionality, which stresses that biases affecting people with multiple marginalised identities compound to more than the sum of their constituents.
Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognition
Venue: arXiv cs.CV 2023
Year: 2023

How humans understand and recognize the actions of others is a complex neuroscientific problem that involves a combination of cognitive mechanisms and neural networks. Research has shown that humans have brain areas that recognize actions that process top-down attentional information, such as the temporoparietal association area. Also, humans have brain regions dedicated to understanding the minds of others and analyzing their intentions, such as the medial prefrontal cortex of the temporal lobe. Skeleton-based action recognition creates mappings for the complex connections between the human skeleton movement patterns and behaviors. Although existing studies encoded meaningful node relationships and synthesized action representations for classification with good results, few of them considered incorporating a priori knowledge to aid potential representation learning for better performance. LA-GCN proposes a graph convolution network using large-scale language models (LLM) knowledge assistance. First, the LLM knowledge is mapped into a priori global relationship (GPR) topology and a priori category relationship (CPR) topology between nodes. The GPR guides the generation of new "bone" representations, aiming to emphasize essential node information from the data level. The CPR mapping simulates category prior knowledge in human brain regions, encoded by the PC-AC module and used to add additional supervision-forcing the model to learn class-distinguishable features. In addition, to improve information transfer efficiency in topology modeling, we propose multi-hop attention graph convolution. It aggregates each node's k-order neighbor simultaneously to speed up model convergence. LA-GCN reaches state-of-the-art on NTU RGB+D, NTU RGB+D 120, and NW-UCLA datasets.
Inverse kinematics and path planning of manipulator using real quantifier elimination based on Comprehensive Gröbner Systems
Venue: arXiv cs.RO 2023
Year: 2023

Methods for inverse kinematics computation and path planning of a three degree-of-freedom (DOF) manipulator using the algorithm for quantifier elimination based on Comprehensive Gröbner Systems (CGS), called CGS-QE method, are proposed. The first method for solving the inverse kinematics problem employs counting the real roots of a system of polynomial equations to verify the solution's existence. In the second method for trajectory planning of the manipulator, the use of CGS guarantees the existence of an inverse kinematics solution. Moreover, it makes the algorithm more efficient by preventing repeated computation of Gröbner basis. In the third method for path planning of the manipulator, for a path of the motion given as a function of a parameter, the CGS-QE method verifies the whole path's feasibility. Computational examples and an experiment are provided to illustrate the effectiveness of the proposed methods.
Infor-Coef: Information Bottleneck-based Dynamic Token Downsampling for Compact and Efficient language model
Venue: arXiv cs.CL 2023
Year: 2023

The prevalence of Transformer-based pre-trained language models (PLMs) has led to their wide adoption for various natural language processing tasks. However, their excessive overhead leads to large latency and computational costs. The statically compression methods allocate fixed computation to different samples, resulting in redundant computation. The dynamic token pruning method selectively shortens the sequences but are unable to change the model size and hardly achieve the speedups as static pruning. In this paper, we propose a model accelaration approaches for large language models that incorporates dynamic token downsampling and static pruning, optimized by the information bottleneck loss. Our model, Infor-Coef, achieves an 18x FLOPs speedup with an accuracy degradation of less than 8\% compared to BERT. This work provides a promising approach to compress and accelerate transformer-based models for NLP tasks.
Continually Improving Extractive QA via Human Feedback
Venue: arXiv cs.CL 2023
Year: 2023

We study continually improving an extractive question answering (QA) system via human user feedback. We design and deploy an iterative approach, where information-seeking users ask questions, receive model-predicted answers, and provide feedback. We conduct experiments involving thousands of user interactions under diverse setups to broaden the understanding of learning from feedback over time. Our experiments show effective improvement from user feedback of extractive QA models over time across different data regimes, including significant potential for domain adaptation.
Word differences in news media of lower and higher peace countries revealed by natural language processing and machine learning
Venue: arXiv cs.CY 2023
Year: 2023

Language is both a cause and a consequence of the social processes that lead to conflict or peace. Hate speech can mobilize violence and destruction. What are the characteristics of peace speech that reflect and support the social processes that maintain peace? This study used existing peace indices, machine learning, and on-line, news media sources to identify the words most associated with lower-peace versus higher-peace countries. As each peace index measures different social properties, there is little consensus on the numerical values of these indices. There is however greater consensus with these indices for the countries that are at the extremes of lower-peace and higher-peace. Therefore, a data driven approach was used to find the words most important in distinguishing lower-peace and higher-peace countries. Rather than assuming a theoretical framework that predicts which words are more likely in lower-peace and higher-peace countries, and then searching for those words in news media, in this study, natural language processing and machine learning were used to identify the words that most accurately classified a country as lower-peace or higher-peace. Once the machine learning model was trained on the word frequencies from the extreme lower-peace and higher-peace countries, that model was also used to compute a quantitative peace index for these and other intermediate-peace countries. The model successfully yielded a quantitative peace index for intermediate-peace countries that was in between that of the lower-peace and higher-peace, even though they were not in the training set. This study demonstrates how natural language processing and machine learning can help to generate new quantitative measures of social systems, which in this study, were linguistic differences resulting in a quantitative index of peace for countries at different levels of peacefulness.
Routing for Intermittently-Powered Sensing Systems
Venue: arXiv cs.NI 2023
Year: 2023

Recently, intermittent computing (IC) has received tremendous attention due to its high potential in perpetual sensing for Internet-of-Things (IoT). By harvesting ambient energy, battery-free devices can perform sensing intermittently without maintenance, thus significantly improving IoT sustainability. To build a practical intermittently-powered sensing system, efficient routing across battery-free devices for data delivery is essential. However, the intermittency of these devices brings new challenges, rendering existing routing protocols inapplicable. In this paper, we propose RICS, the first-of-its-kind routing scheme tailored for intermittently-powered sensing systems. RICS features two major designs, with the goal of achieving low-latency data delivery on a network built with battery-free devices. First, RICS incorporates a fast topology construction protocol for each IC node to establish a path towards the sink node with the least hop count. Second, RICS employs a low-latency message forwarding protocol, which incorporates an efficient synchronization mechanism and a novel technique called pendulum-sync to avoid the time-consuming repeated node synchronization. Our evaluation based on an implementation in OMNeT++ and comprehensive experiments with varying system settings show that RICS can achieve orders of magnitude latency reduction in data delivery compared with the baselines.
M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education
Venue: arXiv cs.HC 2023
Year: 2023

In this article, we present a Web-based System called M2LADS, which supports the integration and visualization of multimodal data recorded in learning sessions in a MOOC in the form of Web-based Dashboards. Based on the edBB platform, the multimodal data gathered contains biometric and behavioral signals including electroencephalogram data to measure learners' cognitive attention, heart rate for affective measures, visual attention from the video recordings. Additionally, learners' static background data and their learning performance measures are tracked using LOGCE and MOOC tracking logs respectively, and both are included in the Web-based System. M2LADS provides opportunities to capture learners' holistic experience during their interactions with the MOOC, which can in turn be used to improve their learning outcomes through feedback visualizations and interventions, as well as to enhance learning analytics models and improve the open content of the MOOC.
Atomic and Subgraph-aware Bilateral Aggregation for Molecular Representation Learning
Venue: arXiv cs.LG 2023
Year: 2023

Molecular representation learning is a crucial task in predicting molecular properties. Molecules are often modeled as graphs where atoms and chemical bonds are represented as nodes and edges, respectively, and Graph Neural Networks (GNNs) have been commonly utilized to predict atom-related properties, such as reactivity and solubility. However, functional groups (subgraphs) are closely related to some chemical properties of molecules, such as efficacy, and metabolic properties, which cannot be solely determined by individual atoms. In this paper, we introduce a new model for molecular representation learning called the Atomic and Subgraph-aware Bilateral Aggregation (ASBA), which addresses the limitations of previous atom-wise and subgraph-wise models by incorporating both types of information. ASBA consists of two branches, one for atom-wise information and the other for subgraph-wise information. Considering existing atom-wise GNNs cannot properly extract invariant subgraph features, we propose a decomposition-polymerization GNN architecture for the subgraph-wise branch. Furthermore, we propose cooperative node-level and graph-level self-supervised learning strategies for ASBA to improve its generalization. Our method offers a more comprehensive way to learn representations for molecular property prediction and has broad potential in drug and material discovery applications. Extensive experiments have demonstrated the effectiveness of our method.
Duplex Diffusion Models Improve Speech-to-Speech Translation
Venue: arXiv cs.CL 2023
Year: 2023

Speech-to-speech translation is a typical sequence-to-sequence learning task that naturally has two directions. How to effectively leverage bidirectional supervision signals to produce high-fidelity audio for both directions? Existing approaches either train two separate models or a multitask-learned model with low efficiency and inferior performance. In this paper, we propose a duplex diffusion model that applies diffusion probabilistic models to both sides of a reversible duplex Conformer, so that either end can simultaneously input and output a distinct language's speech. Our model enables reversible speech translation by simply flipping the input and output ends. Experiments show that our model achieves the first success of reversible speech translation with significant improvements of ASR-BLEU scores compared with a list of state-of-the-art baselines.
The HCCL system for VoxCeleb Speaker Recognition Challenge 2022
Venue: arXiv cs.SD 2023
Year: 2023

This report describes our submission to track1 and track3 for VoxCeleb Speaker Recognition Challenge 2022(VoxSRC2022). Our best system achieves minDCF 0.1397 and EER 2.414 in track1, minDCF 0.388 and EER 7.030 in track3.
Reflective Linguistic Programming (RLP): A Stepping Stone in Socially-Aware AGI (SocialAGI)
Venue: arXiv cs.AI 2023
Year: 2023

This paper presents Reflective Linguistic Programming (RLP), a unique approach to conversational AI that emphasizes self-awareness and strategic planning. RLP encourages models to introspect on their own predefined personality traits, emotional responses to incoming messages, and planned strategies, enabling contextually rich, coherent, and engaging interactions. A striking illustration of RLP's potential involves a toy example, an AI persona with an adversarial orientation, a demon named `Bogus' inspired by the children's fairy tale Hansel & Gretel. Bogus exhibits sophisticated behaviors, such as strategic deception and sensitivity to user discomfort, that spontaneously arise from the model's introspection and strategic planning. These behaviors are not pre-programmed or prompted, but emerge as a result of the model's advanced cognitive modeling. The potential applications of RLP in socially-aware AGI (Social AGI) are vast, from nuanced negotiations and mental health support systems to the creation of diverse and dynamic AI personas. Our exploration of deception serves as a stepping stone towards a new frontier in AGI, one filled with opportunities for advanced cognitive modeling and the creation of truly human `digital souls'.
Estimating Discrete Total Curvature with Per Triangle Normal Variation
Venue: arXiv cs.GR 2023
Year: 2023

We introduce a novel approach for measuring the total curvature at every triangle of a discrete surface. This method takes advantage of the relationship between per triangle total curvature and the Dirichlet energy of the Gauss map. This new tool can be used on both triangle meshes and point clouds and has numerous applications. In this study, we demonstrate the effectiveness of our technique by using it for feature-aware mesh decimation, and show that it outperforms existing curvature-estimation methods from popular libraries such as Meshlab, Trimesh2, and Libigl. When estimating curvature on point clouds, our method outperforms popular libraries PCL and CGAL.
On the Boomerang Spectrum of Power Permutation $X^{2^{3n}+2^{2n}+2^{n}-1}$ over $\GF{2^{4n}}$ and Extraction of Optimal Uniformity Boomerang Functions
Venue: arXiv cs.IT 2023
Year: 2023

A substitution box (S-box) in a symmetric primitive is a mapping $F$ that takes $k$ binary inputs and whose image is a binary $m$-tuple for some positive integers $k$ and $m$, which is usually the only nonlinear element of the most modern block ciphers. Therefore, employing S-boxes with good cryptographic properties to resist various attacks is significant. For power permutation $F$ over finite field $\GF{2^k}$, the multiset of values $β_F(1,b)=\#\{x\in \GF{2^k}\mid F^{-1}(F(x)+b)+F^{-1}(F(x+1)+b)=1\}$ for $b\in \GF{2^k}$ is called the boomerang spectrum of $F$. The maximum value in the boomerang spectrum is called boomerang uniformity. This paper determines the boomerang spectrum of the power permutation $X^{2^{3n}+2^{2n}+2^{n}-1}$ over $\GF{2^{4n}}$. The boomerang uniformity of that power permutation is $3(2^{2n}-2^n)$. However, on a large subset $\{b\in \GF{2^{4n}}\mid \mathbf{Tr}_n^{4n}(b)\neq 0\}$ of $\GF{2^{4n}}$ of cardinality $2^{4n}-2^{3n}$ (where $ \mathbf{Tr}_n^{4n}$ is the (relative) trace function from $\GF{2^{4n}}$ to $\GF{2^{n}}$), we prove that the studied function $F$ achieves the optimal boomerang uniformity $2$. It is known that obtaining such functions is a challenging problem. More importantly, the set of $b$'s giving this value is explicitly determined for any value in the boomerang spectrum.
Angle-based SLAM on 5G mmWave Systems: Design, Implementation, and Measurement
Venue: arXiv cs.IT 2023
Year: 2023

Simultaneous localization and mapping (SLAM) is a key technology that provides user equipment (UE) tracking and environment mapping services, enabling the deep integration of sensing and communication. The millimeter-wave (mmWave) communication, with its larger bandwidths and antenna arrays, inherently facilitates more accurate delay and angle measurements than sub-6 GHz communication, thereby providing opportunities for SLAM. However, none of the existing works have realized the SLAM function under the 5G New Radio (NR) standard due to specification and hardware constraints. In this study, we investigate how 5G mmWave communication systems can achieve situational awareness without changing the transceiver architecture and 5G NR standard. We implement 28 GHz mmWave transceivers that deploy OFDM-based 5G NR waveform with 160 MHz channel bandwidth, and we realize beam management following the 5G NR. Furthermore, we develop an efficient successive cancellation-based angle extraction approach to obtain angles of arrival and departure from the reference signal received power measurements. On the basis of angle measurements, we propose an angle-only SLAM algorithm to track UE and map features in the radio environment. Thorough experiments and ray tracing-based computer simulations verify that the proposed angle-based SLAM can achieve sub-meter level localization and mapping accuracy with a single base station and without the requirement of strict time synchronization. Our experiments also reveal many propagation properties critical to the success of SLAM in 5G mmWave communication systems.
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture
Venue: arXiv cs.CL 2023
Year: 2023

Real-world domain experts (e.g., doctors) rarely annotate only a decision label in their day-to-day workflow without providing explanations. Yet, existing low-resource learning techniques, such as Active Learning (AL), that aim to support human annotators mostly focus on the label while neglecting the natural language explanation of a data point. This work proposes a novel AL architecture to support experts' real-world need for label and explanation annotations in low-resource scenarios. Our AL architecture leverages an explanation-generation model to produce explanations guided by human explanations, a prediction model that utilizes generated explanations toward prediction faithfully, and a novel data diversity-based AL sampling strategy that benefits from the explanation annotations. Automated and human evaluations demonstrate the effectiveness of incorporating explanations into AL sampling and the improved human annotation efficiency and trustworthiness with our AL architecture. Additional ablation studies illustrate the potential of our AL architecture for transfer learning, generalizability, and integration with large language models (LLMs). While LLMs exhibit exceptional explanation-generation capabilities for relatively simple tasks, their effectiveness in complex real-world tasks warrants further in-depth study.