OpenRaiser/Intern-Atlas huggingface dataset:
Online Learning with Adversaries: A Differential-Inclusion Analysis
Venue: arXiv cs.LG 2023
Year: 2023

We introduce an observation-matrix-based framework for fully asynchronous online Federated Learning (FL) with adversaries. In this work, we demonstrate its effectiveness in estimating the mean of a random vector. Our main result is that the proposed algorithm almost surely converges to the desired mean $μ.$ This makes ours the first asynchronous FL method to have an a.s. convergence guarantee in the presence of adversaries. We derive this convergence using a novel differential-inclusion-based two-timescale analysis. Two other highlights of our proof include (a) the use of a novel Lyapunov function to show that $μ$ is the unique global attractor for our algorithm's limiting dynamics, and (b) the use of martingale and stopping-time theory to show that our algorithm's iterates are almost surely bounded.
G2PTL: A Pre-trained Model for Delivery Address and its Applications in Logistics System
Venue: arXiv cs.AI 2023
Year: 2023

Text-based delivery addresses, as the data foundation for logistics systems, contain abundant and crucial location information. How to effectively encode the delivery address is a core task to boost the performance of downstream tasks in the logistics system. Pre-trained Models (PTMs) designed for Natural Language Process (NLP) have emerged as the dominant tools for encoding semantic information in text. Though promising, those NLP-based PTMs fall short of encoding geographic knowledge in the delivery address, which considerably trims down the performance of delivery-related tasks in logistic systems such as Cainiao. To tackle the above problem, we propose a domain-specific pre-trained model, named G2PTL, a Geography-Graph Pre-trained model for delivery address in Logistics field. G2PTL combines the semantic learning capabilities of text pre-training with the geographical-relationship encoding abilities of graph modeling. Specifically, we first utilize real-world logistics delivery data to construct a large-scale heterogeneous graph of delivery addresses, which contains abundant geographic knowledge and delivery information. Then, G2PTL is pre-trained with subgraphs sampled from the heterogeneous graph. Comprehensive experiments are conducted to demonstrate the effectiveness of G2PTL through four downstream tasks in logistics systems on real-world datasets. G2PTL has been deployed in production in Cainiao's logistics system, which significantly improves the performance of delivery-related tasks. The code of G2PTL is available at https://huggingface.co/Cainiao-AI/G2PTL.
An interpretability framework for Similar case matching
Venue: arXiv cs.CL 2023
Year: 2023

Similar Case Matching (SCM) plays a pivotal role in the legal system by facilitating the efficient identification of similar cases for legal professionals. While previous research has primarily concentrated on enhancing the performance of SCM models, the aspect of interpretability has been neglected. To bridge the gap, this study proposes an integrated pipeline framework for interpretable SCM. The framework comprises four modules: judicial feature sentence identification, case matching, feature sentence alignment, and conflict resolution. In contrast to current SCM methods, our framework first extracts feature sentences within a legal case that contain essential information. Then it conducts case matching based on these extracted features. Subsequently, our framework aligns the corresponding sentences in two legal cases to provide evidence of similarity. In instances where the results of case matching and feature sentence alignment exhibit conflicts, the conflict resolution module resolves these inconsistencies. The experimental results show the effectiveness of our proposed framework, establishing a new benchmark for interpretable SCM.
Reduced-Precision Floating-Point Arithmetic in Systolic Arrays with Skewed Pipelines
Venue: arXiv cs.AR 2023
Year: 2023

The acceleration of deep-learning kernels in hardware relies on matrix multiplications that are executed efficiently on Systolic Arrays (SA). To effectively trade off deep-learning training/inference quality with hardware cost, SA accelerators employ reduced-precision Floating-Point (FP) arithmetic. In this work, we demonstrate the need for new pipeline organizations to reduce latency and improve energy efficiency of reduced-precision FP operators for the chained multiply-add operation imposed by the structure of the SA. The proposed skewed pipeline design reorganizes the pipelined operation of the FP multiply-add units to enable new forwarding paths for the exponent logic, which allow for parallel execution of the pipeline stages of consecutive PEs. As a result, the latency of the matrix multiplication operation within the SA is significantly reduced with minimal hardware cost, thereby yielding an energy reduction of 8% and 11% for the examined state-of-the-art CNNs.
Low-PAPR Joint Channel Estimation and Data Detection in ZP-OTFS System
Venue: arXiv cs.IT 2023
Year: 2023

Orthogonal Time Frequency Space (OTFS) systems face significant challenges in channel estimation due to high pilot overhead and peak-to-average power ratio (PAPR). To address these issues, we propose a two-step channel estimation method for Zero-Pad OTFS (ZP-OTFS), a modified OTFS system characterized by multiple zero rows along the delay axis. This method strategically inserts pilot sequences into the zero bins of the ZP-OTFS system, effectively mitigating overhead and PAPR. Comprehensive simulation results validate the effectiveness of our proposed method, demonstrating its superior performance over traditional embedded pilot estimation in high Signal-to-Noise Ratio (SNR) scenarios. Specifically, our method achieves a lower normalized mean square error (NMSE) and better bit error rates (BER) at high SNRs.
Privacy-Preserving Federated Discovery of DNA Motifs with Differential Privacy
Venue: arXiv cs.CR 2023
Year: 2023

DNA motif discovery is an important issue in gene research, which aims to identify transcription factor binding sites (i.e., motifs) in DNA sequences to reveal the mechanisms that regulate gene expression. However, the phenomenon of data silos and the problem of privacy leakage have seriously hindered the development of DNA motif discovery. On the one hand, the phenomenon of data silos makes data collection difficult. On the other hand, the collection and use of DNA data become complicated and difficult because DNA is sensitive private information. In this context, how discovering DNA motifs under the premise of ensuring privacy and security and alleviating data silos has become a very important issue. Therefore, this paper proposes a novel method, namely DP-FLMD, to address this problem. Note that this is the first application of federated learning to the field of genetics research. The federated learning technique is used to solve the problem of data silos. It has the advantage of enabling multiple participants to train models together and providing privacy protection services. To address the challenges of federated learning in terms of communication costs, this paper applies a sampling method and a strategy for reducing communication costs to DP-FLMD. In addition, differential privacy, a privacy protection technique with rigorous mathematical proof, is also applied to DP-FLMD. Experiments on the DNA datasets show that DP-FLMD has high mining accuracy and runtime efficiency, and the performance of the algorithm is affected by some parameters.
Performance of 802.11be Wi-Fi 7 with Multi-Link Operation on AR Applications
Venue: arXiv cs.NI 2023
Year: 2023

Since its first release in the late 1990s, Wi-Fi has been updated to keep up with evolving user needs. Recently, Wi-Fi and other radio access technologies have been pushed to their edge when serving Augmented Reality (AR) applications. AR applications require high throughput, low latency, and high reliability to ensure a high-quality user experience. The 802.11be amendment, which will be marketed as Wi-Fi 7, introduces several features that aim to enhance its capabilities to support challenging applications like AR. One of the main features introduced in this amendment is Multi-Link Operation (MLO) which allows nodes to transmit and receive over multiple links concurrently. When using MLO, traffic is distributed among links using an implementation-specific traffic-to-link allocation policy. This paper aims to evaluate the performance of MLO, using different policies, in serving AR applications compared to Single-Link (SL). Experimental simulations using an event-based Wi-Fi simulator have been conducted. Our results show the general superiority of MLO when serving AR applications. MLO achieves lower latency and serves a higher number of AR users compared to SL with the same frequency resources. In addition, increasing the number of links can improve the performance of MLO. Regarding traffic-to-link allocation policies, we found that policies can be more susceptible to channel blocking, resulting in possible performance degradation.
Towards a Flexible User Interface for 'Quick and Dirty' Learning Analytics Indicator Design
Venue: arXiv cs.CY 2023
Year: 2023

Research on Human-Centered Learning Analytics (HCLA) has provided demonstrations of a successful co-design process for LA tools with different stakeholders. However, there is a need for 'quick and dirty' methods to allow the low-cost design of LA indicators. Recently, Indicator Specification Cards (ISC) have been proposed to help different learning analytics stakeholders co-design indicators in a systematic manner. In this paper, we aim at improving the user experience, flexibility, and reliability of the ISC-based indicator design process. To this end, we present the development details of an intuitive and theoretically-sound ISC user interface that allows the low-cost design of LA indicators. Further, we propose two approaches to support the flexible design of indicators, namely a task-driven approach and a data-driven approach.
Adaptive parallelization of multi-agent simulations with localized dynamics
Venue: arXiv cs.DC 2023
Year: 2023

Agent-based modelling constitutes a versatile approach to representing and simulating complex systems. Studying large-scale systems is challenging because of the computational time required for the simulation runs: scaling is at least linear in system size (number of agents). Given the inherently modular nature of MABSs, parallel computing is a natural approach to overcoming this challenge. However, because of the shared information and communication between agents, parellelization is not simple. We present a protocol for shared-memory, parallel execution of MABSs. This approach is useful for models that can be formulated in terms of sequential computations, and that involve updates that are localized, in the sense of involving small numbers of agents. The protocol has a bottom-up and asynchronous nature, allowing it to deal with heterogeneous computation in an adaptive, yet graceful manner. We illustrate the potential performance gains on exemplar cultural dynamics and disease spreading MABSs.
Learning Invariant Representation via Contrastive Feature Alignment for Clutter Robust SAR Target Recognition
Venue: arXiv cs.CV 2023
Year: 2023

The deep neural networks (DNNs) have freed the synthetic aperture radar automatic target recognition (SAR ATR) from expertise-based feature designing and demonstrated superiority over conventional solutions. There has been shown the unique deficiency of ground vehicle benchmarks in shapes of strong background correlation results in DNNs overfitting the clutter and being non-robust to unfamiliar surroundings. However, the gap between fixed background model training and varying background application remains underexplored. Inspired by contrastive learning, this letter proposes a solution called Contrastive Feature Alignment (CFA) aiming to learn invariant representation for robust recognition. The proposed method contributes a mixed clutter variants generation strategy and a new inference branch equipped with channel-weighted mean square error (CWMSE) loss for invariant representation learning. In specific, the generation strategy is delicately designed to better attract clutter-sensitive deviation in feature space. The CWMSE loss is further devised to better contrast this deviation and align the deep features activated by the original images and corresponding clutter variants. The proposed CFA combines both classification and CWMSE losses to train the model jointly, which allows for the progressive learning of invariant target representation. Extensive evaluations on the MSTAR dataset and six DNN models prove the effectiveness of our proposal. The results demonstrated that the CFA-trained models are capable of recognizing targets among unfamiliar surroundings that are not included in the dataset, and are robust to varying signal-to-clutter ratios.
Efficient Generic Quotients Using Exact Arithmetic
Venue: arXiv cs.SC 2023
Year: 2023

The usual formulation of efficient division uses Newton iteration to compute an inverse in a related domain where multiplicative inverses exist. On one hand, Newton iteration allows quotients to be calculated using an efficient multiplication method. On the other hand, working in another domain is not always desirable and can lead to a library structure where arithmetic domains are interdependent. This paper uses the concept of a whole shifted inverse and modified Newton iteration to compute quotients efficiently without leaving the original domain. The iteration is generic to domains having a suitable shift operation, such as integers or polynomials with coefficients that do not necessarily commute.
Using Language Models For Knowledge Acquisition in Natural Language Reasoning Problems
Venue: arXiv cs.AI 2023
Year: 2023

For a natural language problem that requires some non-trivial reasoning to solve, there are at least two ways to do it using a large language model (LLM). One is to ask it to solve it directly. The other is to use it to extract the facts from the problem text and then use a theorem prover to solve it. In this note, we compare the two methods using ChatGPT and GPT4 on a series of logic word puzzles, and conclude that the latter is the right approach.
$k$-SUM in the Sparse Regime
Venue: arXiv cs.CC 2023
Year: 2023

In the average-case $k$-SUM problem, given $r$ integers chosen uniformly at random from $\{0,\dots,M-1\}$, the objective is to find a ``solution'' set of $k$ numbers that sum to $0$ modulo $M$. In the dense regime of $M \leq r^k$, where solutions exist with high probability, the complexity of these problems is well understood. Much less is known in the sparse regime of $M\gg r^k$, where solutions are unlikely to exist. In this work, we initiate the study of the sparse regime for $k$-SUM and its variant $k$-XOR, especially their planted versions, where a random solution is planted in a randomly generated instance and has to be recovered. We provide evidence for the hardness of these problems and suggest new applications to cryptography. Complexity. First we study the complexity of these problems in the sparse regime and show: - Conditional Lower Bounds. Assuming established conjectures about the hardness of average-case (non-planted) $k$-SUM/$k$-XOR when $M = r^k$, we provide non-trivial lower bounds on the running time of algorithms for planted $k$-SUM when $r^k\leq M\leq r^{2k}$. - Hardness Amplification. We show that for any $M \geq r^k$, if an algorithm running in time $T$ solves planted $k$-SUM/$k$-XOR with success probability $Ω(1/\text{polylog}(r))$, then there is an algorithm running in time $\tilde{O}(T)$ that solves it with probability $(1-o(1))$. - New Reductions and Algorithms. We provide reductions for $k$-SUM/$k$-XOR from search to decision, as well as worst-case and average-case reductions to the Subset Sum problem from $k$-SUM, as well as a new algorithm for average-case $k$-XOR at low densities. Cryptography. We show that by additionally assuming mild hardness of $k$-XOR, we can construct Public Key Encryption (PKE) from a weaker variant of the Learning Parity with Noise (LPN) problem than was known before.
CGDTest: A Constrained Gradient Descent Algorithm for Testing Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

In this paper, we propose a new Deep Neural Network (DNN) testing algorithm called the Constrained Gradient Descent (CGD) method, and an implementation we call CGDTest aimed at exposing security and robustness issues such as adversarial robustness and bias in DNNs. Our CGD algorithm is a gradient-descent (GD) method, with the twist that the user can also specify logical properties that characterize the kinds of inputs that the user may want. This functionality sets CGDTest apart from other similar DNN testing tools since it allows users to specify logical constraints to test DNNs not only for $\ell_p$ ball-based adversarial robustness but, more importantly, includes richer properties such as disguised and flow adversarial constraints, as well as adversarial robustness in the NLP domain. We showcase the utility and power of CGDTest via extensive experimentation in the context of vision and NLP domains, comparing against 32 state-of-the-art methods over these diverse domains. Our results indicate that CGDTest outperforms state-of-the-art testing tools for $\ell_p$ ball-based adversarial robustness, and is significantly superior in testing for other adversarial robustness, with improvements in PAR2 scores of over 1500% in some cases over the next best tool. Our evaluation shows that our CGD method outperforms competing methods we compared against in terms of expressibility (i.e., a rich constraint language and concomitant tool support to express a wide variety of properties), scalability (i.e., can be applied to very large real-world models with up to 138 million parameters), and generality (i.e., can be used to test a plethora of model architectures).
Rolling the Dice: Imagining Generative AI as a Dungeons & Dragons Storytelling Companion
Venue: arXiv cs.HC 2023
Year: 2023

AI Advancements have augmented casual writing and story generation, but their usage poses challenges in collaborative storytelling. In role-playing games such as Dungeons & Dragons (D&D), composing prompts using generative AI requires a technical understanding to generate ideal results, which is difficult for novices. Thus, emergent narratives organically developed based on player actions and decisions have yet to be fully utilized. This paper envisions the use of generative AI in transforming storytelling into an interactive drama using dynamic and immersive narratives. First, we describe scenarios where narratives are created and character conversations are designed within an overarching fantasy disposition. Then, we recommend design guidelines to help create tools using generative AI in interactive storytelling. Lastly, we raise questions on its potential impact on player immersion and cognitive load. Our contributions may be expanded within the broader interactive storytelling domain, such as speech-conversational AI and persona-driven chatbots.
SportsPose -- A Dynamic 3D sports pose dataset
Venue: arXiv cs.CV 2023
Year: 2023

Accurate 3D human pose estimation is essential for sports analytics, coaching, and injury prevention. However, existing datasets for monocular pose estimation do not adequately capture the challenging and dynamic nature of sports movements. In response, we introduce SportsPose, a large-scale 3D human pose dataset consisting of highly dynamic sports movements. With more than 176,000 3D poses from 24 different subjects performing 5 different sports activities, SportsPose provides a diverse and comprehensive set of 3D poses that reflect the complex and dynamic nature of sports movements. Contrary to other markerless datasets we have quantitatively evaluated the precision of SportsPose by comparing our poses with a commercial marker-based system and achieve a mean error of 34.5 mm across all evaluation sequences. This is comparable to the error reported on the commonly used 3DPW dataset. We further introduce a new metric, local movement, which describes the movement of the wrist and ankle joints in relation to the body. With this, we show that SportsPose contains more movement than the Human3.6M and 3DPW datasets in these extremum joints, indicating that our movements are more dynamic. The dataset with accompanying code can be downloaded from our website. We hope that SportsPose will allow researchers and practitioners to develop and evaluate more effective models for the analysis of sports performance and injury prevention. With its realistic and diverse dataset, SportsPose provides a valuable resource for advancing the state-of-the-art in pose estimation in sports.
Deep-BIAS: Detecting Structural Bias using Explainable AI
Venue: arXiv cs.NE 2023
Year: 2023

Evaluating the performance of heuristic optimisation algorithms is essential to determine how well they perform under various conditions. Recently, the BIAS toolbox was introduced as a behaviour benchmark to detect structural bias (SB) in search algorithms. The toolbox can be used to identify biases in existing algorithms, as well as to test for bias in newly developed algorithms. In this article, we introduce a novel and explainable deep-learning expansion of the BIAS toolbox, called Deep-BIAS. Where the original toolbox uses 39 statistical tests and a Random Forest model to predict the existence and type of SB, the Deep-BIAS method uses a trained deep-learning model to immediately detect the strength and type of SB based on the raw performance distributions. Through a series of experiments with a variety of structurally biased scenarios, we demonstrate the effectiveness of Deep-BIAS. We also present the results of using the toolbox on 336 state-of-the-art optimisation algorithms, which showed the presence of various types of structural bias, particularly towards the centre of the objective space or exhibiting discretisation behaviour. The Deep-BIAS method outperforms the BIAS toolbox both in detecting bias and for classifying the type of SB. Furthermore, explanations can be derived using XAI techniques.
Use of a Socially Assistive Robot as a Online Shopping Digital Skills Assistan
Venue: arXiv cs.RO 2023
Year: 2023

This work proposes and analyses the application of a robotic platform as an digital skills assistant. analysing the ethical issues relating to the decision making process in the use case of online food shopping in order to inform a co design session on what, and how, the digital skills assistant should make decisions.
Sociocultural knowledge is needed for selection of shots in hate speech detection tasks
Venue: arXiv cs.CL 2023
Year: 2023

We introduce HATELEXICON, a lexicon of slurs and targets of hate speech for the countries of Brazil, Germany, India and Kenya, to aid training and interpretability of models. We demonstrate how our lexicon can be used to interpret model predictions, showing that models developed to classify extreme speech rely heavily on target words when making predictions. Further, we propose a method to aid shot selection for training in low-resource settings via HATELEXICON. In few-shot learning, the selection of shots is of paramount importance to model performance. In our work, we simulate a few-shot setting for German and Hindi, using HASOC data for training and the Multilingual HateCheck (MHC) as a benchmark. We show that selecting shots based on our lexicon leads to models performing better on MHC than models trained on shots sampled randomly. Thus, when given only a few training examples, using our lexicon to select shots containing more sociocultural information leads to better few-shot performance.