OpenRaiser/Intern-Atlas huggingface dataset:
Improving Pre-movement Pattern Detection with Filter Bank Selection
Venue: arXiv cs.HC 2022
Year: 2022

Pre-movement decoding plays an important role in movement detection and is able to detect movement onset with low-frequency electroencephalogram (EEG) signals before the limb moves. In related studies, pre-movement decoding with standard task-related component analysis (STRCA) has been demonstrated to be efficient for classification between movement state and resting state. However, the accuracies of STRCA differ among subbands in the frequency domain. Due to individual differences, the best subband differs among subjects and is difficult to be determined. This study aims to improve the performance of the STRCA method by a feature selection on multiple subbands and avoid the selection of best subbands. This study first compares three frequency range settings ($M_1$: subbands with equally spaced bandwidths; $M_2$: subbands whose high cut-off frequencies are twice the low cut-off frequencies; $M_3$: subbands that start at some specific fixed frequencies and end at the frequencies in an arithmetic sequence.). Then, we develop a mutual information based technique to select the features in these subbands. A binary support vector machine classifier is used to classify the selected essential features. The results show that $M_3$ is a better setting than the other two settings. With the filter banks in $M_3$, the classification accuracy of the proposed FBTRCA achieves 0.8700$\pm$0.1022, which means a significantly improved performance compared to STRCA (0.8287$\pm$0.1101) as well as to the cross validation and testing method (0.8431$\pm$0.1078).
Biases in In Silico Evaluation of Molecular Optimization Methods and Bias-Reduced Evaluation Methodology
Venue: arXiv cs.LG 2022
Year: 2022

We are interested in in silico evaluation methodology for molecular optimization methods. Given a sample of molecules and their properties of our interest, we wish not only to train an agent that can find molecules optimized with respect to the target property but also to evaluate its performance. A common practice is to train a predictor of the target property on the sample and use it for both training and evaluating the agent. We show that this evaluator potentially suffers from two biases; one is due to misspecification of the predictor and the other to reusing the same sample for training and evaluation. We discuss bias reduction methods for each of the biases comprehensively, and empirically investigate their effectiveness.
Möbius Convolutions for Spherical CNNs
Venue: arXiv cs.CV 2022
Year: 2022

Möbius transformations play an important role in both geometry and spherical image processing - they are the group of conformal automorphisms of 2D surfaces and the spherical equivalent of homographies. Here we present a novel, Möbius-equivariant spherical convolution operator which we call Möbius convolution, and with it, develop the foundations for Möbius-equivariant spherical CNNs. Our approach is based on a simple observation: to achieve equivariance, we only need to consider the lower-dimensional subgroup which transforms the positions of points as seen in the frames of their neighbors. To efficiently compute Möbius convolutions at scale we derive an approximation of the action of the transformations on spherical filters, allowing us to compute our convolutions in the spectral domain with the fast Spherical Harmonic Transform. The resulting framework is both flexible and descriptive, and we demonstrate its utility by achieving promising results in both shape classification and image segmentation tasks.
Agent-based simulations for coverage extensions in 5G networks and beyond
Venue: arXiv cs.NI 2022
Year: 2022

Device-to-device (D2D) communications is one of the key emerging technologies for the fifth generation (5G) networks and beyond. It enables direct communication between mobile users and thereby extends coverage for devices lacking direct access to the cellular infrastructure and hence enhances network capacity. D2D networks are complex, highly dynamic and will be strongly augmented by intelligence for decision making at both the edge and core of the network, which makes them particularly difficult to predict and analyze. Conventionally, D2D systems are evaluated, investigated and analyzed using analytical and probabilistic models (e.g., from stochastic geometry). However, applying classical simulation and analytical tools to such a complex system is often hard to track and inaccurate. In this paper, we present a modeling and simulation framework from the perspective of complex-systems science and exhibit an agent-based model for the simulation of D2D coverage extensions. We also present a theoretical study to benchmark our proposed approach for a basic scenario that is less complicated to model mathematically. Our simulation results show that we are indeed able to predict coverage extensions for multi-hop scenarios and quantify the effects of street-system characteristics and pedestrian mobility on the connection time of devices to the base station (BS). To our knowledge, this is the first study that applies agent-based simulations for coverage extensions in D2D.
Six Questions about 6G
Venue: arXiv cs.NI 2022
Year: 2022

Although 5G (Fifth Generation) mobile technology is still in the rollout phase, research and development of 6G (Sixth Generation) wireless have already begun. This paper is an introduction to 6G wireless networks, covering the main drivers for 6G, some of the expected use cases, some of the technical challenges in 6G, example areas that will require research and new technologies, the expected timeline for 6G development and rollout, and a list of some important 6G initiatives world-wide. It was compiled as part of a series of workshops about 6G held by Thinknet 6G and MUENCHNER KREIS in 2021.
Benchmarking Conventional Vision Models on Neuromorphic Fall Detection and Action Recognition Dataset
Venue: arXiv cs.CV 2022
Year: 2022

Neuromorphic vision-based sensors are gaining popularity in recent years with their ability to capture Spatio-temporal events with low power sensing. These sensors record events or spikes over traditional cameras which helps in preserving the privacy of the subject being recorded. These events are captured as per-pixel brightness changes and the output data stream is encoded with time, location, and pixel intensity change information. This paper proposes and benchmarks the performance of fine-tuned conventional vision models on neuromorphic human action recognition and fall detection datasets. The Spatio-temporal event streams from the Dynamic Vision Sensing cameras are encoded into a standard sequence image frames. These video frames are used for benchmarking conventional deep learning-based architectures. In this proposed approach, we fine-tuned the state-of-the-art vision models for this Dynamic Vision Sensing (DVS) application and named these models as DVS-R2+1D, DVS-CSN, DVS-C2D, DVS-SlowFast, DVS-X3D, and DVS-MViT. Upon comparing the performance of these models, we see the current state-of-the-art MViT based architecture DVS-MViT outperforms all the other models with an accuracy of 0.958 and an F-1 score of 0.958. The second best is the DVS-C2D with an accuracy of 0.916 and an F-1 score of 0.916. Third and Fourth are DVS-R2+1D and DVS-SlowFast with an accuracy of 0.875 and 0.833 and F-1 score of 0.875 and 0.861 respectively. DVS-CSN and DVS-X3D were the least performing models with an accuracy of 0.708 and 0.625 and an F1 score of 0.722 and 0.625 respectively.
The Price of Majority Support
Venue: arXiv cs.LG 2022
Year: 2022

We consider the problem of finding a compromise between the opinions of a group of individuals on a number of mutually independent, binary topics. In this paper, we quantify the loss in representativeness that results from requiring the outcome to have majority support, in other words, the "price of majority support". Each individual is assumed to support an outcome if they agree with the outcome on at least as many topics as they disagree on. Our results can also be seen as quantifying Anscombes paradox which states that topic-wise majority outcome may not be supported by a majority. To measure the representativeness of an outcome, we consider two metrics. First, we look for an outcome that agrees with a majority on as many topics as possible. We prove that the maximum number such that there is guaranteed to exist an outcome that agrees with a majority on this number of topics and has majority support, equals $\ceil{(t+1)/2}$ where $t$ is the total number of topics. Second, we count the number of times a voter opinion on a topic matches the outcome on that topic. The goal is to find the outcome with majority support with the largest number of matches. We consider the ratio between this number and the number of matches of the overall best outcome which may not have majority support. We try to find the maximum ratio such that an outcome with majority support and this ratio of matches compared to the overall best is guaranteed to exist. For 3 topics, we show this ratio to be $5/6\approx 0.83$. In general, we prove an upper bound that comes arbitrarily close to $2\sqrt{6}-4\approx 0.90$ as $t$ tends to infinity. Furthermore, we numerically compute a better upper and a non-matching lower bound in the relevant range for $t$.
Optimal Transport Tools (OTT): A JAX Toolbox for all things Wasserstein
Venue: arXiv cs.LG 2022
Year: 2022

Optimal transport tools (OTT-JAX) is a Python toolbox that can solve optimal transport problems between point clouds and histograms. The toolbox builds on various JAX features, such as automatic and custom reverse mode differentiation, vectorization, just-in-time compilation and accelerators support. The toolbox covers elementary computations, such as the resolution of the regularized OT problem, and more advanced extensions, such as barycenters, Gromov-Wasserstein, low-rank solvers, estimation of convex maps, differentiable generalizations of quantiles and ranks, and approximate OT between Gaussian mixtures. The toolbox code is available at \texttt{https://github.com/ott-jax/ott}
Improving Group Testing via Gradient Descent
Venue: arXiv cs.IT 2022
Year: 2022

We study the problem of group testing with non-identical, independent priors. So far, the pooling strategies that have been proposed in the literature take the following approach: a hand-crafted test design along with a decoding strategy is proposed, and guarantees are provided on how many tests are sufficient in order to identify all infections in a population. In this paper, we take a different, yet perhaps more practical, approach: we fix the decoder and the number of tests, and we ask, given these, what is the best test design one could use? We explore this question for the Definite Non-Defectives (DND) decoder. We formulate a (non-convex) optimization problem, where the objective function is the expected number of errors for a particular design. We find approximate solutions via gradient descent, which we further optimize with informed initialization. We illustrate through simulations that our method can achieve significant performance improvement over traditional approaches.
Machine Learning Based Relative Orbit Transfer for Swarm Spacecraft Motion Planning
Venue: arXiv cs.RO 2022
Year: 2022

In this paper we describe a machine learning based framework for spacecraft swarm trajectory planning. In particular, we focus on coordinating motions of multi-spacecraft in formation flying through passive relative orbit(PRO) transfers. Accounting for spacecraft dynamics while avoiding collisions between the agents makes spacecraft swarm trajectory planning difficult. Centralized approaches can be used to solve this problem, but are computationally demanding and scale poorly with the number of agents in the swarm. As a result, centralized algorithms are ill-suited for real time trajectory planning on board small spacecraft (e.g. CubeSats) comprising the swarm. In our approach a neural network is used to approximate solutions of a centralized method. The necessary training data is generated using a centralized convex optimization framework through which several instances of the n=10 spacecraft swarm trajectory planning problem are solved. We are interested in answering the following questions which will give insight on the potential utility of deep learning-based approaches to the multi-spacecraft motion planning problem: 1) Can neural networks produce feasible trajectories that satisfy safety constraints (e.g. collision avoidance) and low in fuel cost? 2) Can a neural network trained using n spacecraft data be used to solve problems for spacecraft swarms of differing size?
Constellation: An Edge-Based Semantic Runtime System for Internet of Things Applications
Venue: arXiv cs.DC 2022
Year: 2022

With the global Internet of Things IoT market size predicted to grow to over 1 trillion dollars in the next 5 years, many large corporations are scrambling to solidify their product line as the defacto device suite for consumers. This has led to each corporation developing their devices in a siloed environment with unique protocols and runtime frameworks that explicitly exclude the ability to work with the competitions devices. This development silo has created problems with programming complexity for application developers as well as concurrency and scalability limitations for applications that involve a network of IoT devices. The Constellation project is a distributed IoT runtime system that attempts to address these challenges by creating an operating system layer that decouples applications from devices. This layer provides mechanisms designed to allow applications to interface with an underlying substrate of IoT devices while abstracting away the complexities of application concurrency, device interoperability, and system scalability. This paper provides an overview of the Constellation system as well as details four new project expansions to improve system scalability.
Schema-Free Dependency Parsing via Sequence Generation
Venue: arXiv cs.CL 2022
Year: 2022

Dependency parsing aims to extract syntactic dependency structure or semantic dependency structure for sentences. Existing methods suffer the drawbacks of lacking universality or highly relying on the auxiliary decoder. To remedy these drawbacks, we propose to achieve universal and schema-free Dependency Parsing (DP) via Sequence Generation (SG) DPSG by utilizing only the pre-trained language model (PLM) without any auxiliary structures or parsing algorithms. We first explore different serialization designing strategies for converting parsing structures into sequences. Then we design dependency units and concatenate these units into the sequence for DPSG. Thanks to the high flexibility of the sequence generation, our DPSG can achieve both syntactic DP and semantic DP using a single model. By concatenating the prefix to indicate the specific schema with the sequence, our DPSG can even accomplish multi-schemata parsing. The effectiveness of our DPSG is demonstrated by the experiments on widely used DP benchmarks, i.e., PTB, CODT, SDP15, and SemEval16. DPSG achieves comparable results with the first-tier methods on all the benchmarks and even the state-of-the-art (SOTA) performance in CODT and SemEval16. This paper demonstrates our DPSG has the potential to be a new parsing paradigm. We will release our codes upon acceptance.
Networked Restless Multi-Armed Bandits for Mobile Interventions
Venue: arXiv cs.LG 2022
Year: 2022

Motivated by a broad class of mobile intervention problems, we propose and study restless multi-armed bandits (RMABs) with network effects. In our model, arms are partially recharging and connected through a graph, so that pulling one arm also improves the state of neighboring arms, significantly extending the previously studied setting of fully recharging bandits with no network effects. In mobile interventions, network effects may arise due to regular population movements (such as commuting between home and work). We show that network effects in RMABs induce strong reward coupling that is not accounted for by existing solution methods. We propose a new solution approach for networked RMABs, exploiting concavity properties which arise under natural assumptions on the structure of intervention effects. We provide sufficient conditions for optimality of our approach in idealized settings and demonstrate that it empirically outperforms state-of-the art baselines in three mobile intervention domains using real-world graphs.
Benchmarking Resource Usage for Efficient Distributed Deep Learning
Venue: arXiv cs.LG 2022
Year: 2022

Deep learning (DL) workflows demand an ever-increasing budget of compute and energy in order to achieve outsized gains. Neural architecture searches, hyperparameter sweeps, and rapid prototyping consume immense resources that can prevent resource-constrained researchers from experimenting with large models and carry considerable environmental impact. As such, it becomes essential to understand how different deep neural networks (DNNs) and training leverage increasing compute and energy resources -- especially specialized computationally-intensive models across different domains and applications. In this paper, we conduct over 3,400 experiments training an array of deep networks representing various domains/tasks -- natural language processing, computer vision, and chemistry -- on up to 424 graphics processing units (GPUs). During training, our experiments systematically vary compute resource characteristics and energy-saving mechanisms such as power utilization and GPU clock rate limits to capture and illustrate the different trade-offs and scaling behaviors each representative model exhibits under various resource and energy-constrained regimes. We fit power law models that describe how training time scales with available compute resources and energy constraints. We anticipate that these findings will help inform and guide high-performance computing providers in optimizing resource utilization, by selectively reducing energy consumption for different deep learning tasks/workflows with minimal impact on training.
ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noise
Venue: arXiv cs.LG 2022
Year: 2022

In recent years, large pre-trained Transformer-based language models have led to dramatic improvements in many natural language understanding tasks. To train these models with increasing sizes, many neural network practitioners attempt to increase the batch sizes in order to leverage multiple GPUs to improve training speed. However, increasing the batch size often makes the optimization more difficult, leading to slow convergence or poor generalization that can require orders of magnitude more training time to achieve the same model quality. In this paper, we explore the steepness of the loss landscape of large-batch optimization for adapting pre-trained Transformer-based language models to domain-specific tasks and find that it tends to be highly complex and irregular, posing challenges to generalization on downstream tasks. To tackle this challenge, we propose ScaLA, a novel and efficient method to accelerate the adaptation speed of pre-trained transformer networks. Different from prior methods, we take a sequential game-theoretic approach by adding lightweight adversarial noise into large-batch optimization, which significantly improves adaptation speed while preserving model generalization. Experiment results show that ScaLA attains 2.7--9.8$\times$ adaptation speedups over the baseline for GLUE on BERT-base and RoBERTa-large, while achieving comparable and sometimes higher accuracy than the state-of-the-art large-batch optimization methods. Finally, we also address the theoretical aspect of large-batch optimization with adversarial noise and provide a theoretical convergence rate analysis for ScaLA using techniques for analyzing non-convex saddle-point problems.
Dissimilar Redundancy in DeFi
Venue: arXiv cs.CR 2022
Year: 2022

The meteoric rise of Decentralized Finance (DeFi) has been accompanied by a plethora of frequent and often financially devastating attacks on its protocols There have been over 70 exploits of DeFi protocols, with the total of lost funds amounting to approximately 1.5bn USD. In this paper, we introduce a new approach to minimizing the frequency and severity of such attacks: dissimilar redundancy for smart contracts. In a nutshell, the idea is to implement a program logic more than once, ideally using different programming languages. Then, for each implementation, the results should match before allowing the state of the blockchain to change. This is inspired by and has clear parallels to the field of avionics, where on account of the safety-critical environment, flight control systems typically feature multiple redundant implementations. We argue that the high financial stakes in DeFi protocols merit a conceptually similar approach, and we provide a novel algorithm for implementing dissimilar redundancy for smart contracts.
The HCCL-DKU system for fake audio generation task of the 2022 ICASSP ADD Challenge
Venue: arXiv cs.SD 2022
Year: 2022

The voice conversion task is to modify the speaker identity of continuous speech while preserving the linguistic content. Generally, the naturalness and similarity are two main metrics for evaluating the conversion quality, which has been improved significantly in recent years. This paper presents the HCCL-DKU entry for the fake audio generation task of the 2022 ICASSP ADD challenge. We propose a novel ppg-based voice conversion model that adopts a fully end-to-end structure. Experimental results show that the proposed method outperforms other conversion models, including Tacotron-based and Fastspeech-based models, on conversion quality and spoofing performance against anti-spoofing systems. In addition, we investigate several post-processing methods for better spoofing power. Finally, we achieve second place with a deception success rate of 0.916 in the ADD challenge.
Le Processus Powered Dirichlet-Hawkes comme A Priori Flexible pour Clustering Temporel de Textes
Venue: arXiv cs.CL 2022
Year: 2022

The textual content of a document and its publication date are intertwined. For example, the publication of a news article on a topic is influenced by previous publications on similar issues, according to underlying temporal dynamics. However, it can be challenging to retrieve meaningful information when textual information conveys little. Furthermore, the textual content of a document is not always correlated to its temporal dynamics. We develop a method to create clusters of textual documents according to both their content and publication time, the Powered Dirichlet-Hawkes process (PDHP). PDHP yields significantly better results than state-of-the-art models when temporal information or textual content is weakly informative. PDHP also alleviates the hypothesis that textual content and temporal dynamics are perfectly correlated. We demonstrate that PDHP generalizes previous work --such as DHP and UP. Finally, we illustrate a possible application using a real-world dataset from Reddit.