OpenRaiser/Intern-Atlas huggingface dataset:
Deep Video Prediction for Time Series Forecasting
Venue: arXiv cs.CV 2021
Year: 2021

Time series forecasting is essential for decision making in many domains. In this work, we address the challenge of predicting prices evolution among multiple potentially interacting financial assets. A solution to this problem has obvious importance for governments, banks, and investors. Statistical methods such as Auto Regressive Integrated Moving Average (ARIMA) are widely applied to these problems. In this paper, we propose to approach economic time series forecasting of multiple financial assets in a novel way via video prediction. Given past prices of multiple potentially interacting financial assets, we aim to predict the prices evolution in the future. Instead of treating the snapshot of prices at each time point as a vector, we spatially layout these prices in 2D as an image, such that we can harness the power of CNNs in learning a latent representation for these financial assets. Thus, the history of these prices becomes a sequence of images, and our goal becomes predicting future images. We build on a state-of-the-art video prediction method for forecasting future images. Our experiments involve the prediction task of the price evolution of nine financial assets traded in U.S. stock markets. The proposed method outperforms baselines including ARIMA, Prophet, and variations of the proposed method, demonstrating the benefits of harnessing the power of CNNs in the problem of economic time series forecasting.
Spatio-Temporal Look-Ahead Trajectory Prediction using Memory Neural Network
Venue: arXiv cs.RO 2021
Year: 2021

Prognostication of vehicle trajectories in unknown environments is intrinsically a challenging and difficult problem to solve. The behavior of such vehicles is highly influenced by surrounding traffic, road conditions, and rogue participants present in the environment. Moreover, the presence of pedestrians, traffic lights, stop signs, etc., makes it much harder to infer the behavior of various traffic agents. This paper attempts to solve the problem of Spatio-temporal look-ahead trajectory prediction using a novel recurrent neural network called the Memory Neuron Network. The Memory Neuron Network (MNN) attempts to capture the input-output relationship between the past positions and the future positions of the traffic agents. The proposed model is computationally less intensive and has a simple architecture as compared to other deep learning models that utilize LSTMs and GRUs. It is then evaluated on the publicly available NGSIM dataset and its performance is compared with several state-of-art algorithms. Additionally, the performance is also evaluated on a custom synthetic dataset generated from the CARLA simulator. It is seen that the proposed model outperforms the existing state-of-art algorithms. Finally, the model is integrated with the CARLA simulator to test its robustness in real-time traffic scenarios.
A Computation Offloading Model over Collaborative Cloud-Edge Networks with Optimal Transport Theory
Venue: arXiv cs.NI 2021
Year: 2021

As novel applications spring up in future network scenarios, the requirements on network service capabilities for differentiated services or burst services are diverse. Aiming at the research of collaborative computing and resource allocation in edge scenarios, migrating computing tasks to the edge and cloud for computing requires a comprehensive consideration of energy consumption, bandwidth, and delay. Our paper proposes a collaboration mechanism based on computation offloading, which is flexible and customizable to meet the diversified requirements of differentiated networks. This mechanism handles the terminal's differentiated computing tasks by establishing a collaborative computation offloading model between the cloud server and edge server. Experiments show that our method has more significant improvements over regular optimization algorithms, including reducing the execution time of computing tasks, improving the utilization of server resources, and decreasing the terminal's energy consumption.
Further results on the 2-adic complexity of a class of balanced generalized cyclotomic sequences
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, the 2-adic complexity of a class of balanced Whiteman generalized cyclotomic sequences of period $pq$ is considered. Through calculating the determinant of the circulant matrix constructed by one of these sequences, we derive a lower bound on the 2-adic complexity of the corresponding sequence, which further expands our previous work (Zhao C, Sun Y and Yan T. Study on 2-adic complexity of a class of balanced generalized cyclotomic sequences. Journal of Cryptologic Research,6(4):455-462, 2019). The result shows that the 2-adic complexity of this class of sequences is large enough to resist the attack of the rational approximation algorithm(RAA) for feedback with carry shift registers(FCSRs), i.e., it is in fact lower bounded by $pq-p-q-1$, which is far larger than one half of the period of the sequences. Particularly, the 2-adic complexity is maximal if suitable parameters are chosen.
A new upper bound and optimal constructions of equi-difference conflict-avoiding codes on constant weight
Venue: arXiv cs.IT 2021
Year: 2021

Conflict-avoiding codes (CACs) have been used in multiple-access collision channel without feedback. The size of a CAC is the number of potential users that can be supported in the system. A code with maximum size is called optimal. The use of an optimal CAC enables the largest possible number of asynchronous users to transmit information efficiently and reliably. In this paper, a new upper bound on the maximum size of arbitrary equi-difference CAC is presented. Furthermore, three optimal constructions of equi-difference CACs are also given. One is a generalized construction for prime length $L=p$ and the other two are for two-prime length $L=pq$.
From Universal Language Model to Downstream Task: Improving RoBERTa-Based Vietnamese Hate Speech Detection
Venue: arXiv cs.CL 2021
Year: 2021

Natural language processing is a fast-growing field of artificial intelligence. Since the Transformer was introduced by Google in 2017, a large number of language models such as BERT, GPT, and ELMo have been inspired by this architecture. These models were trained on huge datasets and achieved state-of-the-art results on natural language understanding. However, fine-tuning a pre-trained language model on much smaller datasets for downstream tasks requires a carefully-designed pipeline to mitigate problems of the datasets such as lack of training data and imbalanced data. In this paper, we propose a pipeline to adapt the general-purpose RoBERTa language model to a specific text classification task: Vietnamese Hate Speech Detection. We first tune the PhoBERT on our dataset by re-training the model on the Masked Language Model task; then, we employ its encoder for text classification. In order to preserve pre-trained weights while learning new feature representations, we further utilize different training techniques: layer freezing, block-wise learning rate, and label smoothing. Our experiments proved that our proposed pipeline boosts the performance significantly, achieving a new state-of-the-art on Vietnamese Hate Speech Detection campaign with 0.7221 F1 score.
Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis
Venue: arXiv cs.LG 2021
Year: 2021

The susceptibility of deep neural networks to untrustworthy predictions, including out-of-distribution (OOD) data and adversarial examples, still prevent their widespread use in safety-critical applications. Most existing methods either require a re-training of a given model to achieve robust identification of adversarial attacks or are limited to out-of-distribution sample detection only. In this work, we propose a geometric gradient analysis (GGA) to improve the identification of untrustworthy predictions without retraining of a given model. GGA analyzes the geometry of the loss landscape of neural networks based on the saliency maps of their respective input. To motivate the proposed approach, we provide theoretical connections between gradients' geometrical properties and local minima of the loss function. Furthermore, we demonstrate that the proposed method outperforms prior approaches in detecting OOD data and adversarial attacks, including state-of-the-art and adaptive attacks.
NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques
Venue: arXiv cs.CL 2021
Year: 2021

Toxicity detection of text has been a popular NLP task in the recent years. In SemEval-2021 Task-5 Toxic Spans Detection, the focus is on detecting toxic spans within passages. Most state-of-the-art span detection approaches employ various techniques, each of which can be broadly classified into Token Classification or Span Prediction approaches. In our paper, we explore simple versions of both of these approaches and their performance on the task. Specifically, we use BERT-based models -- BERT, RoBERTa, and SpanBERT for both approaches. We also combine these approaches and modify them to bring improvements for Toxic Spans prediction. To this end, we investigate results on four hybrid approaches -- Multi-Span, Span+Token, LSTM-CRF, and a combination of predicted offsets using union/intersection. Additionally, we perform a thorough ablative analysis and analyze our observed results. Our best submission -- a combination of SpanBERT Span Predictor and RoBERTa Token Classifier predictions -- achieves an F1 score of 0.6753 on the test set. Our best post-eval F1 score is 0.6895 on intersection of predicted offsets from top-3 RoBERTa Token Classification checkpoints. These approaches improve the performance by 3% on average than those of the shared baseline models -- RNNSL and SpaCy NER.
Two Problems about Monomial Bent Functions
Venue: arXiv cs.IT 2021
Year: 2021

In 2008, Langevin and Leander determined the dual function of three classes of monomial bent functions with the help of Stickelberger's theorem: Dillon, Gold and Kasami. In their paper, they proposed one very strong condition such that their method works, and showed that both Gold exponent and Kasami exponent satisfy this condition. In 2018, Pott {\em et al.} investigated the issue of vectorial functions with maximal number of bent components. They found one class of binomial functions which attains the upper bound. They also proposed an open problem regarding monomial function with maximal number of bent components. In this paper, we obtain an interesting result about the condition of Langevin and Leander, and solve the open problem of Pott {\em et al.}. Specifically, we show that: 1) for a monomial bent function over $\mathbb{F}_{2^{2k}}$, if the exponent satisfies the first part of the condition of Langevin and Leander, then it satisfies the entire condition; 2) $x^{2^k+1}$ is the only monomial function over $\mathbb{F}_{2^{2k}}$ which has maximal number of bent components. Fortunately, as a consequence, we also solve an open problem of Ness and Helleseth in 2006.
vrCAPTCHA: Exploring CAPTCHA Designs in Virtual Reality
Venue: arXiv cs.HC 2021
Year: 2021

With the popularity of online access in virtual reality (VR) devices, it will become important to investigate exclusive and interactive CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) designs for VR devices. In this paper, we first present four traditional two-dimensional (2D) CAPTCHAs (i.e., text-based, image-rotated, image-puzzled, and image-selected CAPTCHAs) in VR. Then, based on the three-dimensional (3D) interaction characteristics of VR devices, we propose two vrCAPTCHA design prototypes (i.e., task-driven and bodily motion-based CAPTCHAs). We conducted a user study with six participants for exploring the feasibility of our two vrCAPTCHAs and traditional CAPTCHAs in VR. We believe that our two vrCAPTCHAs can be an inspiration for the further design of CAPTCHAs in VR.
Similarity measure for sparse time course data based on Gaussian processes
Venue: arXiv cs.LG 2021
Year: 2021

We propose a similarity measure for sparsely sampled time course data in the form of a log-likelihood ratio of Gaussian processes (GP). The proposed GP similarity is similar to a Bayes factor and provides enhanced robustness to noise in sparse time series, such as those found in various biological settings, e.g., gene transcriptomics. We show that the GP measure is equivalent to the Euclidean distance when the noise variance in the GP is negligible compared to the noise variance of the signal. Our numerical experiments on both synthetic and real data show improved performance of the GP similarity when used in conjunction with two distance-based clustering methods.
On the Impact of Interpretability Methods in Active Image Augmentation Method
Venue: arXiv cs.CV 2021
Year: 2021

Robustness is a significant constraint in machine learning models. The performance of the algorithms must not deteriorate when training and testing with slightly different data. Deep neural network models achieve awe-inspiring results in a wide range of applications of computer vision. Still, in the presence of noise or region occlusion, some models exhibit inaccurate performance even with data handled in training. Besides, some experiments suggest deep learning models sometimes use incorrect parts of the input information to perform inference. Activate Image Augmentation (ADA) is an augmentation method that uses interpretability methods to augment the training data and improve its robustness to face the described problems. Although ADA presented interesting results, its original version only used the Vanilla Backpropagation interpretability to train the U-Net model. In this work, we propose an extensive experimental analysis of the interpretability method's impact on ADA. We use five interpretability methods: Vanilla Backpropagation, Guided Backpropagation, GradCam, Guided GradCam, and InputXGradient. The results show that all methods achieve similar performance at the ending of training, but when combining ADA with GradCam, the U-Net model presented an impressive fast convergence.
Binary Subspace Chirps
Venue: arXiv cs.IT 2021
Year: 2021

We describe in details the interplay between binary symplectic geometry and quantum computation, with the ultimate goal of constructing highly structured codebooks. The Binary Chirps (BCs) are Complex Grassmannian Lines in $N = 2^m$ dimensions used in deterministic compressed sensing and random/unsourced multiple access in wireless networks. Their entries are fourth roots of unity and can be described in terms of second order Reed-Muller codes. The Binary Subspace Chirps (BSSCs) are a unique collection of BCs of $\textit{ranks}$ ranging from $r=0$ to $r = m$, embedded in $N$ dimensions according to an on-off pattern determined by a rank $r$ binary subspace. This yields a codebook that is asymptotically 2.38 times larger than the codebook of BCs, has the same minimum chordal distance as the codebook of BCs, and the alphabet is minimally extended from $\{\pm 1,\pm i\}$ to $\{\pm 1,\pm i, 0\}$. Equivalently, we show that BSSCs are stabilizer states, and we characterize them as columns of a well-controlled collection of Clifford matrices. By construction, the BSSCs inherit all the properties of BCs, which in turn makes them good candidates for a variety of applications. For applications in wireless communication, we use the rich algebraic structure of BSSCs to construct a low complexity decoding algorithm that is reliable against Gaussian noise. In simulations, BSSCs exhibit an error probability comparable or slightly lower than BCs, both for single-user and multi-user transmissions.
Temporal Energy Analysis of Symbol Sequences for Fiber Nonlinear Interference Modelling via Energy Dispersion Index
Venue: arXiv cs.IT 2021
Year: 2021

The stationary statistical properties of independent, identically distributed (i.i.d.) input symbols provide insights on the induced nonlinear interference (NLI) during fiber transmission. For example, kurtosis is known to predict the modulation format-dependent NLI. These statistical properties can be used in the design of probabilistic amplitude shaping (PAS), which is a popular scheme that relies on an amplitude shaper for increasing spectral efficiencies of fiber-optic systems. One property of certain shapers used in PAS -- including constant-composition distribution matchers -- that is often overlooked is that a time-dependency between amplitudes is introduced. This dependency results in symbols that are non-i.i.d., which have time-varying statistical properties. Somewhat surprisingly, the effective signal-to-noise ratio (SNR) in PAS has been shown to increase when the shaping blocklength decreases. This blocklength dependency of SNR has been attributed to time-varying statistical properties of the symbol sequences, in particular, to variation of the symbol energies. In this paper, we investigate the temporal energy behavior of symbol sequences, and introduce a new metric called energy dispersion index (EDI). EDI captures the time-varying statistical properties of symbol energies. Numerical results show strong correlations between EDI and effective SNR, with absolute correlation coefficients above 99% for different transmission distances.
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization
Venue: arXiv cs.LG 2021
Year: 2021

Numerous empirical evidences have corroborated the importance of noise in nonconvex optimization problems. The theory behind such empirical observations, however, is still largely unknown. This paper studies this fundamental problem through investigating the nonconvex rectangular matrix factorization problem, which has infinitely many global minima due to rotation and scaling invariance. Hence, gradient descent (GD) can converge to any optimum, depending on the initialization. In contrast, we show that a perturbed form of GD with an arbitrary initialization converges to a global optimum that is uniquely determined by the injected noise. Our result implies that the noise imposes implicit bias towards certain optima. Numerical experiments are provided to support our theory.
References in Wikipedia: The Editors' Perspective
Venue: arXiv cs.CL 2021
Year: 2021

References are an essential part of Wikipedia. Each statement in Wikipedia should be referenced. In this paper, we explore the creation and collection of references for new Wikipedia articles from an editors' perspective. We map out the workflow of editors when creating a new article, emphasising how they select references.
Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Communication is a important factor that enables agents work cooperatively in multi-agent reinforcement learning (MARL). Most previous work uses continuous message communication whose high representational capacity comes at the expense of interpretability. Allowing agents to learn their own discrete message communication protocol emerged from a variety of domains can increase the interpretability for human designers and other agents.This paper proposes a method to generate discrete messages analogous to human languages, and achieve communication by a broadcast-and-listen mechanism based on self-attention. We show that discrete message communication has performance comparable to continuous message communication but with much a much smaller vocabulary size.Furthermore, we propose an approach that allows humans to interactively send discrete messages to agents.
Disentangling brain heterogeneity via semi-supervised deep-learning and MRI: dimensional representations of Alzheimer's Disease
Venue: arXiv cs.LG 2021
Year: 2021

Heterogeneity of brain diseases is a challenge for precision diagnosis/prognosis. We describe and validate Smile-GAN (SeMI-supervised cLustEring-Generative Adversarial Network), a novel semi-supervised deep-clustering method, which dissects neuroanatomical heterogeneity, enabling identification of disease subtypes via their imaging signatures relative to controls. When applied to MRIs (2 studies; 2,832 participants; 8,146 scans) including cognitively normal individuals and those with cognitive impairment and dementia, Smile-GAN identified 4 neurodegenerative patterns/axes: P1, normal anatomy and highest cognitive performance; P2, mild/diffuse atrophy and more prominent executive dysfunction; P3, focal medial temporal atrophy and relatively greater memory impairment; P4, advanced neurodegeneration. Further application to longitudinal data revealed two distinct progression pathways: P1$\rightarrow$P2$\rightarrow$P4 and P1$\rightarrow$P3$\rightarrow$P4. Baseline expression of these patterns predicted the pathway and rate of future neurodegeneration. Pattern expression offered better yet complementary performance in predicting clinical progression, compared to amyloid/tau. These deep-learning derived biomarkers offer promise for precision diagnostics and targeted clinical trial recruitment.
A New Algorithm for Euclidean Shortest Paths in the Plane
Venue: arXiv cs.CG 2021
Year: 2021

Given a set of pairwise disjoint polygonal obstacles in the plane, finding an obstacle-avoiding Euclidean shortest path between two points is a classical problem in computational geometry and has been studied extensively. Previously, Hershberger and Suri [SIAM J. Comput. 1999] gave an algorithm of $O(n\log n)$ time and $O(n\log n)$ space, where $n$ is the total number of vertices of all obstacles. Recently, by modifying Hershberger and Suri's algorithm, Wang [SODA 2021] reduced the space to $O(n)$ while the runtime of the algorithm is still $O(n\log n)$. In this paper, we present a new algorithm of $O(n+h\log h)$ time and $O(n)$ space, provided that a triangulation of the free space is given, where $h$ is the number of obstacles. The algorithm, which improves the previous work when $h=o(n)$, is optimal in both time and space as $Ω(n+h\log h)$ is a lower bound on the runtime. Our algorithm builds a shortest path map for a source point $s$, so that given any query point $t$, the shortest path length from $s$ to $t$ can be computed in $O(\log n)$ time and a shortest $s$-$t$ path can be produced in additional time linear in the number of edges of the path.
AniGAN: Style-Guided Generative Adversarial Networks for Unsupervised Anime Face Generation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a novel framework to translate a portrait photo-face into an anime appearance. Our aim is to synthesize anime-faces which are style-consistent with a given reference anime-face. However, unlike typical translation tasks, such anime-face translation is challenging due to complex variations of appearances among anime-faces. Existing methods often fail to transfer the styles of reference anime-faces, or introduce noticeable artifacts/distortions in the local shapes of their generated faces. We propose AniGAN, a novel GAN-based translator that synthesizes high-quality anime-faces. Specifically, a new generator architecture is proposed to simultaneously transfer color/texture styles and transform local facial shapes into anime-like counterparts based on the style of a reference anime-face, while preserving the global structure of the source photo-face. We propose a double-branch discriminator to learn both domain-specific distributions and domain-shared distributions, helping generate visually pleasing anime-faces and effectively mitigate artifacts. Extensive experiments on selfie2anime and a new face2anime dataset qualitatively and quantitatively demonstrate the superiority of our method over state-of-the-art methods. The new dataset is available at https://github.com/bing-li-ai/AniGAN .