OpenRaiser/Intern-Atlas huggingface dataset:
Sets of Low Correlation Sequences from Cyclotomy
Venue: arXiv cs.IT 2021
Year: 2021

Low correlation (finite length) sequences are used in communications and remote sensing. One seeks codebooks of sequences in which each sequence has low aperiodic autocorrelation at all nonzero shifts, and each pair of distinct sequences has low aperiodic crosscorrelation at all shifts. An overall criterion of codebook quality is the demerit factor, which normalizes all sequences to unit Euclidean norm, sums the squared magnitudes of all the correlations between every pair of sequences in the codebook (including sequences with themselves to cover autocorrelations), and divides by the square of the number of sequences in the codebook. This demerit factor is expected to be $1+1/N-1/(\ell N)$ for a codebook of $N$ randomly selected binary sequences of length $\ell$, but we want demerit factors much closer to the absolute minimum value of $1$. For each $N$ such that there is an $N\times N$ Hadamard matrix, we use cyclotomy to construct an infinite family of codebooks of binary sequences, in which each codebook has $N-1$ sequences of length $p$, where $p$ runs through the primes with $N\mid p-1$. As $p$ tends to infinity, the demerit factor of the codebooks tends to $1+1/(6(N-1))$, and the maximum magnitude of the undesirable correlations (crosscorrelations between distinct sequences and off-peak autocorrelations) is less than a small constant times $\sqrt{p}\log(p)$. This construction also generalizes to nonbinary sequences.
LeSICiN: A Heterogeneous Graph-based Approach for Automatic Legal Statute Identification from Indian Legal Documents
Venue: arXiv cs.CL 2021
Year: 2021

The task of Legal Statute Identification (LSI) aims to identify the legal statutes that are relevant to a given description of Facts or evidence of a legal case. Existing methods only utilize the textual content of Facts and legal articles to guide such a task. However, the citation network among case documents and legal statutes is a rich source of additional information, which is not considered by existing models. In this work, we take the first step towards utilising both the text and the legal citation network for the LSI task. We curate a large novel dataset for this task, including Facts of cases from several major Indian Courts of Law, and statutes from the Indian Penal Code (IPC). Modeling the statutes and training documents as a heterogeneous graph, our proposed model LeSICiN can learn rich textual and graphical features, and can also tune itself to correlate these features. Thereafter, the model can be used to inductively predict links between test documents (new nodes whose graphical features are not available to the model) and statutes (existing nodes). Extensive experiments on the dataset show that our model comfortably outperforms several state-of-the-art baselines, by exploiting the graphical structure along with textual features. The dataset and our codes are available at https://github.com/Law-AI/LeSICiN.
Deep Graph Clustering via Dual Correlation Reduction
Venue: arXiv cs.LG 2021
Year: 2021

Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different groups, has attracted intensive attention in recent years. However, we observe that, in the process of node encoding, existing methods suffer from representation collapse which tends to map all data into the same representation. Consequently, the discriminative capability of the node representation is limited, leading to unsatisfied clustering performance. To address this issue, we propose a novel self-supervised deep graph clustering method termed Dual Correlation Reduction Network (DCRN) by reducing information correlation in a dual manner. Specifically, in our method, we first design a siamese network to encode samples. Then by forcing the cross-view sample correlation matrix and cross-view feature correlation matrix to approximate two identity matrices, respectively, we reduce the information correlation in the dual-level, thus improving the discriminative capability of the resulting features. Moreover, in order to alleviate representation collapse caused by over-smoothing in GCN, we introduce a propagation regularization term to enable the network to gain long-distance information with the shallow network structure. Extensive experimental results on six benchmark datasets demonstrate the effectiveness of the proposed DCRN against the existing state-of-the-art methods.
Modeling Prejudice and Its Effect on Societal Prosperity
Venue: arXiv cs.MA 2021
Year: 2021

Existing studies on prejudice, which is important in multi-group dynamics in societies, focus on the social-psychological knowledge behind the processes involving prejudice and its propagation. We instead create a multi-agent framework that simulates the propagation of prejudice and measures its tangible impact on the prosperity of individuals as well as of larger social structures, including groups and factions within. Groups in society help us define prejudice, and factions represent smaller tight-knit circles of individuals with similar opinions. We model social interactions using the Continuous Prisoner's Dilemma (CPD) and a type of agent called a prejudiced agent, whose cooperation is affected by a prejudice attribute, updated over time based both on the agent's own experiences and those of others in its faction. Our simulations show that modeling prejudice as an exclusively out-group phenomenon generates implicit in-group promotion, which eventually leads to higher relative prosperity of the prejudiced population. This skew in prosperity is shown to be correlated to factors such as size difference between groups and the number of prejudiced agents in a group. Although prejudiced agents achieve higher prosperity within prejudiced societies, their presence degrades the overall prosperity levels of their societies. Our proposed system model can serve as a basis for promoting a deeper understanding of origins, propagation, and ramifications of prejudice through rigorous simulative studies grounded in apt theoretical backgrounds. This can help conduct impactful research on prominent social issues such as racism, religious discrimination, and unfair immigrant treatment. This model can also serve as a foundation to study other socio-psychological phenomena in tandem with prejudice such as the distribution of wealth, social status, and ethnocentrism in a society.
Anomaly Detection in Cyber-Physical Systems: Reconstruction of a Prediction Error Feature Space
Venue: arXiv cs.CR 2021
Year: 2021

Cyber-physical systems are infrastructures that use digital information such as network communications and sensor readings to control entities in the physical world. Many cyber-physical systems in airports, hospitals and nuclear power plants are regarded as critical infrastructures since a disruption of its normal functionality can result in negative consequences for the society. In the last few years, some security solutions for cyber-physical systems based on artificial intelligence have been proposed. Nevertheless, knowledge domain is required to properly setup and train artificial intelligence algorithms. Our work proposes a novel anomaly detection framework based on error space reconstruction, where genetic algorithms are used to perform hyperparameter optimization of machine learning methods. The proposed method achieved an F1-score of 87.89% in the SWaT dataset.
Using word clouds for fast identification of papers' subject domain and reviewers' competences
Venue: arXiv cs.IR 2021
Year: 2021

Generating word (tag) clouds is a powerful data visualization technique that allows people to get easily acquainted with the content of a large collection of textual documents and identify their subject domains for a matter of seconds, without reading them at all. This paper suggests applying word clouds visualization to conference management systems (specialized document management and decision support systems) in order to support and facilitate decision making in at least two important processes - forming the Programme Committee by inviting suitable reviewers and manual (re)assignment of reviewers to papers. Word clouds proved to be very useful tool for fast identification of papers' subject domain and reviewers' competences.
Analytical Models for Motifs in Temporal Networks: Discovering Trends and Anomalies
Venue: arXiv cs.SI 2021
Year: 2021

Dynamic evolving networks capture temporal relations in domains such as social networks, communication networks, and financial transaction networks. In such networks, temporal motifs, which are repeated sequences of time-stamped edges/transactions, offer valuable information about the networks' evolution and function. However, currently no analytical models for temporal graphs exist and there are no models that would allow for scalable modeling of temporal motif frequencies over time. Here, we develop the Temporal Activity State Block Model (TASBM), to model temporal motifs in temporal graphs. We develop efficient model fitting methods and derive closed-form expressions for the expected motif frequencies and their variances in a given temporal network, thus enabling the discovery of statistically significant temporal motifs. Our TASMB framework can accurately track the changes in the expected motif frequencies over time, and also scales well to networks with tens of millions of edges/transactions as it does not require time-consuming generation of many random temporal networks and then computing motif counts for each one of them. We show that TASBM is able to model changes in temporal activity over time in a network of financial transactions, a phone call, and an email network. Additionally, we show that deviations from the expected motif counts calculated by our analytical framework correspond to anomalies in the financial transactions and phone call networks.
A Deep Inference System for Differential Linear Logic
Venue: arXiv cs.LO 2021
Year: 2021

Differential linear logic (DiLL) provides a fine analysis of resource consumption in cut-elimination. We investigate the subsystem of DiLL without promotion in a deep inference formalism, where cuts are at an atomic level. In our system every provable formula admits a derivation in normal form, via a normalization procedure that commutes with the translation from sequent calculus to deep inference.
Exploring the pattern of Emotion in children with ASD as an early biomarker through Recurring-Convolution Neural Network (R-CNN)
Venue: arXiv cs.CV 2021
Year: 2021

Autism Spectrum Disorder (ASD) is found to be a major concern among various occupational therapists. The foremost challenge of this neurodevelopmental disorder lies in the fact of analyzing and exploring various symptoms of the children at their early stage of development. Such early identification could prop up the therapists and clinicians to provide proper assistive support to make the children lead an independent life. Facial expressions and emotions perceived by the children could contribute to such early intervention of autism. In this regard, the paper implements in identifying basic facial expression and exploring their emotions upon a time variant factor. The emotions are analyzed by incorporating the facial expression identified through CNN using 68 landmark points plotted on the frontal face with a prediction network formed by RNN known as RCNN-FER system. The paper adopts R-CNN to take the advantage of increased accuracy and performance with decreased time complexity in predicting emotion as a textual network analysis. The papers proves better accuracy in identifying the emotion in autistic children when compared over simple machine learning models built for such identifications contributing to autistic society.
Frequency Selection for Platoon Communications in Secondary Spectrum Using Radio Environment Maps
Venue: arXiv cs.NI 2021
Year: 2021

Platoon-based driving is an idea that vehicles follow each other at a close distance, in order to increase road throughput and fuel savings. This requires reliable wireless communications to adjust the speeds of vehicles. Although there is a dedicated frequency band for vehicle-to-vehicle (V2V) communications, studies have shown that it is too congested to provide reliable transmission for the platoons. Additional spectrum resources, i.e., secondary spectrum channels, can be utilized when these are not occupied by other users. Characteristics of interference in these channels are usually location-dependent and can be stored in the so-called Radio Environment Maps (REMs). This paper aims to design REM, in order to support the selection of secondary spectrum channel for intra-platoon communications. We propose to assess the channel's quality in terms of outage probability computed, with the use of estimated interference distributions stored in REM. A frequency selection algorithm that minimizes the number of channel switches along the planned platoon route is proposed. Additionally, the REM creation procedure is shown that reduces the number of database entries using (Density-Based Spatial Clustering of Applications with Noise) DBSCAN algorithm. The proposals are tested using real IQ samples captured on a real road. Application of the DBSCAN clustering to the constructed REM provided 7% reduction in its size. Utilization of the proposed channel selection algorithm resulted in a 35 times reduction of channel switches concerning channel assignment performed independently in every location.
ChunkFormer: Learning Long Time Series with Multi-stage Chunked Transformer
Venue: arXiv cs.LG 2021
Year: 2021

The analysis of long sequence data remains challenging in many real-world applications. We propose a novel architecture, ChunkFormer, that improves the existing Transformer framework to handle the challenges while dealing with long time series. Original Transformer-based models adopt an attention mechanism to discover global information along a sequence to leverage the contextual data. Long sequential data traps local information such as seasonality and fluctuations in short data sequences. In addition, the original Transformer consumes more resources by carrying the entire attention matrix during the training course. To overcome these challenges, ChunkFormer splits the long sequences into smaller sequence chunks for the attention calculation, progressively applying different chunk sizes in each stage. In this way, the proposed model gradually learns both local and global information without changing the total length of the input sequences. We have extensively tested the effectiveness of this new architecture on different business domains and have proved the advantage of such a model over the existing Transformer-based models.
Investigations of Smart Health Reliability
Venue: arXiv cs.HC 2021
Year: 2021

A balanced investigation into the reliability of wireless smart health devices when it comes to the collection of biometric data under varying network/environmental conditions. Followed by a program implementation to begin introductory analysis on measurement accuracy and data collection to gauge the reliability of smart health devices.
Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning
Venue: arXiv cs.AI 2021
Year: 2021

Online reinforcement learning (RL) algorithms are often difficult to deploy in complex human-facing applications as they may learn slowly and have poor early performance. To address this, we introduce a practical algorithm for incorporating human insight to speed learning. Our algorithm, Constraint Sampling Reinforcement Learning (CSRL), incorporates prior domain knowledge as constraints/restrictions on the RL policy. It takes in multiple potential policy constraints to maintain robustness to misspecification of individual constraints while leveraging helpful ones to learn quickly. Given a base RL learning algorithm (ex. UCRL, DQN, Rainbow) we propose an upper confidence with elimination scheme that leverages the relationship between the constraints, and their observed performance, to adaptively switch among them. We instantiate our algorithm with DQN-type algorithms and UCRL as base algorithms, and evaluate our algorithm in four environments, including three simulators based on real data: recommendations, educational activity sequencing, and HIV treatment sequencing. In all cases, CSRL learns a good policy faster than baselines.
Elimination (a,b)-trees with fast, durable updates
Venue: arXiv cs.DC 2021
Year: 2021

Many concurrent dictionary implementations are designed and optimized for read-mostly workloads with uniformly distributed keys, and often perform poorly on update-heavy workloads. In this work, we first present a concurrent (a,b)-tree, the OCC-ABtree, which outperforms its fastest competitor by up to 2x on uniform update-heavy workloads, and is competitive on other workloads. We then turn our attention to skewed update-heavy workloads (which feature many inserts/deletes on the same key) and introduce the Elim-ABtree, which uses a new optimization called publishing elimination. In publishing elimination, concurrent inserts and deletes to a key are reordered to eliminate them. This reduces the number of writes in the data structure. The Elim-ABtree achieves up to 2.5x the performance of its fastest competitor (including the OCC-ABtree). The OCC-ABtree and Elim-ABtree are linearizable. We also introduce durable linearizable versions (for systems with Intel Optane DCPMM non-volatile main memory) that are nearly as fast.
Echo state graph neural networks with analogue random resistor arrays
Venue: arXiv cs.ET 2021
Year: 2021

Recent years have witnessed an unprecedented surge of interest, from social networks to drug discovery, in learning representations of graph-structured data. However, graph neural networks, the machine learning models for handling graph-structured data, face significant challenges when running on conventional digital hardware, including von Neumann bottleneck incurred by physically separated memory and processing units, slowdown of Moore's law due to transistor scaling limit, and expensive training cost. Here we present a novel hardware-software co-design, the random resistor array-based echo state graph neural network, which addresses these challenges. The random resistor arrays not only harness low-cost, nanoscale and stackable resistors for highly efficient in-memory computing using simple physical laws, but also leverage the intrinsic stochasticity of dielectric breakdown to implement random projections in hardware for an echo state network that effectively minimizes the training cost thanks to its fixed and random weights. The system demonstrates state-of-the-art performance on both graph classification using the MUTAG and COLLAB datasets and node classification using the CORA dataset, achieving 34.2x, 93.2x, and 570.4x improvement of energy efficiency and 98.27%, 99.46%, and 95.12% reduction of training cost compared to conventional graph learning on digital hardware, respectively, which may pave the way for the next generation AI system for graph learning.
BP-Net: Cuff-less, Calibration-free, and Non-invasive Blood Pressure Estimation via a Generic Deep Convolutional Architecture
Venue: arXiv cs.LG 2021
Year: 2021

Objective: The paper focuses on development of robust and accurate processing solutions for continuous and cuff-less blood pressure (BP) monitoring. In this regard, a robust deep learning-based framework is proposed for computation of low latency, continuous, and calibration-free upper and lower bounds on the systolic and diastolic BP. Method: Referred to as the BP-Net, the proposed framework is a novel convolutional architecture that provides longer effective memory while achieving superior performance due to incorporation of casual dialated convolutions and residual connections. To utilize the real potential of deep learning in extraction of intrinsic features (deep features) and enhance the long-term robustness, the BP-Net uses raw Electrocardiograph (ECG) and Photoplethysmograph (PPG) signals without extraction of any form of hand-crafted features as it is common in existing solutions. Results: By capitalizing on the fact that datasets used in recent literature are not unified and properly defined, a benchmark dataset is constructed from the MIMIC-I and MIMIC-III databases obtained from PhysioNet. The proposed BP-Net is evaluated based on this benchmark dataset demonstrating promising performance and shows superior generalizable capacity. Conclusion: The proposed BP-Net architecture is more accurate than canonical recurrent networks and enhances the long-term robustness of the BP estimation task. Significance: The proposed BP-Net architecture addresses key drawbacks of existing BP estimation solutions, i.e., relying heavily on extraction of hand-crafted features, such as pulse arrival time (PAT), and; Lack of robustness. Finally, the constructed BP-Net dataset provides a unified base for evaluation and comparison of deep learning-based BP estimation algorithms.
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation
Venue: arXiv cs.CV 2021
Year: 2021

Conventional methods for the image-text generation tasks mainly tackle the naturally bidirectional generation tasks separately, focusing on designing task-specific frameworks to improve the quality and fidelity of the generated samples. Recently, Vision-Language Pre-training models have greatly improved the performance of the image-to-text generation tasks, but large-scale pre-training models for text-to-image synthesis task are still under-developed. In this paper, we propose ERNIE-ViLG, a unified generative pre-training framework for bidirectional image-text generation with transformer model. Based on the image quantization models, we formulate both image generation and text generation as autoregressive generative tasks conditioned on the text/image input. The bidirectional image-text generative modeling eases the semantic alignments across vision and language. For the text-to-image generation process, we further propose an end-to-end training method to jointly learn the visual sequence generator and the image reconstructor. To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7.9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.
Neural Hierarchical Factorization Machines for User's Event Sequence Analysis
Venue: arXiv cs.LG 2021
Year: 2021

Many prediction tasks of real-world applications need to model multi-order feature interactions in user's event sequence for better detection performance. However, existing popular solutions usually suffer two key issues: 1) only focusing on feature interactions and failing to capture the sequence influence; 2) only focusing on sequence information, but ignoring internal feature relations of each event, thus failing to extract a better event representation. In this paper, we consider a two-level structure for capturing the hierarchical information over user's event sequence: 1) learning effective feature interactions based event representation; 2) modeling the sequence representation of user's historical events. Experimental results on both industrial and public datasets clearly demonstrate that our model achieves significantly better performance compared with state-of-the-art baselines.