OpenRaiser/Intern-Atlas huggingface dataset:
Privacy Assured Recovery of Compressively Sensed ECG signals
Venue: arXiv cs.CR 2021
Year: 2021

Cloud computing for storing data and running complex algorithms have been steadily increasing. As connected IoT devices such as wearable ECG recorders generally have less storage and computational capacity, acquired signals get sent to a remote center for storage and possible analysis on demand. Recently, compressive sensing (CS) has been used as secure, energy-efficient method of signal sampling in such recorders. In this paper, we propose a secure procedure to outsource the total recovery of CS measurement to the cloud and introduce a privacy-assured signal recovery technique in the cloud. We present a fast, and lightweight encryption for secure CS recovery outsourcing that can be used in wearable devices, such as ECG Holter monitors. In the proposed technique, instead of full recovery of CS-compressed ECG signal in the cloud, to preserve privacy, an encrypted version of ECG signal is recovered by using a randomly bipolar permuted measurement matrix. The user with a key, decrypts the encrypted ECG from the cloud to obtain the original ECG signal. We demonstrate our proposed method using the ECG signals available in the MITBIH Arrhythmia Database. We also demonstrate the strength of the proposed method against partial exposure of the key.
A paper's corresponding affiliation and first affiliation are consistent at the country level in Web of Science
Venue: arXiv cs.DL 2021
Year: 2021

The purpose of this study is to explore the relationship between the first affiliation and the corresponding affiliation at the different levels via the scientometric analysis We select over 18 million papers in the core collection database of Web of Science (WoS) published from 2000 to 2015, and measure the percentage of match between the first and the corresponding affiliation at the country and institution level. We find that a paper's the first affiliation and the corresponding affiliation are highly consistent at the country level, with over 98% of the match on average. However, the match at the institution level is much lower, which varies significantly with time and country. Hence, for studies at the country level, using the first and corresponding affiliations are almost the same. But we may need to take more cautions to select affiliation when the institution is the focus of the investigation. In the meanwhile, we find some evidence that the recorded corresponding information in the WoS database has undergone some changes since 2013, which sheds light on future studies on the comparison of different databases or the affiliation accuracy of WoS. Our finding relies on the records of WoS, which may not be entirely accurate. Given the scale of the analysis, our findings can serve as a useful reference for further studies when country allocation or institute allocation is needed. Existing studies on comparisons of straight counting methods usually cover a limited number of papers, a particular research field or a limited range of time. More importantly, using the number counted can not sufficiently tell if the corresponding and first affiliation are similar. This paper uses a metric similar to Jaccard similarity to measure the percentage of the match and performs a comprehensive analysis based on a large-scale bibliometric database.
Explainable Artificial Intelligence Approaches: A Survey
Venue: arXiv cs.AI 2021
Year: 2021

The lack of explainability of a decision from an Artificial Intelligence (AI) based "black box" system/model, despite its superiority in many real-world applications, is a key stumbling block for adopting AI in many high stakes applications of different domain or industry. While many popular Explainable Artificial Intelligence (XAI) methods or approaches are available to facilitate a human-friendly explanation of the decision, each has its own merits and demerits, with a plethora of open challenges. We demonstrate popular XAI methods with a mutual case study/task (i.e., credit default prediction), analyze for competitive advantages from multiple perspectives (e.g., local, global), provide meaningful insight on quantifying explainability, and recommend paths towards responsible or human-centered AI using XAI as a medium. Practitioners can use this work as a catalog to understand, compare, and correlate competitive advantages of popular XAI methods. In addition, this survey elicits future research directions towards responsible or human-centric AI systems, which is crucial to adopt AI in high stakes applications.
Improved Training of Sparse Coding Variational Autoencoder via Weight Normalization
Venue: arXiv cs.LG 2021
Year: 2021

Learning a generative model of visual information with sparse and compositional features has been a challenge for both theoretical neuroscience and machine learning communities. Sparse coding models have achieved great success in explaining the receptive fields of mammalian primary visual cortex with sparsely activated latent representation. In this paper, we focus on a recently proposed model, sparse coding variational autoencoder (SVAE) (Barello et al., 2018), and show that the end-to-end training scheme of SVAE leads to a large group of decoding filters not fully optimized with noise-like receptive fields. We propose a few heuristics to improve the training of SVAE and show that a unit $L_2$ norm constraint on the decoder is critical to produce sparse coding filters. Such normalization can be considered as local lateral inhibition in the cortex. We verify this claim empirically on both natural image patches and MNIST dataset and show that projection of the filters onto unit norm drastically increases the number of active filters. Our results highlight the importance of weight normalization for learning sparse representation from data and suggest a new way of reducing the number of inactive latent components in VAE learning.
Deep Learning for General Game Playing with Ludii and Polygames
Venue: arXiv cs.AI 2021
Year: 2021

Combinations of Monte-Carlo tree search and Deep Neural Networks, trained through self-play, have produced state-of-the-art results for automated game-playing in many board games. The training and search algorithms are not game-specific, but every individual game that these approaches are applied to still requires domain knowledge for the implementation of the game's rules, and constructing the neural network's architecture -- in particular the shapes of its input and output tensors. Ludii is a general game system that already contains over 500 different games, which can rapidly grow thanks to its powerful and user-friendly game description language. Polygames is a framework with training and search algorithms, which has already produced superhuman players for several board games. This paper describes the implementation of a bridge between Ludii and Polygames, which enables Polygames to train and evaluate models for games that are implemented and run through Ludii. We do not require any game-specific domain knowledge anymore, and instead leverage our domain knowledge of the Ludii system and its abstract state and move representations to write functions that can automatically determine the appropriate shapes for input and output tensors for any game implemented in Ludii. We describe experimental results for short training runs in a wide variety of different board games, and discuss several open problems and avenues for future research.
HyCoR: Fault-Tolerant Replicated Containers Based on Checkpoint and Replay
Venue: arXiv cs.DC 2021
Year: 2021

HyCoR is a fully-operational fault tolerance mechanism for multiprocessor workloads, based on container replication, using a hybrid of checkpointing and replay. HyCoR derives from two insights regarding replication mechanisms: 1) deterministic replay can overcome a key disadvantage of checkpointing alone -- unacceptably long delays of outputs to clients, and 2) checkpointing can overcome a key disadvantage of active replication with deterministic replay alone -- vulnerability to even rare replay failures due to an untracked nondeterministic events. With HyCoR, the primary sends periodic checkpoints to the backup and logs the outcomes of sources of nondeterminism. Outputs to clients are delayed only by the short time it takes to send the corresponding log to the backup. Upon primary failure, the backup replays only the short interval since the last checkpoint, thus minimizing the window of vulnerability. HyCoR includes a "best effort" mechanism that results in a high recovery rate even in the presence of data races, as long as their rate is low. The evaluation includes measurement of the recovery rate and recovery latency based on fault injection. On average, HyCoR delays responses to clients by less than 1ms and recovers in less than 1s. For a set of eight real-world benchmarks, if data races are eliminated, the performance overhead of HyCoR is under 59%.
Implementing WordNet Measures of Lexical Semantic Similarity in a Fuzzy Logic Programming System
Venue: arXiv cs.PL 2021
Year: 2021

This paper introduces techniques to integrate WordNet into a Fuzzy Logic Programming system. Since WordNet relates words but does not give graded information on the relation between them, we have implemented standard similarity measures and new directives allowing the proximity equations linking two words to be generated with an approximation degree. Proximity equations are the key syntactic structures which, in addition to a weak unification algorithm, make a flexible query-answering process possible in this kind of programming language. This addition widens the scope of Fuzzy Logic Programming, allowing certain forms of lexical reasoning, and reinforcing Natural Language Processing applications. [Under consideration in Theory and Practice of Logic Programming (TPLP)]
WangchanBERTa: Pretraining transformer-based Thai Language Models
Venue: arXiv cs.CL 2021
Year: 2021

Transformer-based language models, more specifically BERT-based architectures have achieved state-of-the-art performance in many downstream tasks. However, for a relatively low-resource language such as Thai, the choices of models are limited to training a BERT-based model based on a much smaller dataset or finetuning multi-lingual models, both of which yield suboptimal downstream performance. Moreover, large-scale multi-lingual pretraining does not take into account language-specific features for Thai. To overcome these limitations, we pretrain a language model based on RoBERTa-base architecture on a large, deduplicated, cleaned training set (78GB in total size), curated from diverse domains of social media posts, news articles and other publicly available datasets. We apply text processing rules that are specific to Thai most importantly preserving spaces, which are important chunk and sentence boundaries in Thai before subword tokenization. We also experiment with word-level, syllable-level and SentencePiece tokenization with a smaller dataset to explore the effects on tokenization on downstream performance. Our model wangchanberta-base-att-spm-uncased trained on the 78.5GB dataset outperforms strong baselines (NBSVM, CRF and ULMFit) and multi-lingual models (XLMR and mBERT) on both sequence classification and token classification tasks in human-annotated, mono-lingual contexts.
Multi-Task Time Series Forecasting With Shared Attention
Venue: arXiv cs.LG 2021
Year: 2021

Time series forecasting is a key component in many industrial and business decision processes and recurrent neural network (RNN) based models have achieved impressive progress on various time series forecasting tasks. However, most of the existing methods focus on single-task forecasting problems by learning separately based on limited supervised objectives, which often suffer from insufficient training instances. As the Transformer architecture and other attention-based models have demonstrated its great capability of capturing long term dependency, we propose two self-attention based sharing schemes for multi-task time series forecasting which can train jointly across multiple tasks. We augment a sequence of paralleled Transformer encoders with an external public multi-head attention function, which is updated by all data of all tasks. Experiments on a number of real-world multi-task time series forecasting tasks show that our proposed architectures can not only outperform the state-of-the-art single-task forecasting baselines but also outperform the RNN-based multi-task forecasting method.
Pruning and Quantization for Deep Neural Network Acceleration: A Survey
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks have been applied in many applications exhibiting extraordinary abilities in the field of computer vision. However, complex network architectures challenge efficient real-time deployment and require significant computation resources and energy costs. These challenges can be overcome through optimizations such as network compression. Network compression can often be realized with little loss of accuracy. In some cases accuracy may even improve. This paper provides a survey on two types of network compression: pruning and quantization. Pruning can be categorized as static if it is performed offline or dynamic if it is performed at run-time. We compare pruning techniques and describe criteria used to remove redundant computations. We discuss trade-offs in element-wise, channel-wise, shape-wise, filter-wise, layer-wise and even network-wise pruning. Quantization reduces computations by reducing the precision of the datatype. Weights, biases, and activations may be quantized typically to 8-bit integers although lower bit width implementations are also discussed including binary neural networks. Both pruning and quantization can be used independently or combined. We compare current techniques, analyze their strengths and weaknesses, present compressed network accuracy results on a number of frameworks, and provide practical guidance for compressing networks.
Near-Optimal Detection for Both Data and Sneak-Path Interference in Resistive Memories with Random Cell Selector Failures
Venue: arXiv cs.IT 2021
Year: 2021

Resistive random-access memory is one of the most promising candidates for the next generation of non-volatile memory technology. However, its crossbar structure causes severe "sneak-path" interference, which also leads to strong inter-cell correlation. Recent works have mainly focused on sub-optimal data detection schemes by ignoring inter-cell correlation and treating sneak-path interference as independent noise. We propose a near-optimal data detection scheme that can approach the performance bound of the optimal detection scheme. Our detection scheme leverages a joint data and sneak-path interference recovery and can use all inter-cell correlations. The scheme is appropriate for data detection of large memory arrays with only linear operation complexity.
A greedy algorithm for dropping digits (Functional Pearl)
Venue: arXiv cs.PL 2021
Year: 2021

Consider the puzzle: given a number, remove $k$ digits such that the resulting number is as large as possible. Various techniques were employed to derive a linear-time solution to the puzzle: predicate logic was used to justify the structure of a greedy algorithm, a dependently-typed proof assistant was used to give a constructive proof of the greedy condition, and equational reasoning was used to calculate the greedy step as well as the final, linear-time optimisation.
Design and Analysis of Wideband In-Band-Full-Duplex FR2-IAB Networks
Venue: arXiv cs.IT 2021
Year: 2021

This paper develops a 3GPP-inspired design for the in-band-full-duplex (IBFD) integrated access and backhaul (IAB) networks in the frequency range 2 (FR2) band, which can enhance the spectral efficiency (SE) and coverage while reducing the latency. However, the self-interference (SI), which is usually more than 100 dB higher than the signal-of-interest, becomes the major bottleneck in developing these IBFD networks. We design and analyze a subarray-based hybrid beamforming IBFD-IAB system with the RF beamformers obtained via RF codebooks given by a modified Linde-Buzo-Gray (LBG) algorithm. The SI is canceled in three stages, where the first stage of antenna isolation is assumed to be successfully deployed. The second stage consists of the optical domain (OD)-based RF cancellation, where cancelers are connected with the RF chain pairs. The third stage is comprised of the digital cancellation via successive interference cancellation followed by minimum mean-squared error baseband receiver. Multiuser interference in the access link is canceled by zero-forcing at the IAB-node transmitter. Simulations show that under 400 MHz bandwidth, our proposed OD-based RF cancellation can achieve around 25 dB of cancellation with 100 taps. Moreover, the higher the hardware impairment and channel estimation error, the worse digital cancellation ability we can obtain.
The Exact Rate Memory Tradeoff for Large Caches with Coded Placement
Venue: arXiv cs.IT 2021
Year: 2021

The idea of coded caching for content distribution networks was introduced by Maddah-Ali and Niesen, who considered the canonical $(N, K)$ cache network in which a server with $N$ files satisfy the demands of $K$ users (equipped with independent caches of size $M$ each). Among other results, their work provided a characterization of the exact rate memory tradeoff for the problem when $M\geq\frac{N}{K}(K-1)$. In this paper, we improve this result for large caches with $M\geq \frac{N}{K}(K-2)$. For the case $\big\lceil\frac{K+1}{2}\big\rceil\leq N \leq K$, we propose a new coded caching scheme, and derive a matching lower bound to show that the proposed scheme is optimal. This extends the characterization of the exact rate memory tradeoff to the case $M\geq \frac{N}{K}\Big(K-2+\frac{(K-2+1/N)}{(K-1)}\Big)$. For the case $1\leq N\leq \big\lceil\frac{K+1}{2}\big\rceil$, we derive a new lower bound, which demonstrates that the scheme proposed by Yu et al. is optimal and thus extend the characterization of the exact rate memory tradeoff to the case $M\geq \frac{N}{K}(K-2)$.
Evaluating Models of Robust Word Recognition with Serial Reproduction
Venue: arXiv cs.CL 2021
Year: 2021

Spoken communication occurs in a "noisy channel" characterized by high levels of environmental noise, variability within and between speakers, and lexical and syntactic ambiguity. Given these properties of the received linguistic input, robust spoken word recognition -- and language processing more generally -- relies heavily on listeners' prior knowledge to evaluate whether candidate interpretations of that input are more or less likely. Here we compare several broad-coverage probabilistic generative language models in their ability to capture human linguistic expectations. Serial reproduction, an experimental paradigm where spoken utterances are reproduced by successive participants similar to the children's game of "Telephone," is used to elicit a sample that reflects the linguistic expectations of English-speaking adults. When we evaluate a suite of probabilistic generative language models against the yielded chains of utterances, we find that those models that make use of abstract representations of preceding linguistic context (i.e., phrase structure) best predict the changes made by people in the course of serial reproduction. A logistic regression model predicting which words in an utterance are most likely to be lost or changed in the course of spoken transmission corroborates this result. We interpret these findings in light of research highlighting the interaction of memory-based constraints and representations in language processing.
FakeFlow: Fake News Detection by Modeling the Flow of Affective Information
Venue: arXiv cs.CL 2021
Year: 2021

Fake news articles often stir the readers' attention by means of emotional appeals that arouse their feelings. Unlike in short news texts, authors of longer articles can exploit such affective factors to manipulate readers by adding exaggerations or fabricating events, in order to affect the readers' emotions. To capture this, we propose in this paper to model the flow of affective information in fake news articles using a neural architecture. The proposed model, FakeFlow, learns this flow by combining topic and affective information extracted from text. We evaluate the model's performance with several experiments on four real-world datasets. The results show that FakeFlow achieves superior results when compared against state-of-the-art methods, thus confirming the importance of capturing the flow of the affective information in news articles.
A Survey of Novel Cache Hierarchy Designs for High Workloads
Venue: arXiv cs.AR 2021
Year: 2021

Traditional on-die, three-level cache hierarchy design is very commonly used but is also prone to latency, especially at the Level 2 (L2) cache. We discuss three distinct ways of improving this design in order to have better performance. Performance is especially important for systems with high workloads. The first method proposes to eliminate L2 altogether while proposing a new prefetching technique, the second method suggests increasing the size of L2, while the last method advocates the implementation of optical caches. After carefully contemplating the results in performance gains and the advantages and disadvantages of each method, we found the last method to be the best of the three.
Privacy Preserving Techniques Applied to CPNI Data: Analysis and Recommendations
Venue: arXiv cs.CR 2021
Year: 2021

With mobile phone penetration rates reaching 90%, Consumer Proprietary Network Information (CPNI) can offer extremely valuable information to different sectors, including policymakers. Indeed, as part of CPNI, Call Detail Records have been successfully used to provide real-time traffic information, to improve our understanding of the dynamics of people's mobility and so to allow prevention and measures in fighting infectious diseases, and to offer population statistics. While there is no doubt of the usefulness of CPNI data, privacy concerns regarding sharing individuals' data have prevented it from being used to its full potential. Traditional de-anonymization measures, such as pseudonymization and standard de-identification, have been shown to be insufficient to protect privacy. This has been specifically shown on mobile phone datasets. As an example, researchers have shown that with only four data points of approximate place and time information of a user, 95% of users could be re-identified in a dataset of 1.5 million mobile phone users. In this landscape paper, we will discuss the state-of-the-art anonymization techniques and their shortcomings.
GP: Context-free Grammar Pre-training for Text-to-SQL Parsers
Venue: arXiv cs.CL 2021
Year: 2021

A new method for Text-to-SQL parsing, Grammar Pre-training (GP), is proposed to decode deep relations between question and database. Firstly, to better utilize the information of databases, a random value is added behind a question word which is recognized as a column, and the new sentence serves as the model input. Secondly, initialization of vectors for decoder part is optimized, with reference to the former encoding so that question information can be concerned. Finally, a new approach called flooding level is adopted to get the non-zero training loss which can generalize better results. By encoding the sentence with GRAPPA and RAT-SQL model, we achieve better performance on spider, a cross-DB Text-to-SQL dataset (72.8 dev, 69.8 test). Experiments show that our method is easier to converge during training and has excellent robustness.