OpenRaiser/Intern-Atlas huggingface dataset:
A Multilingual Modeling Method for Span-Extraction Reading Comprehension
Venue: arXiv cs.CL 2021
Year: 2021

Span-extraction reading comprehension models have made tremendous advances enabled by the availability of large-scale, high-quality training datasets. Despite such rapid progress and widespread application, extractive reading comprehension datasets in languages other than English remain scarce, and creating such a sufficient amount of training data for each language is costly and even impossible. An alternative to creating large-scale high-quality monolingual span-extraction training datasets is to develop multilingual modeling approaches and systems which can transfer to the target language without requiring training data in that language. In this paper, in order to solve the scarce availability of extractive reading comprehension training data in the target language, we propose a multilingual extractive reading comprehension approach called XLRC by simultaneously modeling the existing extractive reading comprehension training data in a multilingual environment using self-adaptive attention and multilingual attention. Specifically, we firstly construct multilingual parallel corpora by translating the existing extractive reading comprehension datasets (i.e., CMRC 2018) from the target language (i.e., Chinese) into different language families (i.e., English). Secondly, to enhance the final target representation, we adopt self-adaptive attention (SAA) to combine self-attention and inter-attention to extract the semantic relations from each pair of the target and source languages. Furthermore, we propose multilingual attention (MLA) to learn the rich knowledge from various language families. Experimental results show that our model outperforms the state-of-the-art baseline (i.e., RoBERTa_Large) on the CMRC 2018 task, which demonstrate the effectiveness of our proposed multi-lingual modeling approach and show the potentials in multilingual NLP tasks.
Lower Bounds for the Number of Repetitions in 2D Strings
Venue: arXiv cs.FL 2021
Year: 2021

A two-dimensional string is simply a two-dimensional array. We continue the study of the combinatorial properties of repetitions in such strings over the binary alphabet, namely the number of distinct tandems, distinct quartics, and runs. First, we construct an infinite family of $n\times n$ 2D strings with $Ω(n^{3})$ distinct tandems. Second, we construct an infinite family of $n\times n$ 2D strings with $Ω(n^{2}\log n)$ distinct quartics. Third, we construct an infinite family of $n\times n$ 2D strings with $Ω(n^{2}\log n)$ runs. This resolves an open question of Charalampopoulos, Radoszewski, Rytter, Waleń, and Zuba [ESA 2020], who asked if the number of distinct quartics and runs in an $n\times n$ 2D string is $\mathcal{O}(n^{2})$.
Do Multilingual Neural Machine Translation Models Contain Language Pair Specific Attention Heads?
Venue: arXiv cs.CL 2021
Year: 2021

Recent studies on the analysis of the multilingual representations focus on identifying whether there is an emergence of language-independent representations, or whether a multilingual model partitions its weights among different languages. While most of such work has been conducted in a "black-box" manner, this paper aims to analyze individual components of a multilingual neural translation (NMT) model. In particular, we look at the encoder self-attention and encoder-decoder attention heads (in a many-to-one NMT model) that are more specific to the translation of a certain language pair than others by (1) employing metrics that quantify some aspects of the attention weights such as "variance" or "confidence", and (2) systematically ranking the importance of attention heads with respect to translation quality. Experimental results show that surprisingly, the set of most important attention heads are very similar across the language pairs and that it is possible to remove nearly one-third of the less important heads without hurting the translation quality greatly.
VidFace: A Full-Transformer Solver for Video FaceHallucination with Unaligned Tiny Snapshots
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we investigate the task of hallucinating an authentic high-resolution (HR) human face from multiple low-resolution (LR) video snapshots. We propose a pure transformer-based model, dubbed VidFace, to fully exploit the full-range spatio-temporal information and facial structure cues among multiple thumbnails. Specifically, VidFace handles multiple snapshots all at once and harnesses the spatial and temporal information integrally to explore face alignments across all the frames, thus avoiding accumulating alignment errors. Moreover, we design a recurrent position embedding module to equip our transformer with facial priors, which not only effectively regularises the alignment mechanism but also supplants notorious pre-training. Finally, we curate a new large-scale video face hallucination dataset from the public Voxceleb2 benchmark, which challenges prior arts on tackling unaligned and tiny face snapshots. To the best of our knowledge, we are the first attempt to develop a unified transformer-based solver tailored for video-based face hallucination. Extensive experiments on public video face benchmarks show that the proposed method significantly outperforms the state of the arts.
A new distance based on minimal absent words and applications to biological sequences
Venue: arXiv cs.FL 2021
Year: 2021

A minimal absent word of a sequence x, is a sequence yt hat is not a factorof x, but all of its proper factors are factors of x as well. The set of minimal absent words uniquely defines the sequence itself. In recent times minimal absent words have been used in order to compare sequences. In fact, to do this, one can compare the sets of their minimal absent words. Chairungasee and Crochemorein [2] define a distance between pairs of sequences x and y, where the symmetric difference of the sets of minimal absent words of x and y is involved. Here, weconsider a different distance, introduced in [1], based on a specific subset of such symmetric difference that, in our opinion, better capture the different features ofthe considered sequences. We show the result of some experiments where the distance is tested on a dataset of genetic sequences by 11 living species, in order to compare the new distance with the ones existing in literature.
ArChes -- Automatic generation of component fault trees from continuous function charts
Venue: arXiv cs.SE 2021
Year: 2021

The growing size and complexity of software in embedded systems poses new challenges to the safety assessment of embedded control systems. In industrial practice, the control software is mostly treated as a black box during the system's safety analysis. The appropriate representation of the failure propagation of the software is a pressing need in order to increase the accuracy of safety analyses. However, it also increase the effort for creating and maintaining the safety analysis models (such as fault trees) significantly. In this work, we present a method to automatically generate Component Fault Trees from Continuous Function Charts. This method aims at generating the failure propagation model of the detailed software specification. Hence, control software can be included into safety analyses without additional manual effort required to construct the safety analysis models of the software. Moreover, safety analyses created during early system specification phases can be verified by comparing it with the automatically generated one in the detailed specification phased.
Locally Private $k$-Means Clustering with Constant Multiplicative Approximation and Near-Optimal Additive Error
Venue: arXiv cs.DS 2021
Year: 2021

Given a data set of size $n$ in $d'$-dimensional Euclidean space, the $k$-means problem asks for a set of $k$ points (called centers) so that the sum of the $\ell_2^2$-distances between points of a given data set of size $n$ and the set of $k$ centers is minimized. Recent work on this problem in the locally private setting achieves constant multiplicative approximation with additive error $\tilde{O} (n^{1/2 + a} \cdot k \cdot \max \{\sqrt{d}, \sqrt{k} \})$ and proves a lower bound of $Ω(\sqrt{n})$ on the additive error for any solution with a constant number of rounds. In this work we bridge the gap between the exponents of $n$ in the upper and lower bounds on the additive error with two new algorithms. Given any $α>0$, our first algorithm achieves a multiplicative approximation guarantee which is at most a $(1+α)$ factor greater than that of any non-private $k$-means clustering algorithm with $k^{\tilde{O}(1/α^2)} \sqrt{d' n} \mbox{poly}\log n$ additive error. Given any $c>\sqrt{2}$, our second algorithm achieves $O(k^{1 + \tilde{O}(1/(2c^2-1))} \sqrt{d' n} \mbox{poly} \log n)$ additive error with constant multiplicative approximation. Both algorithms go beyond the $Ω(n^{1/2 + a})$ factor that occurs in the additive error for arbitrarily small parameters $a$ in previous work, and the second algorithm in particular shows for the first time that it is possible to solve the locally private $k$-means problem in a constant number of rounds with constant factor multiplicative approximation and polynomial dependence on $k$ in the additive error arbitrarily close to linear.
Digital Contact Tracing for Covid 19
Venue: arXiv cs.HC 2021
Year: 2021

The COVID19 pandemic created a worldwide emergency as it is estimated that such a large number of infections are due to human-to-human transmission of the COVID19. As a necessity, there is a need to track users who came in contact with users having travel history, asymptomatic and not yet symptomatic, but they can be in the future. To solve this problem, the present work proposes a solution for contact tracing based on assisted GPS and cloud computing technologies. An application is developed to collect each user's assisted GPS coordinates once all the users install this application. This application periodically sends assisted GPS data to the cloud. To determine which devices are within the permissible limit of 5m, we perform clustering over assisted GPS coordinates and track the clusters for about t mins to allow the measure of spread. We assume that it takes around 3 or 5 mins to get the virus from an infected object. For clustering, the proposed M way like tree data structure stores the assisted GPS coordinates in degree, minute, and second format. Thus, every user is mapped to a leaf node of the tree. We split the "seconds" part of the assisted GPS location into m equal parts, which amount to d meter in latitude(longitude). Hence, two users who are within d meter range will map to the same leaf node. Thus, by mapping assisted GPS locations every t mins, we can find out how many users came in contact with a particular user for at least t mins. Our work's salient feature is that it runs in linear time O(n) for n users in the static case, i.e., when users are not moving. We also propose a variant of our solution to handle the dynamic case, that is, when users are moving. Besides, the proposed solution offers potential hotspot detection and safe-route recommendation as an additional feature, and proof of concept is presented through experiments on simulated data of 10M users.
Truthful Mechanisms for Two-Sided Markets via Prophet Inequalities
Venue: arXiv cs.GT 2021
Year: 2021

We design novel mechanisms for welfare-maximization in two-sided markets. That is, there are buyers willing to purchase items and sellers holding items initially, both acting rationally and strategically in order to maximize utility. Our mechanisms are designed based on a powerful correspondence between two-sided markets and prophet inequalities. They satisfy individual rationality, dominant-strategy incentive compatibility, budget-balance constraints and give constant-factor approximations to the optimal social welfare. We improve previous results in several settings: Our main focus is on matroid double auctions, where the set of buyers who obtain an item needs to be independent in a matroid. We construct two mechanisms, the first being a $1/3$-approximation of the optimal social welfare satisfying strong budget-balance and requiring the agents to trade in a customized order, the second being a $1/2$-approximation, weakly budget-balanced and able to deal with online arrival determined by an adversary. In addition, we construct constant-factor approximations in two-sided markets when buyers need to fulfill a knapsack constraint. Also, in combinatorial double auctions, where buyers have valuation functions over item bundles instead of being interested in only one item, using similar techniques, we design a mechanism which is a $1/2$-approximation of the optimal social welfare, strongly budget-balanced and can deal with online arrival of agents in an adversarial order.
Telling Stories through Multi-User Dialogue by Modeling Character Relations
Venue: arXiv cs.CL 2021
Year: 2021

This paper explores character-driven story continuation, in which the story emerges through characters' first- and second-person narration as well as dialogue -- requiring models to select language that is consistent with a character's persona and their relationships with other characters while following and advancing the story. We hypothesize that a multi-task model that trains on character dialogue plus character relationship information improves transformer-based story continuation. To this end, we extend the Critical Role Dungeons and Dragons Dataset (Rameshkumar and Bailey, 2020) -- consisting of dialogue transcripts of people collaboratively telling a story while playing the role-playing game Dungeons and Dragons -- with automatically extracted relationships between each pair of interacting characters as well as their personas. A series of ablations lend evidence to our hypothesis, showing that our multi-task model using character relationships improves story continuation accuracy over strong baselines.
Using Pareto Simulated Annealing to Address Algorithmic Bias in Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

Algorithmic Bias can be due to bias in the training data or issues with the algorithm itself. These algorithmic issues typically relate to problems with model capacity and regularisation. This underestimation bias may arise because the model has been optimised for good generalisation accuracy without any explicit consideration of bias or fairness. In a sense, we should not be surprised that a model might be biased when it hasn't been "asked" not to be. In this paper, we consider including bias (underestimation) as an additional criterion in model training. We present a multi-objective optimisation strategy using Pareto Simulated Annealing that optimise for both balanced accuracy and underestimation. We demonstrate the effectiveness of this strategy on one synthetic and two real-world datasets.
Detecting Fetal Alcohol Spectrum Disorder in children using Artificial Neural Network
Venue: arXiv cs.LG 2021
Year: 2021

Fetal alcohol spectrum disorder (FASD) is a syndrome whose only difference compared to other children's conditions is the mother's alcohol consumption during pregnancy. An earlier diagnosis of FASD improving the quality of life of children and adolescents. For this reason, this study focus on evaluating the use of the artificial neural network (ANN) to classify children with FASD and explore how accurate it is. ANN has been used to diagnose cancer, diabetes, and other diseases in the medical area, being a tool that presents good results. The data used is from a battery of tests from children for 5-18 years old (include tests of psychometric, saccade eye movement, and diffusion tensor imaging (DTI)). We study the different configurations of ANN with dense layers. The first one predicts 75\% of the outcome correctly for psychometric data. The others models include a feature layer, and we used it to predict FASD using every test individually. The models accurately predict over 70\% of the cases, and psychometric and memory guides predict over 88\% accuracy. The results suggest that the ANN approach is a competitive and efficient methodology to detect FASD. However, we could be careful in used as a diagnostic technique.
Systematic investigation into generalization of COVID-19 CT deep learning models with Gabor ensemble for lung involvement scoring
Venue: arXiv cs.CV 2021
Year: 2021

The COVID-19 pandemic has inspired unprecedented data collection and computer vision modelling efforts worldwide, focusing on diagnosis and stratification of COVID-19 from medical images. Despite this large-scale research effort, these models have found limited practical application due in part to unproven generalization of these models beyond their source study. This study investigates the generalizability of key published models using the publicly available COVID-19 Computed Tomography data through cross dataset validation. We then assess the predictive ability of these models for COVID-19 severity using an independent new dataset that is stratified for COVID-19 lung involvement. Each inter-dataset study is performed using histogram equalization, and contrast limited adaptive histogram equalization with and without a learning Gabor filter. The study shows high variability in the generalization of models trained on these datasets due to varied sample image provenances and acquisition processes amongst other factors. We show that under certain conditions, an internally consistent dataset can generalize well to an external dataset despite structural differences between these datasets with f1 scores up to 86%. Our best performing model shows high predictive accuracy for lung involvement score for an independent dataset for which expertly labelled lung involvement stratification is available. Creating an ensemble of our best model for disease positive prediction with our best model for disease negative prediction using a min-max function resulted in a superior model for lung involvement prediction with average predictive accuracy of 75% for zero lung involvement and 96% for 75-100% lung involvement with almost linear relationship between these stratifications.
Small-Scale Spatial-Temporal Correlation and Degrees of Freedom for Reconfigurable Intelligent Surfaces
Venue: arXiv cs.NI 2021
Year: 2021

The reconfigurable intelligent surface (RIS) is an emerging promising candidate technology for future wireless networks, where the element spacing is usually of sub-wavelength. Only limited knowledge, however, has been gained about the spatial-temporal correlation behavior among the elements in an RIS. In this paper, we investigate the spatial-temporal correlation for an RIS-enabled wireless communication system. Specifically, a joint small-scale spatial-temporal correlation model is derived under isotropic scattering, which can be represented by a four-dimensional sinc function. Furthermore, based upon the spatial-only correlation at a certain time instant, an essential RIS property -- the spatial degrees of freedom (DoF) -- is revisited, and an analytical expression is propounded to characterize the spatial DoF for RISs with realistic hence non-infinitesimal element spacing and finite aperture sizes. The results are vital to the accurate evaluation of various system performance metrics.
Zero-bias Deep Learning Enabled Quick and Reliable Abnormality Detection in IoT
Venue: arXiv cs.NI 2021
Year: 2021

Abnormality detection is essential to the performance of safety-critical and latency-constrained systems. However, as systems are becoming increasingly complicated with a large quantity of heterogeneous data, conventional statistical change point detection methods are becoming less effective and efficient. Although Deep Learning (DL) and Deep Neural Networks (DNNs) are increasingly employed to handle heterogeneous data, they still lack theoretic assurable performance and explainability. This paper integrates zero-bias DNN and Quickest Event Detection algorithms to provide a holistic framework for quick and reliable detection of both abnormalities and time-dependent abnormal events in the Internet of Things (IoT). We first use the zero-bias dense layer to increase the explainability of DNN. We provide a solution to convert zero-bias DNN classifiers into performance assured binary abnormality detectors. Using the converted abnormality detector, we then present a sequential quickest detection scheme that provides the theoretically assured lowest abnormal event detection delay under false alarm constraints. Finally, we demonstrate the effectiveness of the framework using both massive signal records from real-world aviation communication systems and simulated data. Code and data of our work is available at \url{https://github.com/pcwhy/AbnormalityDetectionInZbDNN}
Skin-Health Monitoring system using a Wireless Body Area Network
Venue: arXiv cs.NI 2021
Year: 2021

A new class of sensing paradigm known as lab-onskin where stretchable and flexible smart sensor devices are integrated into the skin, provides direct monitoring and diagnostic interfaces to the body. Distributed lab-on-skin wireless sensors have the ability to provide continuous long term assessment of the skin health. This paper proposes a distributed skin health monitoring system using a wireless body area network. The system is responsive to the dynamic changes in the skin health, and remotely reports on the same. The proposed algorithm detects the abnormal skin and creates an energy efficient data aggregation tree covering the affected area while putting the unnecessary sensors to sleep mode. The algorithm responds to the changing conditions of the skin by dynamically adapting the size and shape of the monitoring trees to that of the abnormal skin areas thus providing a comprehensive monitoring. Simulation results demonstrate the application and utility of the proposed algorithm for changing wound shapes and sizes.
Anchor Nodes Positioning for Self-localization in Wireless Sensor Networks using Belief Propagation and Evolutionary Algorithms
Venue: arXiv cs.NI 2021
Year: 2021

Locating each node in a wireless sensor network is essential for starting the monitoring job and sending information about the area. One method that has been used in hard and inaccessible environments is randomly scattering each node in the area. In order to reduce the cost of using GPS at each node, some nodes should be equipped with GPS (anchors), Then using the belief propagation algorithm, locate other nodes. The number of anchor nodes must be reduced since they are expensive. Furthermore, the location of these nodes affects the algorithm's performance. Using multi-objective optimization, an algorithm is introduced in this paper that minimizes the estimated location error and the number of anchor nodes. According to simulation results, This algorithm proposes a set of solutions with less energy consumption and less error than similar algorithms.
Applications of Artificial Intelligence, Machine Learning and related techniques for Computer Networking Systems
Venue: arXiv cs.NI 2021
Year: 2021

This article presents a primer/overview of applications of Artificial Intelligence and Machine Learning (AI/ML) techniques to address problems in the domain of computer networking. In particular, the techniques have been used to support efficient and accurate traffic prediction, traffic classification, anomaly detection, network management, network security, network resource allocation and optimization, network scheduling algorithms, fault diagnosis and many more such applications. The article first summarizes some of the key networking concepts and a few representative machine learning techniques and algorithms. The article then presents details regarding the availability of data sets for networking applications and machine learning software and toolkits for processing these data sets. Highlights of some of the standards activities, pursued by ITU-T and ETSI, which are related to AI/ML for networking, are also presented. Finally, the article discusses a small set of representative networking problems where AI/ML techniques have been successfully applied.