OpenRaiser/Intern-Atlas huggingface dataset:
DataLoc+: A Data Augmentation Technique for Machine Learning in Room-Level Indoor Localization
Venue: arXiv cs.CV 2021
Year: 2021

Indoor localization has been a hot area of research over the past two decades. Since its advent, it has been steadily utilizing the emerging technologies to improve accuracy, and machine learning has been at the heart of that. Machine learning has been increasingly used in fingerprint-based indoor localization to replace or emulate the radio map that is used to predict locations given a location signature. The prediction quality of a machine learning model primarily depends on how well the model was trained, which relies on the amount and quality of data used to train it. Data augmentation has been used to improve quality of the trained models by synthetically producing more training data, and several approaches were used in the literature that tackles the problem of lack of training data from different angles. In this paper, we propose DataLoc+, a data augmentation technique for room-level indoor localization that combines different approaches in a simple algorithm. We evaluate the technique by comparing it to the typical direct snapshot approach using data collected from a field experiment conducted in a hospital. Our evaluation shows that the model trained using the proposed technique achieves higher accuracy. We also show that the technique adapts to larger problems using a limited dataset while maintaining high accuracy.
Separating Adaptive Streaming from Oblivious Streaming
Venue: arXiv cs.DS 2021
Year: 2021

We present a streaming problem for which every adversarially-robust streaming algorithm must use polynomial space, while there exists a classical (oblivious) streaming algorithm that uses only polylogarithmic space. This is the first separation between oblivious streaming and adversarially-robust streaming, and resolves one of the central open questions in adversarial robust streaming.
An In-depth Review of Privacy Concerns Raised by the COVID-19 Pandemic
Venue: arXiv cs.CR 2021
Year: 2021

COVID-19 has hugely changed our lives, work, and interactions with people. With more and more online activities, people are easily exposed to privacy threats. In this paper, we explore how users self-disclose on social media and privacy concerns raised from these behaviors. Based on recent news, techniques, and research, we indicate three increasing privacy threats caused by the COVID-19 pandemic. After that, we provide a systematic analysis of potential privacy issues related to the COVID pandemic. Furthermore, we propose a series of research directions about online user self-disclosure and privacy issues for future work as well as possible solutions.
On statistical inference when fixed points of belief propagation are unstable
Venue: arXiv cs.DS 2021
Year: 2021

Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a planted constraint satisfaction problem such as planted 3-SAT, the clauses are sparse observations from which the hidden assignment is to be recovered. In the problem of community detection in a stochastic block model, the community labels are hidden variables that are to be recovered from the edges of the graph. Inspired by ideas from statistical physics, the presence of a stable fixed point for belief propogation has been widely conjectured to characterize the computational tractability of these problems. For community detection in stochastic block models, many of these predictions have been rigorously confirmed. In this work, we consider a general model of statistical inference problems that includes both community detection in stochastic block models, and all planted constraint satisfaction problems as special cases. We carry out the cavity method calculations from statistical physics to compute the regime of parameters where detection and recovery should be algorithmically tractable. At precisely the predicted tractable regime, we give: (i) a general polynomial-time algorithm for the problem of detection: distinguishing an input with a planted signal from one without; (ii) a general polynomial-time algorithm for the problem of recovery: outputting a vector that correlates with the hidden assignment significantly better than a random guess would.
HexCNN: A Framework for Native Hexagonal Convolutional Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Hexagonal CNN models have shown superior performance in applications such as IACT data analysis and aerial scene classification due to their better rotation symmetry and reduced anisotropy. In order to realize hexagonal processing, existing studies mainly use the ZeroOut method to imitate hexagonal processing, which causes substantial memory and computation overheads. We address this deficiency with a novel native hexagonal CNN framework named HexCNN. HexCNN takes hexagon-shaped input and performs forward and backward propagation on the original form of the input based on hexagon-shaped filters, hence avoiding computation and memory overheads caused by imitation. For applications with rectangle-shaped input but require hexagonal processing, HexCNN can be applied by padding the input into hexagon-shape as preprocessing. In this case, we show that the time and space efficiency of HexCNN still outperforms existing hexagonal CNN methods substantially. Experimental results show that compared with the state-of-the-art models, which imitate hexagonal processing but using rectangle-shaped filters, HexCNN reduces the training time by up to 42.2%. Meanwhile, HexCNN saves the memory space cost by up to 25% and 41.7% for loading the input and performing convolution, respectively.
Attention Can Reflect Syntactic Structure (If You Let It)
Venue: arXiv cs.CL 2021
Year: 2021

Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism. However, much of such work focused almost exclusively on English -- a language with rigid word order and a lack of inflectional morphology. In this study, we present decoding experiments for multilingual BERT across 18 languages in order to test the generalizability of the claim that dependency syntax is reflected in attention patterns. We show that full trees can be decoded above baseline accuracy from single attention heads, and that individual relations are often tracked by the same heads across languages. Furthermore, in an attempt to address recent debates about the status of attention as an explanatory mechanism, we experiment with fine-tuning mBERT on a supervised parsing objective while freezing different series of parameters. Interestingly, in steering the objective to learn explicit linguistic structure, we find much of the same structure represented in the resulting attention patterns, with interesting differences with respect to which parameters are frozen.
Unsupervised clustering of series using dynamic programming and neural processes
Venue: arXiv cs.LG 2021
Year: 2021

Following the work of arXiv:2101.09512, we are interested in clustering a given multi-variate series in an unsupervised manner. We would like to segment and cluster the series such that the resulting blocks present in each cluster are coherent with respect to a predefined model structure (e.g. a physics model with a functional form defined by a number of parameters). However, such approach might have its limitation, partly because there may exist multiple models that describe the same data, and partly because the exact model behind the data may not immediately known. Hence, it is useful to establish a general framework that enables the integration of plausible models and also accommodates data-driven approach into one approximated model to assist the clustering task. Hence, in this work, we investigate the use of neural processes to build the approximated model while yielding the same assumptions required by the algorithm presented in arXiv:2101.09512.
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
Venue: arXiv cs.CL 2021
Year: 2021

Multilingual pretrained language models have demonstrated remarkable zero-shot cross-lingual transfer capabilities. Such transfer emerges by fine-tuning on a task of interest in one language and evaluating on a distinct language, not seen during the fine-tuning. Despite promising results, we still lack a proper understanding of the source of this transfer. Using a novel layer ablation technique and analyses of the model's internal representations, we show that multilingual BERT, a popular multilingual language model, can be viewed as the stacking of two sub-networks: a multilingual encoder followed by a task-specific language-agnostic predictor. While the encoder is crucial for cross-lingual transfer and remains mostly unchanged during fine-tuning, the task predictor has little importance on the transfer and can be reinitialized during fine-tuning. We present extensive experiments with three distinct tasks, seventeen typologically diverse languages and multiple domains to support our hypothesis.
Cross-Lingual Named Entity Recognition Using Parallel Corpus: A New Approach Using XLM-RoBERTa Alignment
Venue: arXiv cs.CL 2021
Year: 2021

We propose a novel approach for cross-lingual Named Entity Recognition (NER) zero-shot transfer using parallel corpora. We built an entity alignment model on top of XLM-RoBERTa to project the entities detected on the English part of the parallel data to the target language sentences, whose accuracy surpasses all previous unsupervised models. With the alignment model we can get pseudo-labeled NER data set in the target language to train task-specific model. Unlike using translation methods, this approach benefits from natural fluency and nuances in target-language original corpus. We also propose a modified loss function similar to focal loss but assigns weights in the opposite direction to further improve the model training on noisy pseudo-labeled data set. We evaluated this proposed approach over 4 target languages on benchmark data sets and got competitive F1 scores compared to most recent SOTA models. We also gave extra discussions about the impact of parallel corpus size and domain on the final transfer performance.
Real-time oblivious erasure correction with linear time decoding and constant feedback
Venue: arXiv cs.IT 2021
Year: 2021

We continue the study of rateless codes for transmission of information across channels whose rate of erasure is unknown. In such a code, an infinite stream of encoding symbols can be generated from the message and sent across the erasure channel, and the decoder can decode the message after it has successfully collected a certain number of encoding symbols. A rateless erasure code is real-time oblivious if rather than collecting encoding symbols as they are received, the receiver either immediately decodes or discards each symbol it receives. Efficient real-time oblivious erasure correction uses a feedback channel in order to maximize the probability that a received encoding symbol is decoded rather than discarded. We construct codes which are real-time oblivious, but require fewer feedback messages and have faster decoding compared to previous work. Specifically, for a message of length $k'$, we improve the expected complexity of the feedback channel from $O(\sqrt{k'})$ to $O(1)$, and the expected decoding complexity from $O(k'\log(k'))$ to $O(k')$. Our method involves using an appropriate block erasure code to first encode the $k'$ message symbols, and then using a truncated version of the real-time oblivious erasure correction of Beimel et al (2007) to transmit the encoded message to the receiver, which then uses the decoding algorithm for the outer code to recover the message.
Cloud based VANET Simulator (CVANETSIM)
Venue: arXiv cs.NI 2021
Year: 2021

Vehicular ad hoc network (VANET) is an integral part of vehicular communication. VANET suffers many problems such as scalability. To solve scalability and other problems of VANET, clustering is proposed. VANET clustering is different than any other kind of clustering due to the high mobility of the vehicles. Likewise, VANET and VANET clustering, VANET simulator requires some unique features such as internet based real-time data processing, huge data analysis, the complex calculation to maintain hierarchy among the vehicles, etc.; however, neither web based VANET simulator nor clustering module available in the existing simulators. Therefore, a simulator that will be able to simulate any feature of VANET equipped with a clustering module and accessible via the internet is a growing need in vehicular communication research. At the Telecom and Network Research Lab (TNRL), University of Oklahoma, we have developed a fully functional discrete-event VANET simulator that includes all the features of VANET clustering. Moreover, the cloud based VANET simulator (CVANETSIM) is coming with an easy and interactive web interface. To our best of our knowledge, CVANETSIM is the first of its kind which integrates features of the VANET simulator, built-in VANET clustering module, and accessible through the internet.
On the Importance of Capturing a Sufficient Diversity of Perspective for the Classification of micro-PCBs
Venue: arXiv cs.CV 2021
Year: 2021

We present a dataset consisting of high-resolution images of 13 micro-PCBs captured in various rotations and perspectives relative to the camera, with each sample labeled for PCB type, rotation category, and perspective categories. We then present the design and results of experimentation on combinations of rotations and perspectives used during training and the resulting impact on test accuracy. We then show when and how well data augmentation techniques are capable of simulating rotations vs. perspectives not present in the training data. We perform all experiments using CNNs with and without homogeneous vector capsules (HVCs) and investigate and show the capsules' ability to better encode the equivariance of the sub-components of the micro-PCBs. The results of our experiments lead us to conclude that training a neural network equipped with HVCs, capable of modeling equivariance among sub-components, coupled with training on a diversity of perspectives, achieves the greatest classification accuracy on micro-PCB data.
LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction
Venue: arXiv cs.CL 2021
Year: 2021

Open Information Extraction (OIE) systems seek to compress the factual propositions of a sentence into a series of n-ary tuples. These tuples are useful for downstream tasks in natural language processing like knowledge base creation, textual entailment, and natural language understanding. However, current OIE datasets are limited in both size and diversity. We introduce a new dataset by converting the QA-SRL 2.0 dataset to a large-scale OIE dataset (LSOIE). Our LSOIE dataset is 20 times larger than the next largest human-annotated OIE dataset. We construct and evaluate several benchmark OIE models on LSOIE, providing baselines for future improvements on the task. Our LSOIE data, models, and code are made publicly available
Bayes-Optimal Convolutional AMP
Venue: arXiv cs.IT 2021
Year: 2021

To improve the convergence property of approximate message-passing (AMP), convolutional AMP (CAMP) has been proposed. CAMP replaces the Onsager correction in AMP with a convolution of messages in all preceding iterations while it uses the same low-complexity matched filter (MF) as AMP. This paper derives state evolution (SE) equations to design the Bayes-optimal denoiser in CAMP. Numerical results imply that CAMP with the Bayes-optimal denoiser--called Bayes-optimal CAMP--can achieve the Bayes-optimal performance for right-orthogonally invariant sensing matrices with low-to-moderate condition numbers.
Harvest: A Reliable and Energy Efficient Bulk Data Collection Service for Large Scale Wireless Sensor Networks
Venue: arXiv cs.NI 2021
Year: 2021

We present a bulk data collection service, Harvest, for energy constrained wireless sensor nodes. To increase spatial reuse and thereby decrease latency, Harvest performs concurrent, pipelined exfiltration from multiple nodes to a base station. To this end, it uses a distance-k coloring of the nodes, notably with a constant number of colors, which yields a TDMA schedule whereby nodes can communicate concurrently with low packet losses due to collision. This coloring is based on a randomized CSMA approach which does not exploit location knowledge. Given a bounded degree of the network, each node waits only O$(1)$ time to obtain a unique color among its distance-k neighbors, in contrast to the traditional deterministic distributed distance-k vertex coloring wherein each node waits O$(Δ^{2})$ time to obtain a color. Harvest offers the option of limiting memory use to only a small constant number of bytes or of improving latency with increased memory use; it can be used with or without additional mechanisms for reliability of message forwarding. We experimentally evaluate the performance of Harvest using 51 motes in the Kansei testbed. We also provide theoretical as well as TOSSIM-based comparison of Harvest with Straw, an extant data collection service implemented for TinyOS platforms that use one-node at a time exfiltration. For networks with more than 3-hops, Harvest reduces the latency by at least 33% as compared to that of Straw.
What We Can Learn From Visual Artists About Software Development
Venue: arXiv cs.HC 2021
Year: 2021

This paper explores software's role in visual art production by examining how artists use and develop software. We conducted interviews with professional artists who were collaborating with software developers, learning software development, and building and maintaining software. We found artists were motivated to learn software development for intellectual growth and access to technical communities. Artists valued efficient workflows through skilled manual execution and personal software development, but avoided high-level forms of software automation. Artists identified conflicts between their priorities and those of professional developers and computational art communities, which influenced how they used computational aesthetics in their work. These findings contribute to efforts in systems engineering research to integrate end-user programming and creativity support across software and physical media, suggesting opportunities for artists as collaborators. Artists' experiences writing software can guide technical implementations of domain-specific representations, and their experiences in interdisciplinary production can aid inclusive community building around computational tools.
A Survey on 5G Radio Access Network Energy Efficiency: Massive MIMO, Lean Carrier Design, Sleep Modes, and Machine Learning
Venue: arXiv cs.NI 2021
Year: 2021

Cellular networks have changed the world we are living in, and the fifth generation (5G) of radio technology is expected to further revolutionise our everyday lives, by enabling a high degree of automation, through its larger capacity, massive connectivity, and ultra-reliable low-latency communications. In addition, the third generation partnership project (3GPP) new radio (NR) specification also provides tools to significantly decrease the energy consumption and the green house emissions of next generations networks, thus contributing towards information and communication technology (ICT) sustainability targets. In this survey paper, we thoroughly review the state-of-the-art on current energy efficiency research. We first categorise and carefully analyse the different power consumption models and energy efficiency metrics, which have helped to make progress on the understanding of green networks. Then, as a main contribution, we survey in detail -- from a theoretical and a practical viewpoint -- the main energy efficiency enabling technologies that 3GPP NR provides, together with their main benefits and challenges. Special attention is paid to four key enabling technologies, i.e., massive multiple-input multiple-output (MIMO), lean carrier design, and advanced idle modes, together with the role of artificial intelligence capabilities. We dive into their implementation and operational details, and thoroughly discuss their optimal operation points and theoretical-trade-offs from an energy consumption perspective. This will help the reader to grasp the fundamentals of -- and the status on -- green networking. Finally, the areas of research where more effort is needed to make future networks greener are also discussed.
Improved algorithms for non-adaptive group testing with consecutive positives
Venue: arXiv cs.IT 2021
Year: 2021

The goal of group testing is to efficiently identify a few specific items, called positives, in a large population of items via tests. A test is an action on a subset of items which returns positive if the subset contains at least one positive and negative otherwise. In non-adaptive group testing, all tests are fixed in advance and can be performed in parallel. In this work, we consider non-adaptive group testing with consecutive positives in which the items are linearly ordered and the positives are consecutive in that order. We present two contributions here. The first is the direct use of a binary code to construct measurement matrices compared to the use of Gray code in the state-of-the-art work, which is a rearrangement of the binary code, when the maximum number of consecutive positives is known. This leads to a reduction in decoding time in practice. The second one is efficient designs to identify positives when the number of consecutive positives is known. To the best of our knowledge, this setting has not been surveyed yet. Our simulations verify the efficiency of our proposed designs. In particular, it only requires up to $300$ tests to identify up to $100$ positives in a set of $2^{32} \approx 4.3\mathrm{B}$ items in less than $300$ nanoseconds. When the maximum number of consecutive positives is known, the simulations validate the superiority of our proposed design in decoding compared to the state-of-the-art work. Moreover, when the number of consecutive positives is known, the number of tests and the decoding time are almost reduced half.
Joint Source-Channel Coding for Semantics-Aware Grant-Free Radio Access in IoT Fog Networks
Venue: arXiv cs.IT 2021
Year: 2021

A fog-radio access network (F-RAN) architecture is studied for an Internet-of-Things (IoT) system in which wireless sensors monitor a number of multi-valued events and transmit in the uplink using grant-free random access to multiple edge nodes (ENs). Each EN is connected to a central processor (CP) via a finite-capacity fronthaul link. In contrast to conventional information-agnostic protocols based on separate source-channel (SSC) coding, where each device uses a separate codebook, this paper considers an information-centric approach based on joint source-channel (JSC) coding via a non-orthogonal generalization of type-based multiple access (TBMA). By leveraging the semantics of the observed signals, all sensors measuring the same event share the same codebook (with non-orthogonal codewords), and all such sensors making the same local estimate of the event transmit the same codeword. The F-RAN architecture directly detects the events values without first performing individual decoding for each device. Cloud and edge detection schemes based on Bayesian message passing are designed and trade-offs between cloud and edge processing are assessed.