OpenRaiser/Intern-Atlas huggingface dataset:
Continuity of Generalized Entropy and Statistical Learning
Venue: arXiv cs.LG 2021
Year: 2021

We study the continuity property of the generalized entropy as a function of the underlying probability distribution, defined with an action space and a loss function, and use this property to answer the basic questions in statistical learning theory: the excess risk analyses for various learning methods. We first derive upper and lower bounds for the entropy difference of two distributions in terms of several commonly used f-divergences, the Wasserstein distance, a distance that depends on the action space and the loss function, and the Bregman divergence generated by the entropy, which also induces bounds in terms of the Euclidean distance between the two distributions. Examples are given along with the discussion of each general result, comparisons are made with the existing entropy difference bounds, and new mutual information upper bounds are derived based on the new results. We then apply the entropy difference bounds to the theory of statistical learning. It is shown that the excess risks in the two popular learning paradigms, the frequentist learning and the Bayesian learning, both can be studied with the continuity property of different forms of the generalized entropy. The analysis is then extended to the continuity of generalized conditional entropy. The extension provides performance bounds for Bayes decision making with mismatched distributions. It also leads to excess risk bounds for a third paradigm of learning, where the decision rule is optimally designed under the projection of the empirical distribution to a predefined family of distributions. We thus establish a unified method of excess risk analysis for the three major paradigms of statistical learning, through the continuity of generalized entropy.
Detection in Molecular Communications with Ligand Receptors under Molecular Interference
Venue: arXiv cs.ET 2021
Year: 2021

Molecular Communications (MC) is a bio-inspired communication technique that uses molecules to transfer information among bio-nano devices. In this paper, we focus on the detection problem for biological MC receivers employing ligand receptors to infer the transmitted messages encoded into the concentration of molecules, i.e., ligands. In practice, receptors are not ideally selective against target ligands, and in physiological environments, they can interact with multiple types of ligands at different reaction rates depending on their binding affinity. This molecular cross-talk can cause a substantial interference on MC. Here we consider a particular scenario, where there is non-negligible concentration of interferer molecules in the channel, which have similar receptor-binding characteristics with the information molecules, and the receiver employs single type of receptors. We investigate the performance of four different detection methods, which make use of different statistics of the ligand-receptor binding reactions: instantaneous number of bound receptors, unbound time durations of receptors, bound time durations of receptors, and combination of unbound and bound time durations of receptors within a sampling time interval. The performance of the introduced detection methods are evaluated in terms of bit error probability for varying strength of molecular interference, similarity between information and interferer molecules, number of receptors, and received concentration difference between bit-0 and bit-1 transmissions. We propose synthetic receptor designs that can convert the required receptor statistics to the concentration of intracellular molecules, and chemical reaction networks that can chemically perform the computations required for detection.
Adaptive Surgical Robotic Training Using Real-Time Stylistic Behavior Feedback Through Haptic Cues
Venue: arXiv cs.RO 2021
Year: 2021

Surgical skill directly affects surgical procedure outcomes; thus, effective training is needed to ensure satisfactory results. Many objective assessment metrics have been developed and some are widely used in surgical training simulators. These objective metrics provide the trainee with descriptive feedback about their performance however, often lack feedback on how to proceed to improve performance. The most effective training method is one that is intuitive, easy to understand, personalized to the user and provided in a timely manner. We propose a framework to enable user-adaptive training using near-real-time detection of performance, based on intuitive styles of surgical movements (e.g., fluidity, smoothness, crispness, etc.), and propose a haptic feedback framework to assist with correcting styles of movement. We evaluate the ability of three types of force feedback (spring, damping, and spring plus damping feedback), computed based on prior user positions, to improve different stylistic behaviors of the user during kinematically constrained reaching movement tasks. The results indicate that four out of the six styles studied here were statistically significantly improved (p<0.05) using spring guidance force feedback and a significant reduction in task time was also found using spring feedback. The path straightness and targeting error in the task were other task performance metrics studied which were improved significantly using the spring-damping feedback. This study presents a groundwork for adaptive training in robotic surgery based on near-real-time human-centric models of surgical behavior.
Achievable Rates for Binary Two-hop Channel with Energy Harvesting Relay and Finite Battery
Venue: arXiv cs.IT 2021
Year: 2021

We study the problem of joint information and energy transfer in a binary two-hop channel with an energy harvesting relay. We consider a finite battery size at the relay and energy loss in transmitting energy. In other words, to be able to send an energy-contained symbol, the relay must receive multiple energy-contained symbols. Thus, we face a kind of channel with memory. We model the energy saved in the battery as the channel state with the challenge that the receiver does not know the channel state. We propose two different achievable schemes, the first one is based on state-dependent superposition coding and the second one is based on the equivalent timing channel approach. Both of our schemes are based on block Markov coding and backward decoding techniques. Due to these two approaches, we find achievable rates with a single-letter expression for the model.
Sensei: Self-Supervised Sensor Name Segmentation
Venue: arXiv cs.CL 2021
Year: 2021

A sensor name, typically an alphanumeric string, encodes the key context (e.g., function and location) of a sensor needed for deploying smart building applications. Sensor names, however, are curated in a building vendor-specific manner using different structures and vocabularies that are often esoteric. They thus require tremendous manual effort to annotate on a per-building basis; even to just segment these sensor names into meaningful chunks. In this paper, we propose a fully automated self-supervised framework, Sensei, which can learn to segment sensor names without any human annotation. Specifically, we employ a neural language model to capture the underlying sensor naming structure and then induce self-supervision based on information from the language model to build the segmentation model. Extensive experiments on five real-world buildings comprising thousands of sensors demonstrate the superiority of Sensei over baseline methods.
Sub-Gaussian Error Bounds for Hypothesis Testing
Venue: arXiv cs.IT 2021
Year: 2021

We interpret likelihood-based test functions from a geometric perspective where the Kullback-Leibler (KL) divergence is adopted to quantify the distance from a distribution to another. Such a test function can be seen as a sub-Gaussian random variable, and we propose a principled way to calculate its corresponding sub-Gaussian norm. Then an error bound for binary hypothesis testing can be obtained in terms of the sub-Gaussian norm and the KL divergence, which is more informative than Pinsker's bound when the significance level is prescribed. For $M$-ary hypothesis testing, we also derive an error bound which is complementary to Fano's inequality by being more informative when the number of hypotheses or the sample size is not large.
Unifying Discourse Resources with Dependency Framework
Venue: arXiv cs.CL 2021
Year: 2021

For text-level discourse analysis, there are various discourse schemes but relatively few labeled data, because discourse research is still immature and it is labor-intensive to annotate the inner logic of a text. In this paper, we attempt to unify multiple Chinese discourse corpora under different annotation schemes with discourse dependency framework by designing semi-automatic methods to convert them into dependency structures. We also implement several benchmark dependency parsers and research on how they can leverage the unified data to improve performance.
CIZSL++: Creativity Inspired Generative Zero-Shot Learning
Venue: arXiv cs.CV 2021
Year: 2021

Zero-shot learning (ZSL) aims at understanding unseen categories with no training examples from class-level descriptions. To improve the discriminative power of ZSL, we model the visual learning process of unseen categories with inspiration from the psychology of human creativity for producing novel art. First, we propose CIZSL-v1 as a creativity inspired model for generative ZSL. We relate ZSL to human creativity by observing that ZSL is about recognizing the unseen, and creativity is about creating a likable unseen. We introduce a learning signal inspired by creativity literature that explores the unseen space with hallucinated class-descriptions and encourages careful deviation of their visual feature generations from seen classes while allowing knowledge transfer from seen to unseen classes. Second, CIZSL-v2 is proposed as an improved version of CIZSL-v1 for generative zero-shot learning. CIZSL-v2 consists of an investigation of additional inductive losses for unseen classes along with a semantic guided discriminator. Empirically, we show consistently that CIZSL losses can improve generative ZSL models on the challenging task of generalized ZSL from a noisy text on CUB and NABirds datasets. We also show the advantage of our approach to Attribute-based ZSL on AwA2, aPY, and SUN datasets. We also show that CIZSL-v2 has improved performance compared to CIZSL-v1.
Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning
Venue: arXiv cs.LG 2021
Year: 2021

This paper focuses on inverse reinforcement learning for autonomous navigation using distance and semantic category observations. The objective is to infer a cost function that explains demonstrated behavior while relying only on the expert's observations and state-control trajectory. We develop a map encoder, that infers semantic category probabilities from the observation sequence, and a cost encoder, defined as a deep neural network over the semantic features. Since the expert cost is not directly observable, the model parameters can only be optimized by differentiating the error between demonstrated controls and a control policy computed from the cost estimate. We propose a new model of expert behavior that enables error minimization using a closed-form subgradient computed only over a subset of promising states via a motion planning algorithm. Our approach allows generalizing the learned behavior to new environments with new spatial configurations of the semantic categories. We analyze the different components of our model in a minigrid environment. We also demonstrate that our approach learns to follow traffic rules in the autonomous driving CARLA simulator by relying on semantic observations of buildings, sidewalks, and road lanes.
On Explaining Your Explanations of BERT: An Empirical Study with Sequence Classification
Venue: arXiv cs.CL 2021
Year: 2021

BERT, as one of the pretrianed language models, attracts the most attention in recent years for creating new benchmarks across GLUE tasks via fine-tuning. One pressing issue is to open up the blackbox and explain the decision makings of BERT. A number of attribution techniques have been proposed to explain BERT models, but are often limited to sequence to sequence tasks. In this paper, we adapt existing attribution methods on explaining decision makings of BERT in sequence classification tasks. We conduct extensive analyses of four existing attribution methods by applying them to four different datasets in sentiment analysis. We compare the reliability and robustness of each method via various ablation studies. Furthermore, we test whether attribution methods explain generalized semantics across semantically similar tasks. Our work provides solid guidance for using attribution methods to explain decision makings of BERT for downstream classification tasks.
Interplay between RIS and AI in Wireless Communications: Fundamentals, Architectures, Applications, and Open Research Problems
Venue: arXiv cs.IT 2021
Year: 2021

Future wireless communication networks are expected to fulfill the unprecedented performance requirements to support our highly digitized and globally data-driven society. Various technological challenges must be overcome to achieve our goal. Among many potential technologies, reconfigurable intelligent surface (RIS) and artificial intelligence (AI) have attracted extensive attention, thereby leading to a proliferation of studies for utilizing them in wireless communication systems. The RIS-based wireless communication frameworks and AI-enabled technologies, two of the promising technologies for the sixth-generation networks, interact and promote with each other, striving to collaboratively create a controllable, intelligent, reconfigurable, and programmable wireless propagation environment. This paper explores the road to implementing the combination of RIS and AI; specifically, integrating AI-enabled technologies into RIS-based frameworks for maximizing the practicality of RIS to facilitate the realization of smart radio propagation environments, elaborated from shallow to deep insights. We begin with the basic concept and fundamental characteristics of RIS, followed by the overview of the research status of RIS. Then, we analyze the inevitable trend of RIS to be combined with AI. In particular, we focus on recent research about RIS-based architectures embedded with AI, elucidating from the intelligent structures and systems of metamaterials to the AI-embedded RIS-assisted wireless communication systems. Finally, the challenges and potential of the topic are discussed.
Code Generation from Natural Language with Less Prior and More Monolingual Data
Venue: arXiv cs.CL 2021
Year: 2021

Training datasets for semantic parsing are typically small due to the higher expertise required for annotation than most other NLP tasks. As a result, models for this application usually need additional prior knowledge to be built into the architecture or algorithm. The increased dependency on human experts hinders automation and raises the development and maintenance costs in practice. This work investigates whether a generic transformer-based seq2seq model can achieve competitive performance with minimal code-generation-specific inductive bias design. By exploiting a relatively sizeable monolingual corpus of the target programming language, which is cheap to mine from the web, we achieved 81.03% exact match accuracy on Django and 32.57 BLEU score on CoNaLa. Both are SOTA to the best of our knowledge. This positive evidence highlights a potentially easier path toward building accurate semantic parsers in practice.
The Challenges in Modeling Human Performance in 3D Space with Fitts' Law
Venue: arXiv cs.HC 2021
Year: 2021

With the rapid growth in virtual reality technologies, object interaction is becoming increasingly more immersive, elucidating human perception and leading to promising directions towards evaluating human performance under different settings. This spike in technological growth exponentially increased the need for a human performance metric in 3D space. Fitts' law is perhaps the most widely used human prediction model in HCI history attempting to capture human movement in lower dimensions. Despite the collective effort towards deriving an advanced extension of a 3D human performance model based on Fitts' law, a standardized metric is still missing. Moreover, most of the extensions to date assume or limit their findings to certain settings, effectively disregarding important variables that are fundamental to 3D object interaction. In this review, we investigate and analyze the most prominent extensions of Fitts' law and compare their characteristics pinpointing to potentially important aspects for deriving a higher-dimensional performance model. Lastly, we mention the complexities, frontiers as well as potential challenges that may lay ahead.
Robot Adaptation for Generating Consistent Navigational Behaviors over Unstructured Off-Road Terrain
Venue: arXiv cs.RO 2021
Year: 2021

Terrain adaptation is an essential capability for a ground robot to effectively traverse unstructured off-road terrain in real-world field environments such as forests. However, the expected robot behaviors generated by terrain adaptation methods cannot always be executed accurately due to setbacks such as wheel slip and reduced tire pressure. To address this problem, we propose a novel approach for consistent behavior generation that enables the ground robot's actual behaviors to more accurately match expected behaviors while adapting to a variety of unstructured off-road terrain. Our approach learns offset behaviors that are used to compensate for the inconsistency between the actual and expected behaviors without requiring the explicit modeling of various setbacks. Our approach is also able to estimate the importance of the multi-modal features to improve terrain representations for better adaptation. In addition, we develop an algorithmic solver for our formulated regularized optimization problem, which is guaranteed to converge to the global optimal solution. To evaluate the method, we perform extensive experiments using various unstructured off-road terrain in real-world field environments. Experimental results have validated that our approach enables robots to traverse complex unstructured off-road terrain with more navigational behavior consistency, and it outperforms previous methods, particularly so on challenging terrain.
Iranis: A Large-scale Dataset of Farsi License Plate Characters
Venue: arXiv cs.CV 2021
Year: 2021

Providing huge amounts of data is a fundamental demand when dealing with Deep Neural Networks (DNNs). Employing these algorithms to solve computer vision problems resulted in the advent of various image datasets to feed the most common visual imagery deep structures, known as Convolutional Neural Networks (CNNs). In this regard, some datasets can be found that contain hundreds or even thousands of images for license plate detection and optical character recognition purposes. However, no publicly available image dataset provides such data for the recognition of Farsi characters used in car license plates. The gap has to be filled due to the numerous advantages of developing accurate deep learning-based systems for law enforcement and surveillance purposes. This paper introduces a large-scale dataset that includes images of numbers and characters used in Iranian car license plates. The dataset, named Iranis, contains more than 83,000 images of Farsi numbers and letters collected from real-world license plate images captured by various cameras. The variety of instances in terms of camera shooting angle, illumination, resolution, and contrast make the dataset a proper choice for training DNNs. Dataset images are manually annotated for object detection and image classification. Finally, and to build a baseline for Farsi character recognition, the paper provides a performance analysis using a YOLO v.3 object detector.
Identity-aware Facial Expression Recognition in Compressed Video
Venue: arXiv cs.CV 2021
Year: 2021

This paper targets to explore the inter-subject variations eliminated facial expression representation in the compressed video domain. Most of the previous methods process the RGB images of a sequence, while the off-the-shelf and valuable expression-related muscle movement already embedded in the compression format. In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possible to extract identity factors from the I frame with a pre-trained face recognition network. By enforcing the marginal independent of them, the expression feature is expected to be purer for the expression and be robust to identity shifts. We do not need the identity label or multiple expression samples from the same person for identity elimination. Moreover, when the apex frame is annotated in the dataset, the complementary constraint can be further added to regularize the feature-level game. In testing, only the compressed residual frames are required to achieve expression prediction. Our solution can achieve comparable or better performance than the recent decoded image based methods on the typical FER benchmarks with about 3$\times$ faster inference with compressed data.
GIS and Computational Notebooks
Venue: arXiv cs.CY 2021
Year: 2021

Researchers and practitioners across many disciplines have recently adopted computational notebooks to develop, document, and share their scientific workflows - and the GIS community is no exception. This chapter introduces computational notebooks in the geographical context. It begins by explaining the computational paradigm and philosophy that underlie notebooks. Next it unpacks their architecture to illustrate a notebook user's typical workflow. Then it discusses the main benefits notebooks offer GIS researchers and practitioners, including better integration with modern software, more natural access to new forms of data, and better alignment with the principles and benefits of open science. In this context, it identifies notebooks as the glue that binds together a broader ecosystem of open source packages and transferable platforms for computational geography. The chapter concludes with a brief illustration of using notebooks for a set of basic GIS operations. Compared to traditional desktop GIS, notebooks can make spatial analysis more nimble, extensible, and reproducible and have thus evolved into an important component of the geospatial science toolkit.
To Share, or not to Share Online Event Trend Aggregation Over Bursty Event Streams
Venue: arXiv cs.DB 2021
Year: 2021

Complex event processing (CEP) systems continuously evaluate large workloads of pattern queries under tight time constraints. Event trend aggregation queries with Kleene patterns are commonly used to retrieve summarized insights about the recent trends in event streams. State-of-art methods are limited either due to repetitive computations or unnecessary trend construction. Existing shared approaches are guided by statically selected and hence rigid sharing plans that are often sub-optimal under stream fluctuations. In this work, we propose a novel framework Hamlet that is the first to overcome these limitations. Hamlet introduces two key innovations. First, Hamlet adaptively decides whether to share or not to share computations depending on the current stream properties at run time to harvest the maximum sharing benefit. Second, Hamlet is equipped with a highly efficient shared trend aggregation strategy that avoids trend construction. Our experimental study on both real and synthetic data sets demonstrates that Hamlet consistently reduces query latency by up to five orders of magnitude compared to the state-of-the-art approaches.