OpenRaiser/Intern-Atlas huggingface dataset:
Semantic-WER: A Unified Metric for the Evaluation of ASR Transcript for End Usability
Venue: arXiv cs.CL 2021
Year: 2021

Recent advances in supervised, semi-supervised and self-supervised deep learning algorithms have shown significant improvement in the performance of automatic speech recognition(ASR) systems. The state-of-the-art systems have achieved a word error rate (WER) less than 5%. However, in the past, researchers have argued the non-suitability of the WER metric for the evaluation of ASR systems for downstream tasks such as spoken language understanding (SLU) and information retrieval. The reason is that the WER works at the surface level and does not include any syntactic and semantic knowledge.The current work proposes Semantic-WER (SWER), a metric to evaluate the ASR transcripts for downstream applications in general. The SWER can be easily customized for any down-stream task.
Heterogeneous Noisy Short Signal Camouflage in Multi-Domain Environment Decision-Making
Venue: arXiv cs.LG 2021
Year: 2021

Data transmission between two or more digital devices in industry and government demands secure and agile technology. Digital information distribution often requires deployment of Internet of Things (IoT) devices and Data Fusion techniques which have also gained popularity in both, civilian and military environments, such as, emergence of Smart Cities and Internet of Battlefield Things (IoBT). This usually requires capturing and consolidating data from multiple sources. Because datasets do not necessarily originate from identical sensors, fused data typically results in a complex Big Data problem. Due to potentially sensitive nature of IoT datasets, Blockchain technology is used to facilitate secure sharing of IoT datasets, which allows digital information to be distributed, but not copied. However, blockchain has several limitations related to complexity, scalability, and excessive energy consumption. We propose an approach to hide information (sensor signal) by transforming it to an image or an audio signal. In one of the latest attempts to the military modernization, we investigate sensor fusion approach by investigating the challenges of enabling an intelligent identification and detection operation and demonstrates the feasibility of the proposed Deep Learning and Anomaly Detection models that can support future application for specific hand gesture alert system from wearable devices.
Adaptive Epidemic Forecasting and Community Risk Evaluation of COVID-19
Venue: arXiv cs.CY 2021
Year: 2021

Pandemic control measures like lock-down, restrictions on restaurants and gatherings, social-distancing have shown to be effective in curtailing the spread of COVID-19. However, their sustained enforcement has negative economic effects. To craft strategies and policies that reduce the hardship on the people and the economy while being effective against the pandemic, authorities need to understand the disease dynamics at the right geo-spatial granularity. Considering factors like the hospitals' ability to handle the fluctuating demands, evaluating various reopening scenarios, and accurate forecasting of cases are vital to decision making. Towards this end, we present a flexible end-to-end solution that seamlessly integrates public health data with tertiary client data to accurately estimate the risk of reopening a community. At its core lies a state-of-the-art prediction model that auto-captures changing trends in transmission and mobility. Benchmarking against various published baselines confirm the superiority of our forecasting algorithm. Combined with the ability to extend to multiple client-specific requirements and perform deductive reasoning through counter-factual analysis, this solution provides actionable insights to multiple client domains ranging from government to educational institutions, hospitals, and commercial establishments.
Using Trust in Automation to Enhance Driver-(Semi)Autonomous Vehicle Interaction and Improve Team Performance
Venue: arXiv cs.RO 2021
Year: 2021

Trust in robots has been gathering attention from multiple directions, as it has special relevance in the theoretical descriptions of human-robot interactions. It is essential for reaching high acceptance and usage rates of robotic technologies in society, as well as for enabling effective human-robot teaming. Researchers have been trying to model the development of trust in robots to improve the overall rapport between humans and robots. Unfortunately, the miscalibration of trust in automation is a common issue that jeopardizes the effectiveness of automation use. It happens when a user's trust levels are not appropriate to the capabilities of the automation being used. Users can be: under-trusting the automation -- when they do not use the functionalities that the machine can perform correctly because of a lack of trust; or over-trusting the automation -- when, due to an excess of trust, they use the machine in situations where its capabilities are not adequate. The main objective of this work is to examine driver's trust development in the ADS. We aim to model how risk factors (e.g.: false alarms and misses from the ADS) and the short-term interactions associated with these risk factors influence the dynamics of drivers' trust in the ADS. The driving context facilitates the instrumentation to measure trusting behaviors, such as drivers' eye movements and usage time of the automated features. Our findings indicate that a reliable characterization of drivers' trusting behaviors and a consequent estimation of trust levels is possible. We expect that these techniques will permit the design of ADSs able to adapt their behaviors to attempt to adjust driver's trust levels. This capability could avoid under- and over-trusting, which could harm their safety or their performance.
Analysis of draft EU ADS Performance Requirements
Venue: arXiv cs.RO 2021
Year: 2021

Recently, the European Commission published draft regulation for uniform procedures and technical specification for the type-approval of motor vehicles with an automated driving system (ADS). While the draft regulation is welcome progress for an industry ready to deploy life saving automated vehicle technology, we believe that the requirements can be further improved to enhance the safety and societal acceptance of automated vehicles (AVs). In this paper, we evaluate the draft regulation's performance requirements that would impact the Dynamic Driving Task (DDT). We highlight potential problems that can arise from the current proposed requirements and propose practical recommendations to improve the regulation.
Detecting and Adapting to Novelty in Games
Venue: arXiv cs.AI 2021
Year: 2021

Open-world novelty occurs when the rules of an environment can change abruptly, such as when a game player encounters "house rules". To address open-world novelty, game playing agents must be able to detect when novelty is injected, and to quickly adapt to the new rules. We propose a model-based reinforcement learning approach where game state and rules are represented as knowledge graphs. The knowledge graph representation of the state and rules allows novelty to be detected as changes in the knowledge graph, assists with the training of deep reinforcement learners, and enables imagination-based re-training where the agent uses the knowledge graph to perform look-ahead.
NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer
Venue: arXiv cs.CL 2021
Year: 2021

Autoregressive models have been widely used in unsupervised text style transfer. Despite their success, these models still suffer from the content preservation problem that they usually ignore part of the source sentence and generate some irrelevant words with strong styles. In this paper, we propose a Non-Autoregressive generator for unsupervised text Style Transfer (NAST), which alleviates the problem from two aspects. First, we observe that most words in the transferred sentence can be aligned with related words in the source sentence, so we explicitly model word alignments to suppress irrelevant words. Second, existing models trained with the cycle loss align sentences in two stylistic text spaces, which lacks fine-grained control at the word level. The proposed non-autoregressive generator focuses on the connections between aligned words, which learns the word-level transfer between styles. For experiments, we integrate the proposed generator into two base models and evaluate them on two style transfer tasks. The results show that NAST can significantly improve the overall performance and provide explainable word alignments. Moreover, the non-autoregressive generator achieves over 10x speedups at inference. Our codes are available at https://github.com/thu-coai/NAST.
Fluctuation-dissipation Type Theorem in Stochastic Linear Learning
Venue: arXiv cs.LG 2021
Year: 2021

The fluctuation-dissipation theorem (FDT) is a simple yet powerful consequence of the first-order differential equation governing the dynamics of systems subject simultaneously to dissipative and stochastic forces. The linear learning dynamics, in which the input vector maps to the output vector by a linear matrix whose elements are the subject of learning, has a stochastic version closely mimicking the Langevin dynamics when a full-batch gradient descent scheme is replaced by that of stochastic gradient descent. We derive a generalized FDT for the stochastic linear learning dynamics and verify its validity among the well-known machine learning data sets such as MNIST, CIFAR-10 and EMNIST.
Over-the-Air Computation via Broadband Channels
Venue: arXiv cs.IT 2021
Year: 2021

Over-the-air computation (AirComp) has been recognized as a low-latency solution for wireless sensor data fusion, where multiple sensors send their measurement signals to a receiver simultaneously for computation. Most existing work only considered performing AirComp over a single frequency channel. However, for a sensor network with a massive number of nodes, a single frequency channel may not be sufficient to accommodate the large number of sensors, and the AirComp performance will be very limited. So it is highly desirable to have more frequency channels for large-scale AirComp systems to benefit from multi-channel diversity. In this letter, we propose an $M$-frequency AirComp system, where each sensor selects a subset of the $M$ frequencies and broadcasts its signal over these channels under a certain power constraint. We derive the optimal sensors' transmission and receiver's signal processing methods separately, and develop an algorithm for joint design to achieve the best AirComp performance. Numerical results show that increasing one frequency channel can improve the AirComp performance by threefold compared to the single-frequency case.
Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networks
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning requires several design choices, such as the nodes' activation functions and the widths, types, and arrangements of the layers. One consideration when making these choices is the vanishing-gradient problem, which is the phenomenon of algorithms getting stuck at suboptimal points due to small gradients. In this paper, we revisit the vanishing-gradient problem in the context of sigmoid-type activation. We use mathematical arguments to highlight two different sources of the phenomenon, namely large individual parameters and effects across layers, and to illustrate two simple remedies, namely regularization and rescaling. We then demonstrate the effectiveness of the two remedies in practice. In view of the vanishing-gradient problem being a main reason why tanh and other sigmoid-type activation has become much less popular than relu-type activation, our results bring sigmoid-type activation back to the table.
Towards the adoption of model-based engineering for the development of safety-critical systems in industrial practice
Venue: arXiv cs.SE 2021
Year: 2021

Model-based engineering promises to boost productivity and quality of complex systems development. In the context of safety-critical systems, a traditionally highly regulated and conservative domain, the use of models gained importance in the recent years. In this paper, we present a set of practical challenges in developing safety-critical systems with the help of several examples of development projects that belong to different application domains. Following this, we show how could the adoption of model-based engineering for the development of safety-critical systems cope with these challenges.
Transforming Fading Channel from Fast to Slow: Intelligent Refracting Surface Aided High-Mobility Communication
Venue: arXiv cs.IT 2021
Year: 2021

Intelligent reflecting/refracting surface (IRS) has recently emerged as a promising solution to reconfigure wireless propagation environment for enhancing the communication performance. In this paper, we study a new IRS-aided high-mobility communication system by employing the intelligent refracting surface with a high-speed vehicle to aid its passenger's communication with a remote base station (BS). Due to the environment's random scattering and vehicle's high mobility, a rapidly time-varying channel is typically resulted between the static BS and fast-moving IRS/user, which renders the channel estimation for IRS with a large number of elements more challenging. In order to reap the high IRS passive beamforming gain with low channel training overhead, we propose a new and efficient transmission protocol to achieve both IRS channel estimation and refraction optimization for data transmission. Specifically, by exploiting the quasi-static channel between the IRS and user both moving at the same high speed as well as the line-of-sight (LoS) dominant channel between the BS and IRS, the user first estimates the LoS component of the cascaded BS-IRS-user channel, based on which IRS passive refraction is designed to maximize the corresponding IRS-refracted channel gain. Then, the user estimates the resultant IRS-refracted channel as well as the non-IRS-refracted channel for setting an additional common phase shift at all IRS refracting elements so as to align these two channels for maximizing the overall channel gain for data transmission. Simulation results show significant performance improvement of the proposed design as compared to various benchmark schemes. The proposed on-vehicle IRS system is further compared with a baseline scheme of deploying fixed intelligent reflecting surfaces on the roadside to assist high-speed vehicular communications, which achieves significant rate improvement.
Tackling the Background Bias in Sparse Object Detection via Cropped Windows
Venue: arXiv cs.CV 2021
Year: 2021

Object detection on Unmanned Aerial Vehicles (UAVs) is still a challenging task. The recordings are mostly sparse and contain only small objects. In this work, we propose a simple tiling method that improves the detection capability in the remote sensing case without modifying the model itself. By reducing the background bias and enabling the usage of higher image resolutions during training, our method can improve the performance of models substantially. The procedure was validated on three different data sets and outperformed similar approaches in performance and speed.
Modeling the Unigram Distribution
Venue: arXiv cs.CL 2021
Year: 2021

The unigram distribution is the non-contextual probability of finding a specific word form in a corpus. While of central importance to the study of language, it is commonly approximated by each word's sample frequency in the corpus. This approach, being highly dependent on sample size, assigns zero probability to any out-of-vocabulary (oov) word form. As a result, it produces negatively biased probabilities for any oov word form, while positively biased probabilities to in-corpus words. In this work, we argue in favor of properly modeling the unigram distribution -- claiming it should be a central task in natural language processing. With this in mind, we present a novel model for estimating it in a language (a neuralization of Goldwater et al.'s (2011) model) and show it produces much better estimates across a diverse set of 7 languages than the naïve use of neural character-level language models.
Automatic Patch Linkage Detection in Code Review Using TextualContent and File Location Features
Venue: arXiv cs.SE 2021
Year: 2021

Context: Contemporary code review tools are a popular choice for software quality assurance. Using these tools, reviewers are able to post a linkage between two patches during a review discussion. Large development teams that use a review-then-commit model risk being unaware of these linkages. Objective: Our objective is to first explore how patch linkage impacts the review process. We then propose and evaluate models that detect patch linkage based on realistic time intervals. Method: First, we carry out an exploratory study on three open source projects to conduct linkage impact analysis using 942 manually classified linkages. Second, we propose two techniques using textual and file location similarity to build detection models and evaluate their performance. Results: The study provides evidence of latency in the linkage notification. We show that a patch with the Alternative Solution linkage (i.e., patches that implement similar functionality)undergoes a quicker review and avoids additional revisions after the team has been notified, compared to other linkage types. Our detection model experiments show promising recall rates for the Alternative Solution linkage (from 32% to 95%), but precision has room for improvement. Conclusion: Patch linkage detection is promising, with likely improvements if the practice of posting linkages becomes more prevalent. From our implications, this paper lays the groundwork for future research on how to increase patch linkage awareness to facilitate efficient reviews.
Spectral Hypergraph Sparsifiers of Nearly Linear Size
Venue: arXiv cs.DS 2021
Year: 2021

Graph sparsification has been studied extensively over the past two decades, culminating in spectral sparsifiers of optimal size (up to constant factors). Spectral hypergraph sparsification is a natural analogue of this problem, for which optimal bounds on the sparsifier size are not known, mainly because the hypergraph Laplacian is non-linear, and thus lacks the linear-algebraic structure and tools that have been so effective for graphs. Our main contribution is the first algorithm for constructing $ε$-spectral sparsifiers for hypergraphs with $O^*(n)$ hyperedges, where $O^*$ suppresses $(ε^{-1} \log n)^{O(1)}$ factors. This bound is independent of the rank $r$ (maximum cardinality of a hyperedge), and is essentially best possible due to a recent bit complexity lower bound of $Ω(nr)$ for hypergraph sparsification. This result is obtained by introducing two new tools. First, we give a new proof of spectral concentration bounds for sparsifiers of graphs; it avoids linear-algebraic methods, replacing e.g.~the usual application of the matrix Bernstein inequality and therefore applies to the (non-linear) hypergraph setting. To achieve the result, we design a new sequence of hypergraph-dependent $ε$-nets on the unit sphere in $\mathbb{R}^n$. Second, we extend the weight assignment technique of Chen, Khanna and Nagda [FOCS'20] to the spectral sparsification setting. Surprisingly, the number of spanning trees after the weight assignment can serve as a potential function guiding the reweighting process in the spectral setting.
Annotation Curricula to Implicitly Train Non-Expert Annotators
Venue: arXiv cs.CL 2021
Year: 2021

Annotation studies often require annotators to familiarize themselves with the task, its annotation scheme, and the data domain. This can be overwhelming in the beginning, mentally taxing, and induce errors into the resulting annotations; especially in citizen science or crowd sourcing scenarios where domain expertise is not required and only annotation guidelines are provided. To alleviate these issues, we propose annotation curricula, a novel approach to implicitly train annotators. Our goal is to gradually introduce annotators into the task by ordering instances that are annotated according to a learning curriculum. To do so, we first formalize annotation curricula for sentence- and paragraph-level annotation tasks, define an ordering strategy, and identify well-performing heuristics and interactively trained models on three existing English datasets. We then conduct a user study with 40 voluntary participants who are asked to identify the most fitting misconception for English tweets about the Covid-19 pandemic. Our results show that using a simple heuristic to order instances can already significantly reduce the total annotation time while preserving a high annotation quality. Annotation curricula thus can provide a novel way to improve data collection. To facilitate future research, we further share our code and data consisting of 2,400 annotations.
A Deep Local and Global Scene-Graph Matching for Image-Text Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

Conventional approaches to image-text retrieval mainly focus on indexing visual objects appearing in pictures but ignore the interactions between these objects. Such objects occurrences and interactions are equivalently useful and important in this field as they are usually mentioned in the text. Scene graph presentation is a suitable method for the image-text matching challenge and obtained good results due to its ability to capture the inter-relationship information. Both images and text are represented in scene graph levels and formulate the retrieval challenge as a scene graph matching challenge. In this paper, we introduce the Local and Global Scene Graph Matching (LGSGM) model that enhances the state-of-the-art method by integrating an extra graph convolution network to capture the general information of a graph. Specifically, for a pair of scene graphs of an image and its caption, two separate models are used to learn the features of each graph's nodes and edges. Then a Siamese-structure graph convolution model is employed to embed graphs into vector forms. We finally combine the graph-level and the vector-level to calculate the similarity of this image-text pair. The empirical experiments show that our enhancement with the combination of levels can improve the performance of the baseline method by increasing the recall by more than 10% on the Flickr30k dataset.