OpenRaiser/Intern-Atlas huggingface dataset:
Simplify the Usage of Lexicon in Chinese NER
Venue: arXiv cs.CL 2020
Year: 2020

Recently, many works have tried to augment the performance of Chinese named entity recognition (NER) using word lexicons. As a representative, Lattice-LSTM (Zhang and Yang, 2018) has achieved new benchmark results on several public Chinese NER datasets. However, Lattice-LSTM has a complex model architecture. This limits its application in many industrial areas where real-time NER responses are needed. In this work, we propose a simple but effective method for incorporating the word lexicon into the character representations. This method avoids designing a complicated sequence modeling architecture, and for any neural NER model, it requires only subtle adjustment of the character representation layer to introduce the lexicon information. Experimental studies on four benchmark Chinese NER datasets show that our method achieves an inference speed up to 6.15 times faster than those of state-ofthe-art methods, along with a better performance. The experimental results also show that the proposed method can be easily incorporated with pre-trained models like BERT.
Tackling Online Abuse: A Survey of Automated Abuse Detection Methods
Venue: arXiv cs.CL 2020
Year: 2020

Abuse on the Internet represents an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse on online platforms. The psychological effects of such abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abuse detection in the field of natural language processing (NLP). In this paper, we present a comprehensive survey of the methods that have been proposed to date, thus providing a platform for further development of this area. We describe the existing datasets and review the computational approaches to abuse detection, analyzing their strengths and limitations. We discuss the main trends that emerge, highlight the challenges that remain, outline possible solutions, and propose guidelines for ethics and explainability
ZeroER: Entity Resolution using Zero Labeled Examples
Venue: arXiv cs.DB 2020
Year: 2020

Entity resolution (ER) refers to the problem of matching records in one or more relations that refer to the same real-world entity. While supervised machine learning (ML) approaches achieve the state-of-the-art results, they require a large amount of labeled examples that are expensive to obtain and often times infeasible. We investigate an important problem that vexes practitioners: is it possible to design an effective algorithm for ER that requires Zero labeled examples, yet can achieve performance comparable to supervised approaches? In this paper, we answer in the affirmative through our proposed approach dubbed ZeroER. Our approach is based on a simple observation -- the similarity vectors for matches should look different from that of unmatches. Operationalizing this insight requires a number of technical innovations. First, we propose a simple yet powerful generative model based on Gaussian Mixture Models for learning the match and unmatch distributions. Second, we propose an adaptive regularization technique customized for ER that ameliorates the issue of feature overfitting. Finally, we incorporate the transitivity property into the generative model in a novel way resulting in improved accuracy. On five benchmark ER datasets, we show that ZeroER greatly outperforms existing unsupervised approaches and achieves comparable performance to supervised approaches.
On Convex Duality in Linear Inverse Problems
Venue: arXiv cs.LG 2020
Year: 2020

In this article we dwell into the class of so called ill posed Linear Inverse Problems (LIP) in machine learning, which has become almost a classic in recent times. The fundamental task in an LIP is to recover the entire signal / data from its relatively few random linear measurements. Such problems arise in variety of settings with applications ranging from medical image processing, recommender systems etc. We provide an exposition to the convex duality of the linear inverse problems, and obtain a novel and equivalent convex-concave min-max reformulation that gives rise to simple ascend-descent type algorithms to solve an LIP. Moreover, such a reformulation is crucial in developing methods to solve the dictionary learning problem with almost sure recovery constraints.
A Survey on Computational Politics
Venue: arXiv cs.SI 2020
Year: 2020

Computational Politics is the study of computational methods to analyze and moderate users' behaviors related to political activities such as election campaign persuasion, political affiliation, and opinion mining. With the rapid development and ease of access to the Internet, Information Communication Technologies (ICT) have given rise to massive numbers of users joining online communities and the digitization of analogous data such as political debates. These communities and digitized data contain both explicit and latent information about users and their behaviors related to politics. For researchers, it is essential to utilize data from these sources to develop and design systems that not only provide solutions to computational politics but also help other businesses, such as marketers to increase users, participation and interactions. In this survey, we attempt to categorize main areas in computational politics and summarize the prominent studies in one place to better understand computational politics across different and multidimensional platforms. e.g., online social networks, online forums, and political debates. We then conclude this study by highlighting future research directions, opportunities, and challenges.
CFO: A Framework for Building Production NLP Systems
Venue: arXiv cs.CL 2020
Year: 2020

This paper introduces a novel orchestration framework, called CFO (COMPUTATION FLOW ORCHESTRATOR), for building, experimenting with, and deploying interactive NLP (Natural Language Processing) and IR (Information Retrieval) systems to production environments. We then demonstrate a question answering system built using this framework which incorporates state-of-the-art BERT based MRC (Machine Reading Comprehension) with IR components to enable end-to-end answer retrieval. Results from the demo system are shown to be high quality in both academic and industry domain specific settings. Finally, we discuss best practices when (pre-)training BERT based MRC models for production systems.
Software Verification with PDR: Implementation and Empirical Evaluation of the State of the Art
Venue: arXiv cs.SE 2020
Year: 2020

Property-directed reachability (PDR) is a SAT/SMT-based reachability algorithm that incrementally constructs inductive invariants. After it was successfully applied to hardware model checking, several adaptations to software model checking have been proposed. We contribute a replicable and thorough comparative evaluation of the state of the art: We (1) implemented a standalone PDR algorithm and, as improvement, a PDR-based auxiliary-invariant generator for k-induction, and (2) performed an experimental study on the largest publicly available benchmark set of C verification tasks, in which we explore the effectiveness and efficiency of software verification with PDR. The main contribution of our work is to establish a reproducible baseline for ongoing research in the area by providing a well-engineered reference implementation and an experimental evaluation of the existing techniques.
Neural Dynamics on Complex Networks
Venue: arXiv cs.SI 2020
Year: 2020

Learning continuous-time dynamics on complex networks is crucial for understanding, predicting and controlling complex systems in science and engineering. However, this task is very challenging due to the combinatorial complexities in the structures of high dimensional systems, their elusive continuous-time nonlinear dynamics, and their structural-dynamic dependencies. To address these challenges, we propose to combine Ordinary Differential Equation Systems (ODEs) and Graph Neural Networks (GNNs) to learn continuous-time dynamics on complex networks in a data-driven manner. We model differential equation systems by GNNs. Instead of mapping through a discrete number of neural layers in the forward process, we integrate GNN layers over continuous time numerically, leading to capturing continuous-time dynamics on graphs. Our model can be interpreted as a Continuous-time GNN model or a Graph Neural ODEs model. Our model can be utilized for continuous-time network dynamics prediction, structured sequence prediction (a regularly-sampled case), and node semi-supervised classification tasks (a one-snapshot case) in a unified framework. We validate our model by extensive experiments in the above three scenarios. The promising experimental results demonstrate our model's capability of jointly capturing the structure and dynamics of complex systems in a unified framework.
Directed Homotopy in Non-Positively Curved Spaces
Venue: arXiv cs.LO 2020
Year: 2020

A semantics of concurrent programs can be given using precubical sets, in order to study (higher) commutations between the actions, thus encoding the "geometry" of the space of possible executions of the program. Here, we study the particular case of programs using only mutexes, which are the most widely used synchronization primitive. We show that in this case, the resulting programs have non-positive curvature, a notion that we introduce and study here for precubical sets, and can be thought of as an algebraic analogue of the well-known one for metric spaces. Using this it, as well as categorical rewriting techniques, we are then able to show that directed and non-directed homotopy coincide for directed paths in these precubical sets. Finally, we study the geometric realization of precubical sets in metric spaces, to show that our conditions on precubical sets actually coincide with those for metric spaces. Since the category of metric spaces is not cocomplete, we are lead to work with generalized metric spaces and study some of their properties.
SoK: Three Facets of Privacy Policies
Venue: arXiv cs.CY 2020
Year: 2020

Privacy policies are the main way to obtain information related to personal data collection and processing. Originally, privacy policies were presented as textual documents. However, the unsuitability of this format for the needs of today's society gave birth to other means of expression. In this paper, we systematically study the different means of expression of privacy policies. In doing so, we have explored the three main categories, which we call facets, ie, natural language, graphical and machine-readable privacy policies. Each of these facets focuses on the particular needs of the communities they come from, ie, law experts, organizations and privacy advocates, and academics, respectively. We then analyze the benefits and limitations of each facet, and explain why solutions based on a single facet do not cover the needs of other communities. Finally, we set guidelines and discuss challenges of an approach to expressing privacy policies which brings together the benefits of each facet as an attempt to overcome their limitations.
A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Design
Venue: arXiv cs.AI 2020
Year: 2020

This paper aims to examine the potential of using the emerging deep reinforcement learning techniques in flight control. Instead of learning from scratch, we suggest to leverage domain knowledge available in learning to improve learning efficiency and generalisability. More specifically, the proposed approach fixes the autopilot structure as typical three-loop autopilot and deep reinforcement learning is utilised to learn the autopilot gains. To solve the flight control problem, we then formulate a Markovian decision process with a proper reward function that enable the application of reinforcement learning theory. Another type of domain knowledge is exploited for defining the reward function, by shaping reference inputs in consideration of important control objectives and using the shaped reference inputs in the reward function. The state-of-the-art deep deterministic policy gradient algorithm is utilised to learn an action policy that maps the observed states to the autopilot gains. Extensive empirical numerical simulations are performed to validate the proposed computational control algorithm.
An Autonomous Performance Testing Framework using Self-Adaptive Fuzzy Reinforcement Learning
Venue: arXiv cs.SE 2020
Year: 2020

Test automation brings the potential to reduce costs and human effort, but several aspects of software testing remain challenging to automate. One such example is automated performance testing to find performance breaking points. Current approaches to tackle automated generation of performance test cases mainly involve using source code or system model analysis or use-case based techniques. However, source code and system models might not always be available at testing time. On the other hand, if the optimal performance testing policy for the intended objective in a testing process instead could be learned by the testing system, then test automation without advanced performance models could be possible. Furthermore, the learned policy could later be reused for similar software systems under test, thus leading to higher test efficiency. We propose SaFReL, a self-adaptive fuzzy reinforcement learning-based performance testing framework. SaFReL learns the optimal policy to generate performance test cases through an initial learning phase, then reuses it during a transfer learning phase, while keeping the learning running and updating the policy in the long term. Through multiple experiments on a simulated environment, we demonstrate that our approach generates the target performance test cases for different programs more efficiently than a typical testing process, and performs adaptively without access to source code and performance models.
Proposal-free Temporal Moment Localization of a Natural-Language Query in Video using Guided Attention
Venue: arXiv cs.CV 2020
Year: 2020

This paper studies the problem of temporal moment localization in a long untrimmed video using natural language as the query. Given an untrimmed video and a sentence as the query, the goal is to determine the starting, and the ending, of the relevant visual moment in the video, that corresponds to the query sentence. While previous works have tackled this task by a propose-and-rank approach, we introduce a more efficient, end-to-end trainable, and {\em proposal-free approach} that relies on three key components: a dynamic filter to transfer language information to the visual domain, a new loss function to guide our model to attend the most relevant parts of the video, and soft labels to model annotation uncertainty. We evaluate our method on two benchmark datasets, Charades-STA and ActivityNet-Captions. Experimental results show that our approach outperforms state-of-the-art methods on both datasets.
Instance Scale Normalization for image understanding
Venue: arXiv cs.CV 2020
Year: 2020

Scale variation remains a challenging problem for object detection. Common paradigms usually adopt multiscale training & testing (image pyramid) or FPN (feature pyramid network) to process objects in a wide scale range. However, multi-scale methods aggravate more variations of scale that even deep convolution neural networks with FPN cannot handle well. In this work, we propose an innovative paradigm called Instance Scale Normalization (ISN) to resolve the above problem. ISN compresses the scale space of objects into a consistent range (ISN range), in both training and testing phases. This reassures the problem of scale variation fundamentally and reduces the difficulty of network optimization. Experiments show that ISN surpasses multi-scale counterpart significantly for object detection, instance segmentation, and multi-task human pose estimation, on several architectures. On COCO test-dev, our single model based on ISN achieves 46.5 mAP with a ResNet-101 backbone, which is among the state-of-the-art (SOTA) candidates for object detection.
Towards a Structural Framework for Explicit Domain Knowledge in Visual Analytics
Venue: arXiv cs.HC 2020
Year: 2020

Clinicians and other analysts working with healthcare data are in need for better support to cope with large and complex data. While an increasing number of visual analytics environments integrates explicit domain knowledge as a means to deliver a precise representation of the available data, theoretical work so far has focused on the role of knowledge in the visual analytics process. There has been little discussion about how such explicit domain knowledge can be structured in a generalized framework. This paper collects desiderata for such a structural framework, proposes how to address these desiderata based on the model of linked data, and demonstrates the applicability in a visual analytics environment for physiotherapy.
Computing System Congestion Management Using Exponential Smoothing Forecasting
Venue: arXiv cs.PF 2020
Year: 2020

An overloaded computer must finish what it starts and not start what will fail or hang. A congestion management algorithm the author developed, and Siemens Corporation patented for telecom products, effectively manages traffic overload with its unique formulation of Exponential Smoothing forecasting. Siemens filed for exclusive rights to this technique in 2003 and obtained US patent US7301903B2 in 2007 with this author, an employee at the time of the filing, the sole inventor. A computer program, written in C language, which exercises the methodology is listed at the end of this document and available on GitHub.
RadChat: Spectrum Sharing for Automotive Radar Interference Mitigation
Venue: arXiv cs.IT 2020
Year: 2020

In the automotive sector, both radars and wireless communication are susceptible to interference. However, combining the radar and communication systems, i.e., radio frequency (RF) communications and sensing convergence, has the potential to mitigate interference in both systems. This article analyses the mutual interference of spectrally coexistent frequency modulated continuous wave (FMCW) radar and communication systems in terms of occurrence probability and impact, and introduces RadChat, a distributed networking protocol for mitigation of interference among FMCW based automotive radars, including self-interference, using radar communications. The results show that RadChat can significantly reduce radar mutual interference in single-hop vehicular networks in less than 80 ms.
Mish: A Self Regularized Non-Monotonic Activation Function
Venue: arXiv cs.LG 2020
Year: 2020

We propose $\textit{Mish}$, a novel self-regularized non-monotonic activation function which can be mathematically defined as: $f(x)=x\tanh(softplus(x))$. As activation functions play a crucial role in the performance and training dynamics in neural networks, we validated experimentally on several well-known benchmarks against the best combinations of architectures and activation functions. We also observe that data augmentation techniques have a favorable effect on benchmarks like ImageNet-1k and MS-COCO across multiple architectures. For example, Mish outperformed Leaky ReLU on YOLOv4 with a CSP-DarkNet-53 backbone on average precision ($AP_{50}^{val}$) by 2.1$\%$ in MS-COCO object detection and ReLU on ResNet-50 on ImageNet-1k in Top-1 accuracy by $\approx$1$\%$ while keeping all other network parameters and hyperparameters constant. Furthermore, we explore the mathematical formulation of Mish in relation with the Swish family of functions and propose an intuitive understanding on how the first derivative behavior may be acting as a regularizer helping the optimization of deep neural networks. Code is publicly available at https://github.com/digantamisra98/Mish.
Conditional Flow Variational Autoencoders for Structured Sequence Prediction
Venue: arXiv cs.CV 2020
Year: 2020

Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world. Prior work for structured sequence prediction based on latent variable models imposes a uni-modal standard Gaussian prior on the latent variables. This induces a strong model bias which makes it challenging to fully capture the multi-modality of the distribution of the future states. In this work, we introduce Conditional Flow Variational Autoencoders (CF-VAE) using our novel conditional normalizing flow based prior to capture complex multi-modal conditional distributions for effective structured sequence prediction. Moreover, we propose two novel regularization schemes which stabilizes training and deals with posterior collapse for stable training and better fit to the target data distribution. Our experiments on three multi-modal structured sequence prediction datasets -- MNIST Sequences, Stanford Drone and HighD -- show that the proposed method obtains state of art results across different evaluation metrics.
Two High-Performance Amplitude Beamforming Schemes for Secure Precise Communication and Jamming with Phase Alignment
Venue: arXiv cs.IT 2020
Year: 2020

To severely weaken the eavesdropper's ability to intercept confidential message (CM), a precise jamming (PJ) idea is proposed by making use of the concept of secure precise wireless transmission (SPWT). Its basic idea is to focus the transmit energy of artificial noise (AN) onto the neighborhood of eavesdropper (Eve) by using random subcarrier selection (RSS), directional modulation, phase alignment (PA), and amplitude beamforming (AB). By doing so, Eve will be seriously interfered with AN. Here, the conventional joint optimization of phase and amplitude is converted into two independent phase and amplitude optimization problems. Considering PJ and SPWT require PA, the joint optimization problem reduces to an amplitude optimization problem. Then, two efficient AB schemes are proposed: leakage and maximizing receive power(Max-RP). With existing equal AB (EAB) as a performance reference, simulation results show that the proposed Max-RP and leakage AB methods perform much better than conventional method in terms of both bit-error-rate (BER) and secrecy rate (SR) at medium and high signal-to-noise ratio regions. The performance difference between the two proposed leakage and Max-RP amplitude beamformers is trivial. Additionally, we also find the fact that all three AB schemes EA, Max-RP, and leakage can form two main peaks of AN and CM around Eve and the desired receiver (Bob), respectively. This is what we call PJ and SPWT.
Frustrated Random Walks: A Faster Algorithm to Evaluate Node Distances on Connected and Undirected Graphs
Venue: arXiv cs.DM 2020
Year: 2020

Researchers have designed many algorithms to measure the distances between graph nodes, such as average hitting times of random walks, cosine distances from DeepWalk, personalized PageRank, etc. Successful although these algorithms are, still they are either underperforming or too time-consuming to be applicable to huge graphs that we encounter daily in this big data era. To address these issues, here we propose a faster algorithm based on an improved version of random walks that can beat DeepWalk results with more than ten times acceleration. The reason for this significant acceleration is that we can derive an analytical formula to calculate the expected hitting times of this random walk quickly. There is only one parameter (the power expansion order) in our algorithm, and the results are robust with respect to its changes. Therefore, we can directly find the optimal solution without fine-tuning of model parameters. Our method can be widely used for fraud detection, targeted ads, recommendation systems, topic-sensitive search, etc.