OpenRaiser/Intern-Atlas huggingface dataset:
Machine Translation of Mathematical Text
Venue: arXiv cs.CL 2020
Year: 2020

We have implemented a machine translation system, the PolyMath Translator, for LaTeX documents containing mathematical text. The current implementation translates English LaTeX to French LaTeX, attaining a BLEU score of 53.5 on a held-out test corpus of mathematical sentences. It produces LaTeX documents that can be compiled to PDF without further editing. The system first converts the body of an input LaTeX document into English sentences containing math tokens, using the pandoc universal document converter to parse LaTeX input. We have trained a Transformer-based translator model, using OpenNMT, on a combined corpus containing a small proportion of domain-specific sentences. Our full system uses both this Transformer model and Google Translate, the latter being used as a backup to better handle linguistic features that do not appear in our training dataset. If the Transformer model does not have confidence in its translation, as determined by a high perplexity score, then we use Google Translate with a custom glossary. This backup was used 26% of the time on our test corpus of mathematical sentences. The PolyMath Translator is available as a web service at www.polymathtrans.ai.
Shape-aware Generative Adversarial Networks for Attribute Transfer
Venue: arXiv cs.CV 2020
Year: 2020

Generative adversarial networks (GANs) have been successfully applied to transfer visual attributes in many domains, including that of human face images. This success is partly attributable to the facts that human faces have similar shapes and the positions of eyes, noses, and mouths are fixed among different people. Attribute transfer is more challenging when the source and target domain share different shapes. In this paper, we introduce a shape-aware GAN model that is able to preserve shape when transferring attributes, and propose its application to some real-world domains. Compared to other state-of-art GANs-based image-to-image translation models, the model we propose is able to generate more visually appealing results while maintaining the quality of results from transfer learning.
Distributed Resource Allocation with Multi-Agent Deep Reinforcement Learning for 5G-V2V Communication
Venue: arXiv cs.NI 2020
Year: 2020

We consider the distributed resource selection problem in Vehicle-to-vehicle (V2V) communication in the absence of a base station. Each vehicle autonomously selects transmission resources from a pool of shared resources to disseminate Cooperative Awareness Messages (CAMs). This is a consensus problem where each vehicle has to select a unique resource. The problem becomes more challenging when---due to mobility---the number of vehicles in vicinity of each other is changing dynamically. In a congested scenario, allocation of unique resources for each vehicle becomes infeasible and a congested resource allocation strategy has to be developed. The standardized approach in 5G, namely semi-persistent scheduling (SPS) suffers from effects caused by spatial distribution of the vehicles. In our approach, we turn this into an advantage. We propose a novel DIstributed Resource Allocation mechanism using multi-agent reinforcement Learning (DIRAL) which builds on a unique state representation. One challenging issue is to cope with the non-stationarity introduced by concurrently learning agents which causes convergence problems in multi-agent learning systems. We aimed to tackle non-stationarity with unique state representation. Specifically, we deploy view-based positional distribution as a state representation to tackle non-stationarity and perform complex joint behavior in a distributed fashion. Our results showed that DIRAL improves PRR by 20% compared to SPS in challenging congested scenarios.
Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains
Venue: arXiv cs.CV 2020
Year: 2020

GANs can generate photo-realistic images from the domain of their training data. However, those wanting to use them for creative purposes often want to generate imagery from a truly novel domain, a task which GANs are inherently unable to do. It is also desirable to have a level of control so that there is a degree of artistic direction rather than purely curation of random results. Here we present a method for interpolating between generative models of the StyleGAN architecture in a resolution dependent manner. This allows us to generate images from an entirely novel domain and do this with a degree of control over the nature of the output.
Do Language Embeddings Capture Scales?
Venue: arXiv cs.CL 2020
Year: 2020

Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense, and factual knowledge. One form of knowledge that has not been studied yet in this context is information about the scalar magnitudes of objects. We show that pretrained language models capture a significant amount of this information but are short of the capability required for general common-sense reasoning. We identify contextual information in pre-training and numeracy as two key factors affecting their performance and show that a simple method of canonicalizing numbers can have a significant effect on the results.
Early Abandoning PrunedDTW and its application to similarity search
Venue: arXiv cs.LG 2020
Year: 2020

The Dynamic Time Warping ("DTW") distance is widely used in time series analysis, be it for classification, clustering or similarity search. However, its quadratic time complexity prevents it from scaling. Strategies, based on early abandoning DTW or skipping its computation altogether thanks to lower bounds, have been developed for certain use cases such as nearest neighbour search. But vectorization and approximation aside, no advance was made on DTW itself until recently with the introduction of PrunedDTW. This algorithm, able to prune unpromising alignments, was later fitted with early abandoning. We present a new version of PrunedDTW, "EAPrunedDTW", designed with early abandon in mind from the start, and able to early abandon faster than before. We show that EAPrunedDTW significantly improves the computation time of similarity search in the UCR Suite, and renders lower bounds dispensable.
MMH* with arbitrary modulus is always almost-universal
Venue: arXiv cs.CR 2020
Year: 2020

Universal hash functions, discovered by Carter and Wegman in 1979, are of great importance in computer science with many applications. MMH$^*$ is a well-known $\triangle$-universal hash function family, based on the evaluation of a dot product modulo a prime. In this paper, we introduce a generalization of MMH$^*$, that we call GMMH$^*$, using the same construction as MMH$^*$ but with an arbitrary integer modulus $n>1$, and show that GMMH$^*$ is $\frac{1}{p}$-almost-$\triangle$-universal, where $p$ is the smallest prime divisor of $n$. This bound is tight.
It's not a Non-Issue: Negation as a Source of Error in Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

As machine translation (MT) systems progress at a rapid pace, questions of their adequacy linger. In this study we focus on negation, a universal, core property of human language that significantly affects the semantics of an utterance. We investigate whether translating negation is an issue for modern MT systems using 17 translation directions as test bed. Through thorough analysis, we find that indeed the presence of negation can significantly impact downstream quality, in some cases resulting in quality reductions of more than 60%. We also provide a linguistically motivated analysis that directly explains the majority of our findings. We release our annotations and code to replicate our analysis here: https://github.com/mosharafhossain/negation-mt.
Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion
Venue: arXiv cs.LG 2020
Year: 2020

Active Traffic Management strategies are often adopted in real-time to address such sudden flow breakdowns. When queuing is imminent, Speed Harmonization (SH), which adjusts speeds in upstream traffic to mitigate traffic showckwaves downstream, can be applied. However, because SH depends on driver awareness and compliance, it may not always be effective in mitigating congestion. The use of multiagent reinforcement learning for collaborative learning, is a promising solution to this challenge. By incorporating this technique in the control algorithms of connected and autonomous vehicle (CAV), it may be possible to train the CAVs to make joint decisions that can mitigate highway bottleneck congestion without human driver compliance to altered speed limits. In this regard, we present an RL-based multi-agent CAV control model to operate in mixed traffic (both CAVs and human-driven vehicles (HDVs)). The results suggest that even at CAV percent share of corridor traffic as low as 10%, CAVs can significantly mitigate bottlenecks in highway traffic. Another objective was to assess the efficacy of the RL-based controller vis-à-vis that of the rule-based controller. In addressing this objective, we duly recognize that one of the main challenges of RL-based CAV controllers is the variety and complexity of inputs that exist in the real world, such as the information provided to the CAV by other connected entities and sensed information. These translate as dynamic length inputs which are difficult to process and learn from. For this reason, we propose the use of Graphical Convolution Networks (GCN), a specific RL technique, to preserve information network topology and corresponding dynamic length inputs. We then use this, combined with Deep Deterministic Policy Gradient (DDPG), to carry out multi-agent training for congestion mitigation using the CAV controllers.
A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network
Venue: arXiv cs.AI 2020
Year: 2020

Connected Autonomous Vehicle (CAV) Network can be defined as a collection of CAVs operating at different locations on a multilane corridor, which provides a platform to facilitate the dissemination of operational information as well as control instructions. Cooperation is crucial in CAV operating systems since it can greatly enhance operation in terms of safety and mobility, and high-level cooperation between CAVs can be expected by jointly plan and control within CAV network. However, due to the highly dynamic and combinatory nature such as dynamic number of agents (CAVs) and exponentially growing joint action space in a multiagent driving task, achieving cooperative control is NP hard and cannot be governed by any simple rule-based methods. In addition, existing literature contains abundant information on autonomous driving's sensing technology and control logic but relatively little guidance on how to fuse the information acquired from collaborative sensing and build decision processor on top of fused information. In this paper, a novel Deep Reinforcement Learning (DRL) based approach combining Graphic Convolution Neural Network (GCN) and Deep Q Network (DQN), namely Graphic Convolution Q network (GCQ) is proposed as the information fusion module and decision processor. The proposed model can aggregate the information acquired from collaborative sensing and output safe and cooperative lane changing decisions for multiple CAVs so that individual intention can be satisfied even under a highly dynamic and partially observed mixed traffic. The proposed algorithm can be deployed on centralized control infrastructures such as road-side units (RSU) or cloud platforms to improve the CAV operation.
Fuzzy Approximate Reasoning Method based on Least Common Multiple and its Property Analysis
Venue: arXiv cs.AI 2020
Year: 2020

This paper shows a novel fuzzy approximate reasoning method based on the least common multiple (LCM). Its fundamental idea is to obtain a new fuzzy reasoning result by the extended distance measure based on LCM between the antecedent fuzzy set and the consequent one in discrete SISO fuzzy system. The proposed method is called LCM one. And then this paper analyzes its some properties, i.e., the reductive property, information loss occurred in reasoning process, and the convergence of fuzzy control. Theoretical and experimental research results highlight that proposed method meaningfully improve the reductive property and information loss and controllability than the previous fuzzy reasoning methods.
A Generic Framework For Capturing Reliability in Cyber Physical Systems
Venue: arXiv cs.SE 2020
Year: 2020

Cyber Physical Systems solve complex problems through their tight integration between the physical and computational components. Therefore, the reliability of a complex system is the most critical requirement for the cyber physical system because an unreliable system often leads to service disruption, property dam-age, financial loses and sometimes lead to fatality. In order to develop more reliable CPS, this paper proposes a generic framework for reliability modeling and analysis for our ongoing work on cyber physical systems.This paper, at first defines an architecture for general CPS which is comprised of three layers; environment layer, communication layer, and computational layer. Secondly, we formalize a reliability model for the architectural components, and then propose a framework for the reliability of CPS with the consideration of how to capture the reliability. Based on the research method, we demonstrate the proposed frame-work with an illustrative example by using different reliability values from offshore and onshore reliability data library. We confirmed that the reliability model covers almost all possible reliabilities required to general cyber-physical systems.
Modelling Memory for Individual Re-identification in Decentralised Mobile Contact Tracing Applications
Venue: arXiv cs.CY 2020
Year: 2020

In 2020 the coronavirus outbreak changed the lives of people worldwide. After an initial time period in which it was unclear how to battle the virus, social distancing has been recognised globally as an effective method to mitigate the disease spread. This called for technological tools such as Mobile Contact Tracing Applications (MCTA), which are used to digitally trace contacts among people, and in case a positive case is found, people with the application installed which had been in contact will be notified. De-centralised MCTA may suffer from a novel kind of privacy attack, based on the memory of the human beings, which upon notification of the application can identify who is the positive individual responsible for the notification. Our results show that it is indeed possible to identify positive people among the group of contacts of a human being, and this is even easier when the sociability of the positive individual is low. In practice, our simulation results show that identification can be made with an accuracy of more than 90% depending on the scenario. We also provide three mitigation strategies which can be implemented in de-centralised MCTA and analyse which of the three are more effective in limiting this novel kind of attack.
FILM: A Fast, Interpretable, and Low-rank Metric Learning Approach for Sentence Matching
Venue: arXiv cs.CL 2020
Year: 2020

Detection of semantic similarity plays a vital role in sentence matching. It requires to learn discriminative representations of natural language. Recently, owing to more and more sophisticated model architecture, impressive progress has been made, along with a time-consuming training process and not-interpretable inference. To alleviate this problem, we explore a metric learning approach, named FILM (Fast, Interpretable, and Low-rank Metric learning) to efficiently find a high discriminative projection of the high-dimensional data. We construct this metric learning problem as a manifold optimization problem and solve it with the Cayley transformation method with the Barzilai-Borwein step size. In experiments, we apply FILM with triplet loss minimization objective to the Quora Challenge and Semantic Textual Similarity (STS) Task. The results demonstrate that the FILM method achieves superior performance as well as the fastest computation speed, which is consistent with our theoretical analysis of time complexity.
Toward Cross-Lingual Definition Generation for Language Learners
Venue: arXiv cs.CL 2020
Year: 2020

Generating dictionary definitions automatically can prove useful for language learners. However, it's still a challenging task of cross-lingual definition generation. In this work, we propose to generate definitions in English for words in various languages. To achieve this, we present a simple yet effective approach based on publicly available pretrained language models. In this approach, models can be directly applied to other languages after trained on the English dataset. We demonstrate the effectiveness of this approach on zero-shot definition generation. Experiments and manual analyses on newly constructed datasets show that our models have a strong cross-lingual transfer ability and can generate fluent English definitions for Chinese words. We further measure the lexical complexity of generated and reference definitions. The results show that the generated definitions are much simpler, which is more suitable for language learners.
Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection
Venue: arXiv cs.CV 2020
Year: 2020

How to effectively fuse cross-modal information is the key problem for RGB-D salient object detection. Early fusion and the result fusion schemes fuse RGB and depth information at the input and output stages, respectively, hence incur the problem of distribution gap or information loss. Many models use the feature fusion strategy but are limited by the low-order point-to-point fusion methods. In this paper, we propose a novel mutual attention model by fusing attention and contexts from different modalities. We use the non-local attention of one modality to propagate long-range contextual dependencies for the other modality, thus leveraging complementary attention cues to perform high-order and trilinear cross-modal interaction. We also propose to induce contrast inference from the mutual attention and obtain a unified model. Considering low-quality depth data may detriment the model performance, we further propose selective attention to reweight the added depth cues. We embed the proposed modules in a two-stream CNN for RGB-D SOD. Experimental results have demonstrated the effectiveness of our proposed model. Moreover, we also construct a new challenging large-scale RGB-D SOD dataset with high-quality, thus can both promote the training and evaluation of deep models.
How to Hijack Twitter: Online Polarisation Strategies of Germany's Political Far-Right
Venue: arXiv cs.SI 2020
Year: 2020

With a network approach, we examine the case of the German far-right party Alternative für Deutschland (AfD) and their potential use of a "hashjacking" strategy. Our findings suggest that right-wing politicians (and their supporters/retweeters) actively and effectively polarise the discourse not just by using their own party hashtags, but also by "hashjacking" the political party hashtags of other established parties. The results underline the necessity to understand the success of right-wing parties, online and in elections, not entirely as a result of external effects (e.g. migration), but as a direct consequence of their digital political communication strategy.
Impact of Online Health Awareness Campaign: Case of National Eating Disorders Association
Venue: arXiv cs.CY 2020
Year: 2020

National Eating Disorders Association conducts a NEDAwareness week every year, during which it publishes content on social media and news aimed to raise awareness of eating disorders. Measuring the impact of these actions is vital for maximizing the effectiveness of such interventions. This paper is an effort to model the change in behavior of users who engage with NEDAwareness content. We find that, despite popular influencers being involved in the campaign, it is governmental and nonprofit accounts that attract the most retweets. Furthermore, examining the tweeting language of users engaged with this content, we find linguistic categories concerning women, family, and anxiety to be mentioned more within the 15 days after the intervention, and categories concerning affiliation, references to others, and positive emotion mentioned less. We conclude with actionable implications for future campaigns and discussion of the method's limitations.
Rethinking Experience Replay: a Bag of Tricks for Continual Learning
Venue: arXiv cs.LG 2020
Year: 2020

In Continual Learning, a Neural Network is trained on a stream of data whose distribution shifts over time. Under these assumptions, it is especially challenging to improve on classes appearing later in the stream while remaining accurate on previous ones. This is due to the infamous problem of catastrophic forgetting, which causes a quick performance degradation when the classifier focuses on learning new categories. Recent literature proposed various approaches to tackle this issue, often resorting to very sophisticated techniques. In this work, we show that naive rehearsal can be patched to achieve similar performance. We point out some shortcomings that restrain Experience Replay (ER) and propose five tricks to mitigate them. Experiments show that ER, thus enhanced, displays an accuracy gain of 51.2 and 26.9 percentage points on the CIFAR-10 and CIFAR-100 datasets respectively (memory buffer size 1000). As a result, it surpasses current state-of-the-art rehearsal-based methods.
Convolutional Neural Network optimization via Channel Reassessment Attention module
Venue: arXiv cs.CV 2020
Year: 2020

The performance of convolutional neural networks (CNNs) can be improved by adjusting the interrelationship between channels with attention mechanism. However, attention mechanism in recent advance has not fully utilized spatial information of feature maps, which makes a great difference to the results of generated channel attentions. In this paper, we propose a novel network optimization module called Channel Reassessment Attention (CRA) module which uses channel attentions with spatial information of feature maps to enhance representational power of networks. We employ CRA module to assess channel attentions based on feature maps in different channels, then the final features are refined adaptively by product between channel attentions and feature maps.CRA module is a computational lightweight module and it can be embedded into any architectures of CNNs. The experiments on ImageNet, CIFAR and MS COCO datasets demonstrate that the embedding of CRA module on various networks effectively improves the performance under different evaluation standards.
Load What You Need: Smaller Versions of Multilingual BERT
Venue: arXiv cs.CL 2020
Year: 2020

Pre-trained Transformer-based models are achieving state-of-the-art results on a variety of Natural Language Processing data sets. However, the size of these models is often a drawback for their deployment in real production applications. In the case of multilingual models, most of the parameters are located in the embeddings layer. Therefore, reducing the vocabulary size should have an important impact on the total number of parameters. In this paper, we propose to generate smaller models that handle fewer number of languages according to the targeted corpora. We present an evaluation of smaller versions of multilingual BERT on the XNLI data set, but we believe that this method may be applied to other multilingual transformers. The obtained results confirm that we can generate smaller models that keep comparable results, while reducing up to 45% of the total number of parameters. We compared our models with DistilmBERT (a distilled version of multilingual BERT) and showed that unlike language reduction, distillation induced a 1.7% to 6% drop in the overall accuracy on the XNLI data set. The presented models and code are publicly available.