OpenRaiser/Intern-Atlas huggingface dataset:
The Effect of Sociocultural Variables on Sarcasm Communication Online
Venue: arXiv cs.SI 2020
Year: 2020

Online social networks (OSN) play an essential role for connecting people and allowing them to communicate online. OSN users share their thoughts, moments, and news with their network. The messages they share online can include sarcastic posts, where the intended meaning expressed by the written text is different from the literal one. This could result in miscommunication. Previous research in psycholinguistics has studied the sociocultural factors the might lead to sarcasm misunderstanding between speakers and listeners. However, there is a lack of such studies in the context of OSN. In this paper we fill this gap by performing a quantitative analysis on the influence of sociocultural variables, including gender, age, country, and English language nativeness, on the effectiveness of sarcastic communication online. We collect examples of sarcastic tweets directly from the authors who posted them. Further, we ask third-party annotators of different sociocultural backgrounds to label these tweets for sarcasm. Our analysis indicates that age, English language nativeness, and country are significantly influential and should be considered in the design of future social analysis tools that either study sarcasm directly, or look at related phenomena where sarcasm may have an influence. We also make observations about the social ecology surrounding sarcastic exchanges on OSNs. We conclude by suggesting ways in which our findings can be included in future work.
A Flexible Connector for Soft Modular Robots Based on Micropatterned Intersurface Jamming
Venue: arXiv cs.RO 2020
Year: 2020

Soft modular robots enable more flexibility and safer interaction with the changing environment than traditional robots. However, it has remained challenging to create deformable connectors that can be integrated into soft machines. In this work, we propose a flexible connector for soft modular robots based on micropatterned intersurface jamming. The connector is composed of micropatterned dry adhesives made by silicone rubber and a flexible main body with inflatable chambers for active engagement and disengagement. Through connection force tests, we evaluate the characteristics of the connector both in the linear direction and under rotational disruptions. The connector can stably support an average maximum load of 22 N (83 times the connector's body weight) linearly and 10.86 N under planar rotation. The proposed connector demonstrates the potential to create a robust connection between soft modular robots without raising the system's overall stiffness; thus guarantees high flexibility of the robotic system.
A Simplified Run Time Analysis of the Univariate Marginal Distribution Algorithm on LeadingOnes
Venue: arXiv cs.NE 2020
Year: 2020

With elementary means, we prove a stronger run time guarantee for the univariate marginal distribution algorithm (UMDA) optimizing the LeadingOnes benchmark function in the desirable regime with low genetic drift. If the population size is at least quasilinear, then, with high probability, the UMDA samples the optimum within a number of iterations that is linear in the problem size divided by the logarithm of the UMDA's selection rate. This improves over the previous guarantee, obtained by Dang and Lehre (2015) via the deep level-based population method, both in terms of the run time and by demonstrating further run time gains from small selection rates. With similar arguments as in our upper-bound analysis, we also obtain the first lower bound for this problem. Under similar assumptions, we prove that a bound that matches our upper bound up to constant factors holds with high probability.
Negation Detection for Clinical Text Mining in Russian
Venue: arXiv cs.CL 2020
Year: 2020

Developing predictive modeling in medicine requires additional features from unstructured clinical texts. In Russia, there are no instruments for natural language processing to cope with problems of medical records. This paper is devoted to a module of negation detection. The corpus-free machine learning method is based on gradient boosting classifier is used to detect whether a disease is denied, not mentioned or presented in the text. The detector classifies negations for five diseases and shows average F-score from 0.81 to 0.93. The benefits of negation detection have been demonstrated by predicting the presence of surgery for patients with the acute coronary syndrome.
Improved Residual Networks for Image and Video Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Residual networks (ResNets) represent a powerful type of convolutional neural network (CNN) architecture, widely adopted and used in various tasks. In this work we propose an improved version of ResNets. Our proposed improvements address all three main components of a ResNet: the flow of information through the network layers, the residual building block, and the projection shortcut. We are able to show consistent improvements in accuracy and learning convergence over the baseline. For instance, on ImageNet dataset, using the ResNet with 50 layers, for top-1 accuracy we can report a 1.19% improvement over the baseline in one setting and around 2% boost in another. Importantly, these improvements are obtained without increasing the model complexity. Our proposed approach allows us to train extremely deep networks, while the baseline shows severe optimization issues. We report results on three tasks over six datasets: image classification (ImageNet, CIFAR-10 and CIFAR-100), object detection (COCO) and video action recognition (Kinetics-400 and Something-Something-v2). In the deep learning era, we establish a new milestone for the depth of a CNN. We successfully train a 404-layer deep CNN on the ImageNet dataset and a 3002-layer network on CIFAR-10 and CIFAR-100, while the baseline is not able to converge at such extreme depths. Code is available at: https://github.com/iduta/iresnet
Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric
Venue: arXiv cs.CL 2020
Year: 2020

The rapid development of such natural language processing tasks as style transfer, paraphrase, and machine translation often calls for the use of semantic similarity metrics. In recent years a lot of methods to measure the semantic similarity of two short texts were developed. This paper provides a comprehensive analysis for more than a dozen of such methods. Using a new dataset of fourteen thousand sentence pairs human-labeled according to their semantic similarity, we demonstrate that none of the metrics widely used in the literature is close enough to human judgment in these tasks. A number of recently proposed metrics provide comparable results, yet Word Mover Distance is shown to be the most reasonable solution to measure semantic similarity in reformulated texts at the moment.
Estimating Individual Treatment Effects through Causal Populations Identification
Venue: arXiv cs.LG 2020
Year: 2020

Estimating the Individual Treatment Effect from observational data, defined as the difference between outcomes with and without treatment or intervention, while observing just one of both, is a challenging problems in causal learning. In this paper, we formulate this problem as an inference from hidden variables and enforce causal constraints based on a model of four exclusive causal populations. We propose a new version of the EM algorithm, coined as Expected-Causality-Maximization (ECM) algorithm and provide hints on its convergence under mild conditions. We compare our algorithm to baseline methods on synthetic and real-world data and discuss its performances.
Colouring $(sP_1+P_5)$-Free Graphs: a Mim-Width Perspective
Venue: arXiv cs.DS 2020
Year: 2020

We prove that the class of $(K_t,sP_1+P_5)$-free graphs has bounded mim-width for every $s\geq 0$ and $t\geq 1$, and that there is a polynomial-time algorithm that, given a graph in the class, computes a branch decomposition of constant mim-width. A large number of \NP-complete graph problems become polynomial-time solvable on graph classes with bounded mim-width and for which a branch decomposition is quickly computable. The $k$-Colouring problem is an example of such a problem. For this problem, we may assume that the input graph is $K_{k+1}$-free. Then, as a consequence of our result, we obtain a new proof for the known result that for every fixed $k\geq 1$ and $s\geq 0$, $k$-Colouring is polynomial-time solvable for $(sP_1+P_5)$-free graphs. In fact, our findings show that the underlying reason for this polynomial-time algorithm is that the class has bounded mim-width.
A Modified Bayesian Optimization based Hyper-Parameter Tuning Approach for Extreme Gradient Boosting
Venue: arXiv cs.LG 2020
Year: 2020

It is already reported in the literature that the performance of a machine learning algorithm is greatly impacted by performing proper Hyper-Parameter optimization. One of the ways to perform Hyper-Parameter optimization is by manual search but that is time consuming. Some of the common approaches for performing Hyper-Parameter optimization are Grid search Random search and Bayesian optimization using Hyperopt. In this paper, we propose a brand new approach for hyperparameter improvement i.e. Randomized-Hyperopt and then tune the hyperparameters of the XGBoost i.e. the Extreme Gradient Boosting algorithm on ten datasets by applying Random search, Randomized-Hyperopt, Hyperopt and Grid Search. The performances of each of these four techniques were compared by taking both the prediction accuracy and the execution time into consideration. We find that the Randomized-Hyperopt performs better than the other three conventional methods for hyper-paramter optimization of XGBoost.
Joint Learning of Probabilistic and Geometric Shaping for Coded Modulation Systems
Venue: arXiv cs.IT 2020
Year: 2020

We introduce a trainable coded modulation scheme that enables joint optimization of the bit-wise mutual information (BMI) through probabilistic shaping, geometric shaping, bit labeling, and demapping for a specific channel model and for a wide range of signal-to-noise ratios (SNRs). Compared to probabilistic amplitude shaping (PAS), the proposed approach is not restricted to symmetric probability distributions, can be optimized for any channel model, and works with any code rate $k/m$, $m$ being the number of bits per channel use and $k$ an integer within the range from $1$ to $m-1$. The proposed scheme enables learning of a continuum of constellation geometries and probability distributions determined by the SNR. Additionally, the PAS architecture with Maxwell-Boltzmann (MB) as shaping distribution was extended with a neural network (NN) that controls the MB shaping of a quadrature amplitude modulation (QAM) constellation according to the SNR, enabling learning of a continuum of MB distributions for QAM. Simulations were performed to benchmark the performance of the proposed joint probabilistic and geometric shaping scheme on additive white Gaussian noise (AWGN) and mismatched Rayleigh block fading (RBF) channels.
Secret Key Generation from Vector Gaussian Sources with Public and Private Communications
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we consider the problem of secret key generation with one-way communication through both a rate-limited public channel and a rate-limited secure channels where the public channel is from Alice to Bob and Eve and the secure channel is from Alice to Bob. In this model, we do not pose any constraints on the sources, i.e. Bob is not degraded to or less noisy than Eve. We obtain the optimal secret key rate in this problem, both for the discrete memoryless sources and vector Gaussian sources. The vector Gaussian characterization is derived by suitably applying the enhancement argument, and Proving a new extremal inequality. The extremal inequality can be seen as coupling of two extremal inequalities, which are related to the degraded compound MIMO Gaussian broadcast channel, and the vector generalization of Costa's entropy power inequality, accordingly.
Levels of Analysis for Machine Learning
Venue: arXiv cs.CY 2020
Year: 2020

Machine learning is currently involved in some of the most vigorous debates it has ever seen. Such debates often seem to go around in circles, reaching no conclusion or resolution. This is perhaps unsurprising given that researchers in machine learning come to these discussions with very different frames of reference, making it challenging for them to align perspectives and find common ground. As a remedy for this dilemma, we advocate for the adoption of a common conceptual framework which can be used to understand, analyze, and discuss research. We present one such framework which is popular in cognitive science and neuroscience and which we believe has great utility in machine learning as well: Marr's levels of analysis. Through a series of case studies, we demonstrate how the levels facilitate an understanding and dissection of several methods from machine learning. By adopting the levels of analysis in one's own work, we argue that researchers can be better equipped to engage in the debates necessary to drive forward progress in our field.
Beyond Fine-tuning: Few-Sample Sentence Embedding Transfer
Venue: arXiv cs.CL 2020
Year: 2020

Fine-tuning (FT) pre-trained sentence embedding models on small datasets has been shown to have limitations. In this paper we show that concatenating the embeddings from the pre-trained model with those from a simple sentence embedding model trained only on the target data, can improve over the performance of FT for few-sample tasks. To this end, a linear classifier is trained on the combined embeddings, either by freezing the embedding model weights or training the classifier and embedding models end-to-end. We perform evaluation on seven small datasets from NLP tasks and show that our approach with end-to-end training outperforms FT with negligible computational overhead. Further, we also show that sophisticated combination techniques like CCA and KCCA do not work as well in practice as concatenation. We provide theoretical analysis to explain this empirical observation.
Longformer: The Long-Document Transformer
Venue: arXiv cs.CL 2020
Year: 2020

Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly with sequence length, making it easy to process documents of thousands of tokens or longer. Longformer's attention mechanism is a drop-in replacement for the standard self-attention and combines a local windowed attention with a task motivated global attention. Following prior work on long-sequence transformers, we evaluate Longformer on character-level language modeling and achieve state-of-the-art results on text8 and enwik8. In contrast to most prior work, we also pretrain Longformer and finetune it on a variety of downstream tasks. Our pretrained Longformer consistently outperforms RoBERTa on long document tasks and sets new state-of-the-art results on WikiHop and TriviaQA. We finally introduce the Longformer-Encoder-Decoder (LED), a Longformer variant for supporting long document generative sequence-to-sequence tasks, and demonstrate its effectiveness on the arXiv summarization dataset.
Modeling and Mitigating Errors in Belief Propagation for Distributed Detection
Venue: arXiv cs.IT 2020
Year: 2020

We study the behavior of the belief-propagation (BP) algorithm affected by erroneous data exchange in a wireless sensor network (WSN). The WSN conducts a distributed binary hypothesis test where the joint statistical behavior of the sensor observations is modeled by a Markov random field whose parameters are used to build the BP messages exchanged between the sensing nodes. Through linearization of the BP message-update rule, we analyze the behavior of the resulting erroneous decision variables and derive closed-form relationships that describe the impact of stochastic errors on the performance of the BP algorithm. We then develop a decentralized distributed optimization framework to enhance the system performance by mitigating the impact of errors via a distributed linear data-fusion scheme. Finally, we compare the results of the proposed analysis with the existing works and visualize, via computer simulations, the performance gain obtained by the proposed optimization.
Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review
Venue: arXiv cs.CV 2020
Year: 2020

Autonomous vehicles were experiencing rapid development in the past few years. However, achieving full autonomy is not a trivial task, due to the nature of the complex and dynamic driving environment. Therefore, autonomous vehicles are equipped with a suite of different sensors to ensure robust, accurate environmental perception. In particular, the camera-LiDAR fusion is becoming an emerging research theme. However, so far there has been no critical review that focuses on deep-learning-based camera-LiDAR fusion methods. To bridge this gap and motivate future research, this paper devotes to review recent deep-learning-based data fusion approaches that leverage both image and point cloud. This review gives a brief overview of deep learning on image and point cloud data processing. Followed by in-depth reviews of camera-LiDAR fusion methods in depth completion, object detection, semantic segmentation, tracking and online cross-sensor calibration, which are organized based on their respective fusion levels. Furthermore, we compare these methods on publicly available datasets. Finally, we identified gaps and over-looked challenges between current academic researches and real-world applications. Based on these observations, we provide our insights and point out promising research directions.
Sequence Model Design for Code Completion in the Modern IDE
Venue: arXiv cs.SE 2020
Year: 2020

Code completion plays a prominent role in modern integrated development environments (IDEs). Machine learning has become ubiquitous in analogous natural language writing and search software, surfacing more relevant autocompletions and search suggestions in fewer keystrokes. Prior research has reported training high-accuracy, deep neural networks for modeling source code, but little attention has been given to the practical constraints imposed by interactive developer tools. In particular, neural language models for source code modeling like the one described in Maybe Deep Neural Networks are the Best Choice for Modeling Source Code are framed around code completion, but only report accuracy of next-token prediction. However, in order for a language model (LM) to work well within real-world code completion systems, it must also always make suggestions that produce valid code that typechecks to support code completion's role in correctness-checking; return instantaneous results to help programmers code more efficiently in fewer keystrokes; and be small enough to fit comfortably on disk and in memory on developer workstations, since virtually all modern IDEs run locally and support offline usage. To meet these additional requirements, we propose a novel design for predicting top-k next tokens that combines static analysis' ability to enumerate all valid keywords and in-scope identifiers with the ability of a language model to place a probability distribution over them. Our model mixes character-level input representation with token output to represent out-of-vocabulary (OOV) tokens meaningfully and minimize prediction latency. OOV tokens can be predicted through detection of local repetition common in software. This design achieves state-of-art accuracy in source code modeling and fits the constraints imposed by real-world code completion implementations in modern IDEs.
CONTAIN: Privacy-oriented Contact Tracing Protocols for Epidemics
Venue: arXiv cs.CR 2020
Year: 2020

Pandemic and epidemic diseases such as CoVID-19, SARS-CoV2, and Ebola have spread to multiple countries and infected thousands of people. Such diseases spread mainly through person-to-person contacts. Health care authorities recommend contact tracing procedures to prevent the spread to a vast population. Although several mobile applications have been developed to trace contacts, they typically require collection of privacy-intrusive information such as GPS locations, and the logging of privacy-sensitive data on a third party server, or require additional infrastructure such as WiFi APs with known locations. In this paper, we introduce CONTAIN, a privacy-oriented mobile contact tracing application that does not rely on GPS or any other form of infrastructure-based location sensing, nor the continuous logging of any other personally identifiable information on a server. The goal of CONTAIN is to allow users to determine with complete privacy if they have been within a short distance, specifically, Bluetooth wireless range, of someone that is infected, and potentially also when. We identify and prove the privacy guarantees provided by our approach. Our simulation study utilizing an empirical trace dataset (Asturies) involving 100 mobile devices and around 60000 records shows that users can maximize their possibility of identifying if they were near an infected user by turning on the app during active times.
Cooperative relaying in a SWIPT network:Asymptotic analysis using extreme value theory for non-identically distributed RVs
Venue: arXiv cs.IT 2020
Year: 2020

This paper derives the distribution of the maximum end-to-end (e2e) signal to noise ratio (SNR) in an opportunistic relay selection based cooperative relaying (CR) network having large number of non-identical relay links between the source and destination node. The source node is assumed to be simultaneous wireless information and power transfer (SWIPT) enabled and the relays are capable of both time splitting (TS) and power splitting (PS) based energy harvesting (EH). Contrary to the majority of literature in communication, which uses extreme value theory (EVT) to derive the statistics of extremes of sequences of independent and identically distributed (i.i.d.) random variables (RVs), we demonstrate how tools from EVT can be used to derive the asymptotic statistics of sequences of independent and non-identically distributed (i.n.i.d.) normalised SNR RVs and hence characterise the distribution of the maximum e2e SNR RV. Using these results we derive the expressions for ergodic and outage capacities at the destination node. Finally, we present the utility of the asymptotic results for deciding the optimum TS and PS factors of the hybrid EH relays that (i) minimise outage probability and (ii) maximise ergodic capacity at the destination. Furthermore, we demonstrate how stochastic ordering results can be utilised for simplifying these optimisation problems.
Improving Semantic Segmentation through Spatio-Temporal Consistency Learned from Videos
Venue: arXiv cs.CV 2020
Year: 2020

We leverage unsupervised learning of depth, egomotion, and camera intrinsics to improve the performance of single-image semantic segmentation, by enforcing 3D-geometric and temporal consistency of segmentation masks across video frames. The predicted depth, egomotion, and camera intrinsics are used to provide an additional supervision signal to the segmentation model, significantly enhancing its quality, or, alternatively, reducing the number of labels the segmentation model needs. Our experiments were performed on the ScanNet dataset.