OpenRaiser/Intern-Atlas huggingface dataset:
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders
Venue: arXiv cs.CL 2020
Year: 2020

A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense. The encoders are jointly optimized in the same representation space, so that sense disambiguation can be performed by finding the nearest sense embedding for each target word embedding. Our system outperforms previous state-of-the-art models on English all-words WSD; these gains predominantly come from improved performance on rare senses, leading to a 31.1% error reduction on less frequent senses over prior work. This demonstrates that rare senses can be more effectively disambiguated by modeling their definitions.
Approaches and Applications of Early Classification of Time Series: A Review
Venue: arXiv cs.LG 2020
Year: 2020

Early classification of time series has been extensively studied for minimizing class prediction delay in time-sensitive applications such as healthcare and finance. A primary task of an early classification approach is to classify an incomplete time series as soon as possible with some desired level of accuracy. Recent years have witnessed several approaches for early classification of time series. As most of the approaches have solved the early classification problem with different aspects, it becomes very important to make a thorough review of the existing solutions to know the current status of the area. These solutions have demonstrated reasonable performance in a wide range of applications including human activity recognition, gene expression based health diagnostic, industrial monitoring, and so on. In this paper, we present a systematic review of current literature on early classification approaches for both univariate and multivariate time series. We divide various existing approaches into four exclusive categories based on their proposed solution strategies. The four categories include prefix based, shapelet based, model based, and miscellaneous approaches. The authors also discuss the applications of early classification in many areas including industrial monitoring, intelligent transportation, and medical. Finally, we provide a quick summary of the current literature with future research directions.
Piveau: A Large-scale Open Data Management Platform based on Semantic Web Technologies
Venue: arXiv cs.IR 2020
Year: 2020

The publication and (re)utilization of Open Data is still facing multiple barriers on technical, organizational and legal levels. This includes limitations in interfaces, search capabilities, provision of quality information and the lack of definite standards and implementation guidelines. Many Semantic Web specifications and technologies are specifically designed to address the publication of data on the web. In addition, many official publication bodies encourage and foster the development of Open Data standards based on Semantic Web principles. However, no existing solution for managing Open Data takes full advantage of these possibilities and benfits. In this paper, we present our solution "Piveau", a fully-fledged Open Data management solution, based on Semantic Web technologies. It harnesses a variety of standards, like RDF, DCAT, DQV, and SKOS, to overcome the barriers in Open Data publication. The solution puts a strong focus on assuring data quality and scalability. We give a detailed description of the underlying, highly scalable, service-oriented architecture, how we integrated the aforementioned standards, and used a triplestore as our primary database. We have evaluated our work in a comprehensive feature comparison to established solutions and through a practical application in a production environment, the European Data Portal. Our solution is available as Open Source.
Scheduling and Dynamic Pilot Allocation For Massive MIMO with Varying Traffic
Venue: arXiv cs.NI 2020
Year: 2020

In this paper, we consider the problem of joint user scheduling and dynamic pilot allocation in a Time-Division Duplex (TDD) based Massive MIMO network under varying traffic condition. One of the main problems with Massive MIMO systems is that the number of available orthogonal pilot signals is limited, and the dynamic allocation of these signals to different users is crucially needed to utilize the full benefit of these systems. In addition, pilot signals are radio resource control (RRC) configured in practice, and hence the frequent reconfiguration causes high signaling overhead and is costly. Using Lyapunov optimization framework, we develop an optimal algorithm that first assigns pilots dynamically based on queue sizes and the channel conditions of users as well as the reconfiguration cost at large time-scale. Then, it schedules users on a small-time scale. Numerical results show the efficacy of our algorithm and demonstrate that pilots do not need to be configured frequently at the expense of increased queue delay.
Testing the Robustness of AutoML Systems
Venue: arXiv cs.LG 2020
Year: 2020

Automated machine learning (AutoML) systems aim at finding the best machine learning (ML) pipeline that automatically matches the task and data at hand. We investigate the robustness of machine learning pipelines generated with three AutoML systems, TPOT, H2O, and AutoKeras. In particular, we study the influence of dirty data on accuracy, and consider how using dirty training data may help create more robust solutions. Furthermore, we also analyze how the structure of the generated pipelines differs in different cases.
Dual-Sampling Attention Network for Diagnosis of COVID-19 from Community Acquired Pneumonia
Venue: arXiv cs.CV 2020
Year: 2020

The coronavirus disease (COVID-19) is rapidly spreading all over the world, and has infected more than 1,436,000 people in more than 200 countries and territories as of April 9, 2020. Detecting COVID-19 at early stage is essential to deliver proper healthcare to the patients and also to protect the uninfected population. To this end, we develop a dual-sampling attention network to automatically diagnose COVID- 19 from the community acquired pneumonia (CAP) in chest computed tomography (CT). In particular, we propose a novel online attention module with a 3D convolutional network (CNN) to focus on the infection regions in lungs when making decisions of diagnoses. Note that there exists imbalanced distribution of the sizes of the infection regions between COVID-19 and CAP, partially due to fast progress of COVID-19 after symptom onset. Therefore, we develop a dual-sampling strategy to mitigate the imbalanced learning. Our method is evaluated (to our best knowledge) upon the largest multi-center CT data for COVID-19 from 8 hospitals. In the training-validation stage, we collect 2186 CT scans from 1588 patients for a 5-fold cross-validation. In the testing stage, we employ another independent large-scale testing dataset including 2796 CT scans from 2057 patients. Results show that our algorithm can identify the COVID-19 images with the area under the receiver operating characteristic curve (AUC) value of 0.944, accuracy of 87.5%, sensitivity of 86.9%, specificity of 90.1%, and F1-score of 82.0%. With this performance, the proposed algorithm could potentially aid radiologists with COVID-19 diagnosis from CAP, especially in the early stage of the COVID-19 outbreak.
Drosophila-Inspired 3D Moving Object Detection Based on Point Clouds
Venue: arXiv cs.CV 2020
Year: 2020

3D moving object detection is one of the most critical tasks in dynamic scene analysis. In this paper, we propose a novel Drosophila-inspired 3D moving object detection method using Lidar sensors. According to the theory of elementary motion detector, we have developed a motion detector based on the shallow visual neural pathway of Drosophila. This detector is sensitive to the movement of objects and can well suppress background noise. Designing neural circuits with different connection modes, the approach searches for motion areas in a coarse-to-fine fashion and extracts point clouds of each motion area to form moving object proposals. An improved 3D object detection network is then used to estimate the point clouds of each proposal and efficiently generates the 3D bounding boxes and the object categories. We evaluate the proposed approach on the widely-used KITTI benchmark, and state-of-the-art performance was obtained by using the proposed approach on the task of motion detection.
ProbaNet: Proposal-balanced Network for Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Candidate object proposals generated by object detectors based on convolutional neural network (CNN) encounter easy-hard samples imbalance problem, which can affect overall performance. In this study, we propose a Proposal-balanced Network (ProbaNet) for alleviating the imbalance problem. Firstly, ProbaNet increases the probability of choosing hard samples for training by discarding easy samples through threshold truncation. Secondly, ProbaNet emphasizes foreground proposals by increasing their weights. To evaluate the effectiveness of ProbaNet, we train models based on different benchmarks. Mean Average Precision (mAP) of the model using ProbaNet achieves 1.2$\%$ higher than the baseline on PASCAL VOC 2007. Furthermore, it is compatible with existing two-stage detectors and offers a very small amount of additional computational cost.
An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Venue: arXiv cs.CL 2020
Year: 2020

Multi-task learning (MTL) has achieved remarkable success in natural language processing applications. In this work, we study a multi-task learning model with multiple decoders on varieties of biomedical and clinical natural language processing tasks such as text similarity, relation extraction, named entity recognition, and text inference. Our empirical results demonstrate that the MTL fine-tuned models outperform state-of-the-art transformer models (e.g., BERT and its variants) by 2.0% and 1.3% in biomedical and clinical domains, respectively. Pairwise MTL further demonstrates more details about which tasks can improve or decrease others. This is particularly helpful in the context that researchers are in the hassle of choosing a suitable model for new problems. The code and models are publicly available at https://github.com/ncbi-nlp/bluebert
Hierarchical Bayesian Approach for Improving Weights for Solving Multi-Objective Route Optimization Problem
Venue: arXiv cs.AI 2020
Year: 2020

The weighted sum method is a simple and widely used technique that scalarizes multiple conflicting objectives into a single objective function. It suffers from the problem of determining the appropriate weights corresponding to the objectives. This paper proposes a novel Hierarchical Bayesian model based on Multinomial distribution and Dirichlet prior to refine the weights for solving such multi-objective route optimization problems. The model and methodologies revolve around data obtained from a small scale pilot survey. The method aims at improving the existing methods of weight determination in the field of Intelligent Transport Systems as data driven choice of weights through appropriate probabilistic modelling ensures, on an average, much reliable results than non-probabilistic techniques. Application of this model and methodologies to simulated as well as real data sets revealed quite encouraging performances with respect to stabilizing the estimates of weights.
Sparktope: linear programs from algorithms
Venue: arXiv cs.DS 2020
Year: 2020

In a recent paper Avis, Bremner, Tiwary and Watanabe gave a method for constructing linear programs (LPs) based on algorithms written in a simple programming language called Sparks. If an algorithm produces the solution $x$ to a problem in polynomial time and space then the LP constructed is also of polynomial size and its optimum solution contains $x$ as well as a complete execution trace of the algorithm. Their method led us to the construction of a compiler called Sparktope which we describe in this paper. This compiler allows one to generate polynomial sized LPs for problems in P that have exponential extension complexity, such as matching problems in non-bipartite graphs. In this paper we describe Sparktope, the language Sparks, and the assembler instructions and LP constraints it produces. This is followed by two concrete examples, the makespan problem and the problem of testing if a matching in a graph is maximum, both of which are known to have exponential extension complexity. Computational results are given. In discussing these examples we make use of visualization techniques included in Sparktope that may be of independent interest. The extremely large linear programs produced by the compiler appear to be quite challenging to solve using currently available software. Since the optimum LP solutions can be computed independently they may be useful as benchmarks. Further enhancements of the compiler and its application are also discussed.
The computerization of archaeology: survey on AI techniques
Venue: arXiv cs.AI 2020
Year: 2020

This paper analyses the application of artificial intelligence techniques to various areas of archaeology and more specifically: a) The use of software tools as a creative stimulus for the organization of exhibitions; the use of humanoid robots and holographic displays as guides that interact and involve museum visitors; b) The analysis of methods for the classification of fragments found in archaeological excavations and for the reconstruction of ceramics, with the recomposition of the parts of text missing from historical documents and epigraphs; c) The cataloguing and study of human remains to understand the social and historical context of belonging with the demonstration of the effectiveness of the AI techniques used; d) The detection of particularly difficult terrestrial archaeological sites with the analysis of the architectures of the Artificial Neural Networks most suitable for solving the problems presented by the site; the design of a study for the exploration of marine archaeological sites, located at depths that cannot be reached by man, through the construction of a freely explorable 3D version.
Stochastic Bottleneck: Rateless Auto-Encoder for Flexible Dimensionality Reduction
Venue: arXiv cs.LG 2020
Year: 2020

We propose a new concept of rateless auto-encoders (RL-AEs) that enable a flexible latent dimensionality, which can be seamlessly adjusted for varying distortion and dimensionality requirements. In the proposed RL-AEs, instead of a deterministic bottleneck architecture, we use an over-complete representation that is stochastically regularized with weighted dropouts, in a manner analogous to sparse AE (SAE). Unlike SAEs, our RL-AEs employ monotonically increasing dropout rates across the latent representation nodes such that the latent variables become sorted by importance like in principal component analysis (PCA). This is motivated by the rateless property of conventional PCA, where the least important principal components can be discarded to realize variable rate dimensionality reduction that gracefully degrades the distortion. In contrast, since the latent variables of conventional AEs are equally important for data reconstruction, they cannot be simply discarded to further reduce the dimensionality after the AE model is trained. Our proposed stochastic bottleneck framework enables seamless rate adaptation with high reconstruction performance, without requiring predetermined latent dimensionality at training. We experimentally demonstrate that the proposed RL-AEs can achieve variable dimensionality reduction while achieving low distortion compared to conventional AEs.
Pay as You Go: A Generic Crypto Tolling Architecture
Venue: arXiv cs.CR 2020
Year: 2020

The imminent pervasive adoption of vehicular communication, based on dedicated short-range technology (ETSI ITS G5 or IEEE WAVE), 5G, or both, will foster a richer service ecosystem for vehicular applications. The appearance of new cryptography based solutions envisaging digital identity and currency exchange are set to stem new approaches for existing and future challenges. This paper presents a novel tolling architecture that harnesses the availability of 5G C-V2X connectivity for open road tolling using smartphones, IOTA as the digital currency and Hyperledger Indy for identity validation. An experimental feasibility analysis is used to validate the proposed architecture for secure, private and convenient electronic toll payment.
Mood of India During Covid-19 -- An Interactive Web Portal Based on Emotion Analysis of Twitter Data
Venue: arXiv cs.CY 2020
Year: 2020

The severe outbreak of Covid-19 pandemic has affected many countries across the world, and disrupted the day to day activities of many people. During such outbreaks, understanding the emotional state of citizens of a country could be of interest to various organizations to carry out tasks and to take necessary measures. Several studies have been performed on data available on various social media platforms and websites to understand the emotions of people against many events, inclusive of Covid-19, across the world. Twitter and other social media platforms have been bridging the gap between the citizens and government in various countries and are of more prominence in India. Sentiment Analysis of posts on twitter is observed to accurately reveal the sentiments. Analysing real time posts on twitter in India during Covid-19, could help in identifying the mood of the nation. However, most of the existing studies related to Covid-19, on twitter and other social media platforms are performed on data posted during a specific interval. We are not aware of any research that identifies emotional state of India on a daily basis. Hence, we present a web portal that aims to display mood of India during Covid-19, based on real time twitter data. This portal also enables users to select date range, specific date and state in India to display mood of people belonging to the specified region, on the specified date or during the specified date range. Also, the number of Covid-19 cases and mood of people at specific cities and states on specific dates is visualized on the country map. As of May 6 2020, the web portal has about 194370 tweets, and each of these tweets are classified into seven categories that include six basic emotions and a neutral category. A list of Trigger Events are also specified, to allow users to view the mood of India on specific events happening in the country during Covid-19.
Causality in Static Models as an Initial Constraint on the Chronology of Events in System Behavior
Venue: arXiv cs.SE 2020
Year: 2020

This paper analyzes the notion of causality in a conceptual model, mainly as applied in software engineering. Conceptual system modeling can be considered a three-level process that begins with building a static structural description to develop a dynamic model that will identify events used to specify the chronology of events. In this context, the model involves a representation of a portion of reality, based on ontology of different kinds of things and their basic relations to each other. Relations are defined in terms of their participating entities. This paper concerns relations between events, specifically causal relations among events in modeling. We examine causality in many fields of study to understand its role in modeling. The problem is that, according to many researchers, causation is so inextricably bound up in misleading associations that it is hard to define and is shrouded in mystery, controversy, and caution. We study and clarify the notion of causality through several examples, utilizing an event definition as a time thing/machine in a new conceptual modeling methodology. In conclusion, we claim that the purpose of causal relations in a system s static description is to constrain the system s behavior and thus exclude some possible chronologies of events.
Computing-in-Memory for Performance and Energy Efficient Homomorphic Encryption
Venue: arXiv cs.CR 2020
Year: 2020

Homomorphic encryption (HE) allows direct computations on encrypted data. Despite numerous research efforts, the practicality of HE schemes remains to be demonstrated. In this regard, the enormous size of ciphertexts involved in HE computations degrades computational efficiency. Near-memory Processing (NMP) and Computing-in-memory (CiM) - paradigms where computation is done within the memory boundaries - represent architectural solutions for reducing latency and energy associated with data transfers in data-intensive applications such as HE. This paper introduces CiM-HE, a Computing-in-memory (CiM) architecture that can support operations for the B/FV scheme, a somewhat homomorphic encryption scheme for general computation. CiM-HE hardware consists of customized peripherals such as sense amplifiers, adders, bit-shifters, and sequencing circuits. The peripherals are based on CMOS technology, and could support computations with memory cells of different technologies. Circuit-level simulations are used to evaluate our CiM-HE framework assuming a 6T-SRAM memory. We compare our CiM-HE implementation against (i) two optimized CPU HE implementations, and (ii) an FPGA-based HE accelerator implementation. When compared to a CPU solution, CiM-HE obtains speedups between 4.6x and 9.1x, and energy savings between 266.4x and 532.8x for homomorphic multiplications (the most expensive HE operation). Also, a set of four end-to-end tasks, i.e., mean, variance, linear regression, and inference are up to 1.1x, 7.7x, 7.1x, and 7.5x faster (and 301.1x, 404.6x, 532.3x, and 532.8x more energy efficient). Compared to CPU-based HE in a previous work, CiM-HE obtain 14.3x speed-up and >2600x energy savings. Finally, our design offers 2.2x speed-up with 88.1x energy savings compared to a state-of-the-art FPGA-based accelerator.
Evaluating text coherence based on the graph of the consistency of phrases to identify symptoms of schizophrenia
Venue: arXiv cs.CL 2020
Year: 2020

Different state-of-the-art methods of the detection of schizophrenia symptoms based on the estimation of text coherence have been analyzed. The analysis of a text at the level of phrases has been suggested. The method based on the graph of the consistency of phrases has been proposed to evaluate the semantic coherence and the cohesion of a text. The semantic coherence, cohesion, and other linguistic features (lexical diversity, lexical density) have been taken into account to form feature vectors for the training of a model-classifier. The training of the classifier has been performed on the set of English-language interviews. According to the retrieved results, the impact of each feature on the output of the model has been analyzed. The results obtained can indicate that the proposed method based on the graph of the consistency of phrases may be used in the different tasks of the detection of mental illness.
Weakly-Supervised Neural Response Selection from an Ensemble of Task-Specialised Dialogue Agents
Venue: arXiv cs.CL 2020
Year: 2020

Dialogue engines that incorporate different types of agents to converse with humans are popular. However, conversations are dynamic in the sense that a selected response will change the conversation on-the-fly, influencing the subsequent utterances in the conversation, which makes the response selection a challenging problem. We model the problem of selecting the best response from a set of responses generated by a heterogeneous set of dialogue agents by taking into account the conversational history, and propose a \emph{Neural Response Selection} method. The proposed method is trained to predict a coherent set of responses within a single conversation, considering its own predictions via a curriculum training mechanism. Our experimental results show that the proposed method can accurately select the most appropriate responses, thereby significantly improving the user experience in dialogue systems.