OpenRaiser/Intern-Atlas huggingface dataset:
Proximity full-text searches of frequently occurring words with a response time guarantee
Venue: arXiv cs.IR 2020
Year: 2020

Full-text search engines are important tools for information retrieval. In a proximity full-text search, a document is relevant if it contains query terms near each other, especially if the query terms are frequently occurring words. For each word in the text, we use additional indexes to store information about nearby words at distances from the given word of less than or equal to MaxDistance, which is a parameter. A search algorithm for the case when the query consists of high-frequently used words is discussed. In addition, we present results of experiments with different values of MaxDistance to evaluate the search speed dependence on the value of MaxDistance. These results show that the average time of the query execution with our indexes is 94.7-45.9 times (depending on the value of MaxDistance) less than that with standard inverted files when queries that contain high-frequently occurring words are evaluated. This is a pre-print of a contribution published in Pinelas S., Kim A., Vlasov V. (eds) Mathematical Analysis With Applications. CONCORD-90 2018. Springer Proceedings in Mathematics & Statistics, vol 318, published by Springer, Cham. The final authenticated version is available online at: https://doi.org/10.1007/978-3-030-42176-2_37
Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification
Venue: arXiv cs.CL 2020
Year: 2020

Neural-based models have achieved outstanding performance on slot filling and intent classification, when fairly large in-domain training data are available. However, as new domains are frequently added, creating sizeable data is expensive. We show that lightweight augmentation, a set of augmentation methods involving word span and sentence level operations, alleviates data scarcity problems. Our experiments on limited data settings show that lightweight augmentation yields significant performance improvement on slot filling on the ATIS and SNIPS datasets, and achieves competitive performance with respect to more complex, state-of-the-art, augmentation approaches. Furthermore, lightweight augmentation is also beneficial when combined with pre-trained LM-based models, as it improves BERT-based joint intent and slot filling models.
Large-scale Neural Solvers for Partial Differential Equations
Venue: arXiv cs.LG 2020
Year: 2020

Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs. However, recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing. Scanning the parameters of the underlying model significantly increases the runtime as the simulations have to be cold-started for each parameter configuration. Machine Learning based surrogate models denote promising ways for learning complex relationship among input, parameter and solution. However, recent generative neural networks require lots of training data, i.e. full simulation runs making them costly. In contrast, we examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs) solely requiring initial/boundary values and validation points for training but no simulation data. The induced curse of dimensionality is approached by learning a domain decomposition that steers the number of neurons per unit volume and significantly improves runtime. Distributed training on large-scale cluster systems also promises great utilization of large quantities of GPUs which we assess by a comprehensive evaluation study. Finally, we discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
On the Battery Consumption of Mobile Browsers
Venue: arXiv cs.OH 2020
Year: 2020

Mobile web browsing has recently surpassed desktop browsing both in term of popularity and traffic. Following its desktop counterpart, the mobile browsers ecosystem has been growing from few browsers (Chrome, Firefox, and Safari) to a plethora of browsers, each with unique characteristics (battery friendly, privacy preserving, lightweight, etc.). In this paper, we introduce a browser benchmarking pipeline for Android browsers encompassing automation, in-depth experimentation, and result analysis. We tested 15 Android browsers, using Cappuccino a novel testing suite we built for third party Android applications. We perform a battery-centric analysis of such browsers and show that: 1) popular browsers tend also to consume the most, 2) adblocking produces significant battery savings (between 20 and 40% depending on the browser), and 3) dark mode offers an extra 10% battery savings on AMOLED screens. We exploit this observation to build AttentionDim, a screen dimming mechanism driven by browser events. Via integration with the Brave browser and 10 volunteers, we show potential battery savings up to 30%, on both devices with AMOLED and LCD screens.
Trade-off analysis of disruption-tolerant networking protocols
Venue: arXiv cs.NI 2020
Year: 2020

The objective of this analysis was to simulate the performance of three different ad-hoc protocols for disruption-tolerant networking (DTN) - i.e. the transfer of information through a network of nodes in contexts prone to signal interruption/signal degradation - and to perform a trade-off analysis that will yield a recommendation of the best course of action (COA) for a user of a space-based network to employ. This is important for space-based networks in particular since transmitting information over long distances - e.g. from directly from the initial node to the destination node - will result in the terminal signal being relatively weak, which is problematic in a high noise (disruption-prone) environment since it will likely result in packet degradation or loss unless signal power is increased to compensate. Given that changing signal power for each transmission is impractical when one's network is located in space, optimization of the signal route is the best means of ensuring the transmitted signal reaches its destination rapidly and with maximum fidelity. The recommended COA, as determined at the end of the analysis, is one that optimizes performance in a fashion that minimizes error during transmission and transmission time to the greatest extent possible. This analysis takes into consideration the relative value of transmission time and transmission error for a user of space-based communication systems (for instance, a scientific mission - the most likely type of user for a space-based relay network - would prioritize data integrity much higher than transmission time since even though a longer transmission time equals greater cost, a lower level of data integrity could result in the mission's scientific objective being compromised) and provides a recommendation accordingly.
Machine Learning Towards Enabling Spectrum-as-a-Service Dynamic Sharing
Venue: arXiv cs.NI 2020
Year: 2020

The growth in wireless broadband users, devices, and novel applications has led to a significant increase in the demand for new radio frequency spectrum. This is expected to grow even further given the projection that the global traffic per year will reach 4.8 zettabytes by 2022. Moreover, it is projected that the number of Internet users will reach 4.8 billion and the number of connected devices will be close 28.5 billion devices. However, due to the spectrum being mostly allocated and divided, providing more spectrum to expand existing services or offer new ones has become more challenging. To address this, spectrum sharing has been proposed as a potential solution to improve spectrum utilization efficiency. Adopting effective and efficient spectrum sharing mechanisms is in itself a challenging task given the multitude of levels and techniques that can be integrated to enable it. To that end, this paper provides an overview of the different spectrum sharing levels and techniques that have been proposed in the literature. Moreover, it discusses the potential of adopting dynamic sharing mechanisms by offering Spectrum-as-a-Service architecture. Furthermore, it describes the potential role of machine learning models in facilitating the automated and efficient dynamic sharing of the spectrum and offering Spectrum-as-a-Service.
An integrative smoothed particle hydrodynamics framework for modeling cardiac function
Venue: arXiv cs.CE 2020
Year: 2020

Mathematical modeling of cardiac function can provide augmented simulation-based diagnosis tool for complementing and extending human understanding of cardiac diseases which represent the most common cause of worldwide death. As the realistic starting-point for developing an unified meshless approach for total heart modeling, herein we propose an integrative smoothed particle hydrodynamics (SPH) framework for addressing the simulation of the principle aspects of cardiac function, including cardiac electrophysiology, passive mechanical response and electromechanical coupling. To that end, several algorithms, e.g., splitting reaction-by-reaction method combined with quasi-steady-state (QSS) solver , anisotropic SPH-diffusion discretization and total Lagrangian SPH formulation, are introduced and exploited for dealing with the fundamental challenges of developing integrative SPH framework for simulating cardiac function, namely, (i) the correct capturing of the stiff dynamics of the transmembrane potential and the gating variables , (ii) the stable predicting of the large deformations and the strongly anisotropic behavior of the myocardium, and (iii) the proper coupling of electrophysiology and tissue mechanics for electromechanical feedback. A set of numerical examples demonstrate the effectiveness and robustness of the present SPH framework, and render it a potential and powerful alternative that can augment current lines of total cardiac modeling and clinical applications.
Development and comparison of spectral algorithms for numerical modeling of the quasi-static mechanical behavior of inhomogeneous materials
Venue: arXiv cs.CE 2020
Year: 2020

In the current work, a number of algorithms are developed and compared for the numerical solution of periodic (quasi-static) linear elastic mechanical boundary-value problems (BVPs) based on two different discretizations of Fourier series. The first is standard and based on the trapezoidal approximation of the Fourier mode integral, resulting in trapezoidal discretization (TD) of the truncated Fourier series. Less standard is the second discretization based on piecewise-constant approximation of the Fourier mode integrand, yielding a piecewise-constant discretization (PCD) of this series. Employing these, fixed-point algorithms are formulated via Green-function preconditioning (GFP) and finite-difference discretization (of differential operators; FDD). In particular, in the context of PCD, this includes an algorithm based on the so-called "discrete Green operator" (DGO) recently introduced by Eloh et al. (2019), which employs GFP, but not FDD. For computational comparisons, the (classic) benchmark case of a cubic inclusion embedded in a matrix (e.g., Suquet, 1997; Willot, 2015) is employed. Both discontinuous and smooth transitions in elastic stiffness at the matrix-inclusion (MI) interface are considered. In the context of both TD and PCD, a number of GFP- and FDD-based algorithms are developed. Among these, one based on so-called averaged-forward-backward-differencing (AFB) is shown to result in the greatest improvement in convergence rate. As it turns out, AFB is equivalent to the "rotated scheme" (R) of Willot (2015) in the context of TD. In the context of PCD, comparison of the performance and convergence behavior of AFB/R- and DGO-based algorithms shows that the former is more efficient than the latter for larger phase contrasts.
Covid-Transformer: Detecting COVID-19 Trending Topics on Twitter Using Universal Sentence Encoder
Venue: arXiv cs.CL 2020
Year: 2020

The novel corona-virus disease (also known as COVID-19) has led to a pandemic, impacting more than 200 countries across the globe. With its global impact, COVID-19 has become a major concern of people almost everywhere, and therefore there are a large number of tweets coming out from every corner of the world, about COVID-19 related topics. In this work, we try to analyze the tweets and detect the trending topics and major concerns of people on Twitter, which can enable us to better understand the situation, and devise better planning. More specifically we propose a model based on the universal sentence encoder to detect the main topics of Tweets in recent months. We used universal sentence encoder in order to derive the semantic representation and the similarity of tweets. We then used the sentence similarity and their embeddings, and feed them to K-means clustering algorithm to group similar tweets (in semantic sense). After that, the cluster summary is obtained using a text summarization algorithm based on deep learning, which can uncover the underlying topics of each cluster. Through experimental results, we show that our model can detect very informative topics, by processing a large number of tweets on sentence level (which can preserve the overall meaning of the tweets). Since this framework has no restriction on specific data distribution, it can be used to detect trending topics from any other social media and any other context rather than COVID-19. Experimental results show superiority of our proposed approach to other baselines, including TF-IDF, and latent Dirichlet allocation (LDA).
Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we study bidirectional LSTM network for the task of text classification using both supervised and semi-supervised approaches. Several prior works have suggested that either complex pretraining schemes using unsupervised methods such as language modeling (Dai and Le 2015; Miyato, Dai, and Goodfellow 2016) or complicated models (Johnson and Zhang 2017) are necessary to achieve a high classification accuracy. However, we develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results compared with more complex approaches. Furthermore, in addition to cross-entropy loss, by using a combination of entropy minimization, adversarial, and virtual adversarial losses for both labeled and unlabeled data, we report state-of-the-art results for text classification task on several benchmark datasets. In particular, on the ACL-IMDB sentiment analysis and AG-News topic classification datasets, our method outperforms current approaches by a substantial margin. We also show the generality of the mixed objective function by improving the performance on relation extraction task.
Tunable Subnetwork Splitting for Model-parallelism of Neural Network Training
Venue: arXiv cs.LG 2020
Year: 2020

Alternating minimization methods have recently been proposed as alternatives to the gradient descent for deep neural network optimization. Alternating minimization methods can typically decompose a deep neural network into layerwise subproblems, which can then be optimized in parallel. Despite the significant parallelism, alternating minimization methods are rarely explored in training deep neural networks because of the severe accuracy degradation. In this paper, we analyze the reason and propose to achieve a compelling trade-off between parallelism and accuracy by a reformulation called Tunable Subnetwork Splitting Method (TSSM), which can tune the decomposition granularity of deep neural networks. Two methods gradient splitting Alternating Direction Method of Multipliers (gsADMM) and gradient splitting Alternating Minimization (gsAM) are proposed to solve the TSSM formulation. Experiments on five benchmark datasets show that our proposed TSSM can achieve significant speedup without observable loss of training accuracy. The code has been released at https://github.com/xianggebenben/TSSM.
Comparative Study of Language Models on Cross-Domain Data with Model Agnostic Explainability
Venue: arXiv cs.CL 2020
Year: 2020

With the recent influx of bidirectional contextualized transformer language models in the NLP, it becomes a necessity to have a systematic comparative study of these models on variety of datasets. Also, the performance of these language models has not been explored on non-GLUE datasets. The study presented in paper compares the state-of-the-art language models - BERT, ELECTRA and its derivatives which include RoBERTa, ALBERT and DistilBERT. We conducted experiments by finetuning these models for cross domain and disparate data and penned an in-depth analysis of model's performances. Moreover, an explainability of language models coherent with pretraining is presented which verifies the context capturing capabilities of these models through a model agnostic approach. The experimental results establish new state-of-the-art for Yelp 2013 rating classification task and Financial Phrasebank sentiment detection task with 69% accuracy and 88.2% accuracy respectively. Finally, the study conferred here can greatly assist industry researchers in choosing the language model effectively in terms of performance or compute efficiency.
Rule-Guided Graph Neural Networks for Recommender Systems
Venue: arXiv cs.LG 2020
Year: 2020

To alleviate the cold start problem caused by collaborative filtering in recommender systems, knowledge graphs (KGs) are increasingly employed by many methods as auxiliary resources. However, existing work incorporated with KGs cannot capture the explicit long-range semantics between users and items meanwhile consider various connectivity between items. In this paper, we propose RGRec, which combines rule learning and graph neural networks (GNNs) for recommendation. RGRec first maps items to corresponding entities in KGs and adds users as new entities. Then, it automatically learns rules to model the explicit long-range semantics, and captures the connectivity between entities by aggregation to better encode various information. We show the effectiveness of RGRec on three real-world datasets. Particularly, the combination of rule learning and GNNs achieves substantial improvement compared to methods only using either of them.
Adwords in a Panorama
Venue: arXiv cs.DS 2020
Year: 2020

Three decades ago, Karp, Vazirani, and Vazirani (STOC 1990) defined the online matching problem and gave an optimal $1-\frac{1}{e} \approx 0.632$-competitive algorithm. Fifteen years later, Mehta, Saberi, Vazirani, and Vazirani (FOCS 2005) introduced the first generalization called AdWords driven by online advertising and obtained the optimal $1-\frac{1}{e}$ competitive ratio in the special case of small bids. It has been open ever since whether there is an algorithm for general bids better than the $0.5$-competitive greedy algorithm. This paper presents a $0.5016$-competitive algorithm for AdWords, answering this open question on the positive end. The algorithm builds on several ingredients, including a combination of the online primal dual framework and the configuration linear program of matching problems recently explored by Huang and Zhang (STOC 2020), a novel formulation of AdWords which we call the panorama view, and a generalization of the online correlated selection by Fahrbach, Huang, Tao, and Zadimorghaddam (FOCS 2020) which we call the panoramic online correlated selection.
Sparse or Dense: A Comparative Study of Code-Domain NOMA Systems
Venue: arXiv cs.IT 2020
Year: 2020

This paper is focused on code-domain non-orthogonal multiple access (CD-NOMA), which is an emerging paradigm to support massive connectivity for future machine-type wireless networks. We take a comparative approach to study two types of overloaded CD-NOMA, i.e., sparse code multiple access (SCMA) and dense code multiple access (DCMA), which are distinctive from each other in terms of their codebooks having sparsity or not. By analysing their individual diversity orders (DO) in Rayleigh fading channels, it is found that DCMA can be designed to enjoy full DO which is equal to the maximum number of resource nodes in the system. This is in contrast to SCMA whose error rate suffers from limited DO equal to the codebook sparsity (i.e., the effective number of resource nodes occupied by each user). We conduct theoretical analysis for the codebook design criteria and propose to use generalized sphere decoder for DCMA detection. We numerically evaluate two types of multiple access schemes under "4x6" (i.e., six users communicate over four subcarriers) and "5x10" NOMA settings and reveal that DCMA gives rise to significantly improved error rate performance in Rayleigh fading channels, whilst having decoding complexity comparable to that of SCMA.
User documentation and training at Belle II
Venue: arXiv cs.CY 2020
Year: 2020

Belle II is a rapidly growing collaboration with members from one hundred and nineteen institutes spread around the globe. The software development team of the experiment, as well as the software users, are very much decentralised. Together with the active development of the software, such decentralisation makes the adoption of the latest software releases by users an essential, but quite challenging task. To ensure the relevance of the documentation, we adopted the policy of in-code documentation and configured a website that allows us to tie the documentation to given releases. To prevent tutorials from becoming outdated, we covered them by unit-tests. For the user support, we use a question and answer service that not only reduces repetition of the same questions but also turned out to be a place for discussions among the experts. A prototype of a metasearch engine for the different sources of documentation has been developed. For training of the new users, we organise centralised StarterKit workshops attached to the collaboration meetings. The materials of the workshops are later used for self-education and organisation of local training sessions.
MU-GAN: Facial Attribute Editing based on Multi-attention Mechanism
Venue: arXiv cs.CV 2020
Year: 2020

Facial attribute editing has mainly two objectives: 1) translating image from a source domain to a target one, and 2) only changing the facial regions related to a target attribute and preserving the attribute-excluding details. In this work, we propose a Multi-attention U-Net-based Generative Adversarial Network (MU-GAN). First, we replace a classic convolutional encoder-decoder with a symmetric U-Net-like structure in a generator, and then apply an additive attention mechanism to build attention-based U-Net connections for adaptively transferring encoder representations to complement a decoder with attribute-excluding detail and enhance attribute editing ability. Second, a self-attention mechanism is incorporated into convolutional layers for modeling long-range and multi-level dependencies across image regions. experimental results indicate that our method is capable of balancing attribute editing ability and details preservation ability, and can decouple the correlation among attributes. It outperforms the state-of-the-art methods in terms of attribute manipulation accuracy and image quality.