OpenRaiser/Intern-Atlas huggingface dataset:
Deep Learning based Frameworks for Handling Imbalance in DGA, Email, and URL Data Analysis
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning is a state of the art method for a lot of applications. The main issue is that most of the real-time data is highly imbalanced in nature. In order to avoid bias in training, cost-sensitive approach can be used. In this paper, we propose cost-sensitive deep learning based frameworks and the performance of the frameworks is evaluated on three different Cyber Security use cases which are Domain Generation Algorithm (DGA), Electronic mail (Email), and Uniform Resource Locator (URL). Various experiments were performed using cost-insensitive as well as cost-sensitive methods and parameters for both of these methods are set based on hyperparameter tuning. In all experiments, the cost-sensitive deep learning methods performed better than the cost-insensitive approaches. This is mainly due to the reason that cost-sensitive approach gives importance to the classes which have a very less number of samples during training and this helps to learn all the classes in a more efficient manner.
Learning Unsplit-field-based PML for the FDTD Method by Deep Differentiable Forest
Venue: arXiv cs.LG 2020
Year: 2020

Alternative unsplit-filed-based absorbing boundary condition (ABC) computation approach for the finite-difference time-domain (FDTD) is efficiently proposed based on the deep differentiable forest. The deep differentiable forest (DDF) model is introduced to replace the conventional perfectly matched layer (PML) ABC during the computation process of FDTD. The field component data on the interface of traditional PML are adopted to train the DDF-based PML model. DDF has the advantages of both trees and neural networks. Its tree structure is easy to use and explain for the numerical PML data. It has full differentiability like neural networks. DDF could be trained by powerful techniques from deep learning. So compared to the traditional PML implementation, the proposed method can greatly reduce the size of FDTD physical domain and the calculation complexity of FDTD due to the novel model which only involves the one-cell thickness of boundary layer. Numerical simulations have been carried out to benchmark the performance of the proposed approach. Numerical results illustrate that the proposed method can not only easily replace the traditional PML, but also be integrated into the FDTD computation process with satisfactory numerical accuracy and compatibility to the FDTD.
CSRN: Collaborative Sequential Recommendation Networks for News Retrieval
Venue: arXiv cs.IR 2020
Year: 2020

Nowadays, news apps have taken over the popularity of paper-based media, providing a great opportunity for personalization. Recurrent Neural Network (RNN)-based sequential recommendation is a popular approach that utilizes users' recent browsing history to predict future items. This approach is limited that it does not consider the societal influences of news consumption, i.e., users may follow popular topics that are constantly changing, while certain hot topics might be spreading only among specific groups of people. Such societal impact is difficult to predict given only users' own reading histories. On the other hand, the traditional User-based Collaborative Filtering (UserCF) makes recommendations based on the interests of the "neighbors", which provides the possibility to supplement the weaknesses of RNN-based methods. However, conventional UserCF only uses a single similarity metric to model the relationships between users, which is too coarse-grained and thus limits the performance. In this paper, we propose a framework of deep neural networks to integrate the RNN-based sequential recommendations and the key ideas from UserCF, to develop Collaborative Sequential Recommendation Networks (CSRNs). Firstly, we build a directed co-reading network of users, to capture the fine-grained topic-specific similarities between users in a vector space. Then, the CSRN model encodes users with RNNs, and learns to attend to neighbors and summarize what news they are reading at the moment. Finally, news articles are recommended according to both the user's own state and the summarized state of the neighbors. Experiments on two public datasets show that the proposed model outperforms the state-of-the-art approaches significantly.
Policy Gradient using Weak Derivatives for Reinforcement Learning
Venue: arXiv cs.LG 2020
Year: 2020

This paper considers policy search in continuous state-action reinforcement learning problems. Typically, one computes search directions using a classic expression for the policy gradient called the Policy Gradient Theorem, which decomposes the gradient of the value function into two factors: the score function and the Q-function. This paper presents four results:(i) an alternative policy gradient theorem using weak (measure-valued) derivatives instead of score-function is established; (ii) the stochastic gradient estimates thus derived are shown to be unbiased and to yield algorithms that converge almost surely to stationary points of the non-convex value function of the reinforcement learning problem; (iii) the sample complexity of the algorithm is derived and is shown to be $O(1/\sqrt(k))$; (iv) finally, the expected variance of the gradient estimates obtained using weak derivatives is shown to be lower than those obtained using the popular score-function approach. Experiments on OpenAI gym pendulum environment show superior performance of the proposed algorithm.
Brain Interface Based Wheel Chair Control System for Handicap -- An advance and viable approach
Venue: arXiv cs.HC 2020
Year: 2020

This paper presents advancement towards making an efficient and viable wheel chair control system based on brain computer interface via electro-oculogram (EOG) signals. The system utilizes the movement of eye as the element of purpose for controlling the movement of the wheel chair. Skin-surface electrodes are placed over skin for the purpose of acquiring the electro-oculogram signal and with the help of differential amplifier the bio-potential is measured between the reference and the point of interest, afterwards these obtained low voltage pulses are amplified, then passed through a sallen-key filter for noise removal and smoothening. These pulses are then collected on to the micro-controller; based on these pulses motor is switched to move in either right or left direction. A prototype system was developed and tested. The system showed promising results. The test conducted showed 99.5% efficiency of movement in correct direction.
Latent regularization for feature selection using kernel methods in tumor classification
Venue: arXiv cs.LG 2020
Year: 2020

The transcriptomics of cancer tumors are characterized with tens of thousands of gene expression features. Patient prognosis or tumor stage can be assessed by machine learning techniques like supervised classification tasks given a gene expression profile. Feature selection is a useful approach to select the key genes which helps to classify tumors. In this work we propose a feature selection method based on Multiple Kernel Learning that results in a reduced subset of genes and a custom kernel that improves the classification performance when used in support vector classification. During the feature selection process this method performs a novel latent regularisation by relaxing the supervised target problem by introducing unsupervised structure obtained from the latent space learned by a non linear dimensionality reduction model. An improvement of the generalization capacity is obtained and assessed by the tumor classification performance on new unseen test samples when the classifier is trained with the features selected by the proposed method in comparison with other supervised feature selection approaches.
Secret Sharing based Secure Regressions with Applications
Venue: arXiv cs.LG 2020
Year: 2020

Nowadays, the utilization of the ever expanding amount of data has made a huge impact on web technologies while also causing various types of security concerns. On one hand, potential gains are highly anticipated if different organizations could somehow collaboratively share their data for technological improvements. On the other hand, data security concerns may arise for both data holders and data providers due to commercial or sociological concerns. To make a balance between technical improvements and security limitations, we implement secure and scalable protocols for multiple data holders to train linear regression and logistic regression models. We build our protocols based on the secret sharing scheme, which is scalable and efficient in applications. Moreover, our proposed paradigm can be generalized to any secure multiparty training scenarios where only matrix summation and matrix multiplications are used. We demonstrate our approach by experiments which shows the scalability and efficiency of our proposed protocols, and finally present its real-world applications.
An In-depth Walkthrough on Evolution of Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Neural Machine Translation (NMT) methodologies have burgeoned from using simple feed-forward architectures to the state of the art; viz. BERT model. The use cases of NMT models have been broadened from just language translations to conversational agents (chatbots), abstractive text summarization, image captioning, etc. which have proved to be a gem in their respective applications. This paper aims to study the major trends in Neural Machine Translation, the state of the art models in the domain and a high level comparison between them.
Deep Residual Correction Network for Partial Domain Adaptation
Venue: arXiv cs.CV 2020
Year: 2020

Deep domain adaptation methods have achieved appealing performance by learning transferable representations from a well-labeled source domain to a different but related unlabeled target domain. Most existing works assume source and target data share the identical label space, which is often difficult to be satisfied in many real-world applications. With the emergence of big data, there is a more practical scenario called partial domain adaptation, where we are always accessible to a more large-scale source domain while working on a relative small-scale target domain. In this case, the conventional domain adaptation assumption should be relaxed, and the target label space tends to be a subset of the source label space. Intuitively, reinforcing the positive effects of the most relevant source subclasses and reducing the negative impacts of irrelevant source subclasses are of vital importance to address partial domain adaptation challenge. This paper proposes an efficiently-implemented Deep Residual Correction Network (DRCN) by plugging one residual block into the source network along with the task-specific feature layer, which effectively enhances the adaptation from source to target and explicitly weakens the influence from the irrelevant source classes. Specifically, the plugged residual block, which consists of several fully-connected layers, could deepen basic network and boost its feature representation capability correspondingly. Moreover, we design a weighted class-wise domain alignment loss to couple two domains by matching the feature distributions of shared classes between source and target. Comprehensive experiments on partial, traditional and fine-grained cross-domain visual recognition demonstrate that DRCN is superior to the competitive deep domain adaptation approaches.
A Novel Application of Boolean Functions with High Algebraic Immunity in Minimal Codes
Venue: arXiv cs.IT 2020
Year: 2020

Boolean functions with high algebraic immunity are important cryptographic primitives in some stream ciphers. In this paper, two methodologies for constructing binary minimal codes from sets, Boolean functions and vectorial Boolean functions with high algebraic immunity are proposed. More precisely, a general construction of new minimal codes using minimal codes contained in Reed-Muller codes and sets without nonzero low degree annihilators is presented. The other construction allows us to yield minimal codes from certain subcodes of Reed-Muller codes and vectorial Boolean functions with high algebraic immunity. Via these general constructions, infinite families of minimal binary linear codes of dimension $m$ and length less than or equal to $m(m+1)/2$ are obtained. In addition, a lower bound on the minimum distance of the proposed minimal linear codes is established. Conjectures and open problems are also presented. The results of this paper show that Boolean functions with high algebraic immunity have nice applications in several fields such as symmetric cryptography, coding theory and secret sharing schemes.
Stacked Convolutional Deep Encoding Network for Video-Text Retrieval
Venue: arXiv cs.MM 2020
Year: 2020

Existing dominant approaches for cross-modal video-text retrieval task are to learn a joint embedding space to measure the cross-modal similarity. However, these methods rarely explore long-range dependency inside video frames or textual words leading to insufficient textual and visual details. In this paper, we propose a stacked convolutional deep encoding network for video-text retrieval task, which considers to simultaneously encode long-range and short-range dependency in the videos and texts. Specifically, a multi-scale dilated convolutional (MSDC) block within our approach is able to encode short-range temporal cues between video frames or text words by adopting different scales of kernel size and dilation size of convolutional layer. A stacked structure is designed to expand the receptive fields by repeatedly adopting the MSDC block, which further captures the long-range relations between these cues. Moreover, to obtain more robust textual representations, we fully utilize the powerful language model named Transformer in two stages: pretraining phrase and fine-tuning phrase. Extensive experiments on two different benchmark datasets (MSR-VTT, MSVD) show that our proposed method outperforms other state-of-the-art approaches.
Rephrasing visual questions by specifying the entropy of the answer distribution
Venue: arXiv cs.CV 2020
Year: 2020

Visual question answering (VQA) is a task of answering a visual question that is a pair of question and image. Some visual questions are ambiguous and some are clear, and it may be appropriate to change the ambiguity of questions from situation to situation. However, this issue has not been addressed by any prior work. We propose a novel task, rephrasing the questions by controlling the ambiguity of the questions. The ambiguity of a visual question is defined by the use of the entropy of the answer distribution predicted by a VQA model. The proposed model rephrases a source question given with an image so that the rephrased question has the ambiguity (or entropy) specified by users. We propose two learning strategies to train the proposed model with the VQA v2 dataset, which has no ambiguity information. We demonstrate the advantage of our approach that can control the ambiguity of the rephrased questions, and an interesting observation that it is harder to increase than to reduce ambiguity.
Engagement Patterns of Peer-to-Peer Interactions on Mental Health Platforms
Venue: arXiv cs.SI 2020
Year: 2020

Mental illness is a global health problem, but access to mental healthcare resources remain poor worldwide. Online peer-to-peer support platforms attempt to alleviate this fundamental gap by enabling those who struggle with mental illness to provide and receive social support from their peers. However, successful social support requires users to engage with each other and failures may have serious consequences for users in need. Our understanding of engagement patterns on mental health platforms is limited but critical to inform the role, limitations, and design of these platforms. Here, we present a large-scale analysis of engagement patterns of 35 million posts on two popular online mental health platforms, TalkLife and Reddit. Leveraging communication models in human-computer interaction and communication theory, we operationalize a set of four engagement indicators based on attention and interaction. We then propose a generative model to jointly model these indicators of engagement, the output of which is synthesized into a novel set of eleven distinct, interpretable patterns. We demonstrate that this framework of engagement patterns enables informative evaluations and analysis of online support platforms. Specifically, we find that mutual back-and-forth interactions are associated with significantly higher user retention rates on TalkLife. Such back-and-forth interactions, in turn, are associated with early response times and the sentiment of posts.
A New Dataset for Natural Language Inference from Code-mixed Conversations
Venue: arXiv cs.CL 2020
Year: 2020

Natural Language Inference (NLI) is the task of inferring the logical relationship, typically entailment or contradiction, between a premise and hypothesis. Code-mixing is the use of more than one language in the same conversation or utterance, and is prevalent in multilingual communities all over the world. In this paper, we present the first dataset for code-mixed NLI, in which both the premises and hypotheses are in code-mixed Hindi-English. We use data from Hindi movies (Bollywood) as premises, and crowd-source hypotheses from Hindi-English bilinguals. We conduct a pilot annotation study and describe the final annotation protocol based on observations from the pilot. Currently, the data collected consists of 400 premises in the form of code-mixed conversation snippets and 2240 code-mixed hypotheses. We conduct an extensive analysis to infer the linguistic phenomena commonly observed in the dataset obtained. We evaluate the dataset using a standard mBERT-based pipeline for NLI and report results.
Current Practices in the Information Collection for Enterprise Architecture Management
Venue: arXiv cs.OH 2020
Year: 2020

The digital transformation influences business models, processes, and enterprise IT landscape as a whole. Therefore, business-IT alignment is becoming more important than ever before. Enterprise architecture management (EAM) is designed to support and improve this business-IT alignment. The success of EAM crucially depends on the information available about a company's enterprise architecture, such as infrastructure components, applications, and business processes. This paper discusses the results of a qualitative expert survey with 26 experts in the field of EAM. The goal of this survey was to highlight current practices in the information collection for EAM and identify relevant information from enterprise-external data sources. The results provide a comprehensive overview of collected and utilized information in the industry, including an assessment of the relevance of such information. Furthermore, the results highlight challenges in practice and point out investments that organizations plan in the field of EAM.
Deep transfer learning for improving single-EEG arousal detection
Venue: arXiv cs.CV 2020
Year: 2020

Datasets in sleep science present challenges for machine learning algorithms due to differences in recording setups across clinics. We investigate two deep transfer learning strategies for overcoming the channel mismatch problem for cases where two datasets do not contain exactly the same setup leading to degraded performance in single-EEG models. Specifically, we train a baseline model on multivariate polysomnography data and subsequently replace the first two layers to prepare the architecture for single-channel electroencephalography data. Using a fine-tuning strategy, our model yields similar performance to the baseline model (F1=0.682 and F1=0.694, respectively), and was significantly better than a comparable single-channel model. Our results are promising for researchers working with small databases who wish to use deep learning models pre-trained on larger databases.
Automatically Assessing Quality of Online Health Articles
Venue: arXiv cs.CY 2020
Year: 2020

The information ecosystem today is overwhelmed by an unprecedented quantity of data on versatile topics are with varied quality. However, the quality of information disseminated in the field of medicine has been questioned as the negative health consequences of health misinformation can be life-threatening. There is currently no generic automated tool for evaluating the quality of online health information spanned over a broad range. To address this gap, in this paper, we applied a data mining approach to automatically assess the quality of online health articles based on 10 quality criteria. We have prepared a labeled dataset with 53012 features and applied different feature selection methods to identify the best feature subset with which our trained classifier achieved an accuracy of 84%-90% varied over 10 criteria. Our semantic analysis of features shows the underpinning associations between the selected features & assessment criteria and further rationalize our assessment approach. Our findings will help in identifying high-quality health articles and thus aiding users in shaping their opinion to make the right choice while picking health-related help from online.
One Model to Recognize Them All: Marginal Distillation from NER Models with Different Tag Sets
Venue: arXiv cs.CL 2020
Year: 2020

Named entity recognition (NER) is a fundamental component in the modern language understanding pipeline. Public NER resources such as annotated data and model services are available in many domains. However, given a particular downstream application, there is often no single NER resource that supports all the desired entity types, so users must leverage multiple resources with different tag sets. This paper presents a marginal distillation (MARDI) approach for training a unified NER model from resources with disjoint or heterogeneous tag sets. In contrast to recent works, MARDI merely requires access to pre-trained models rather than the original training datasets. This flexibility makes it easier to work with sensitive domains like healthcare and finance. Furthermore, our approach is general enough to integrate with different NER architectures, including local models (e.g., BiLSTM) and global models (e.g., CRF). Experiments on two benchmark datasets show that MARDI performs on par with a strong marginal CRF baseline, while being more flexible in the form of required NER resources. MARDI also sets a new state of the art on the progressive NER task. MARDI significantly outperforms the start-of-the-art model on the task of progressive NER.
Stochastic Multi-Agent-Based Model to Measure Community Resilience-Part 2: Simulation Results
Venue: arXiv cs.CY 2020
Year: 2020

In this paper we investigate the resiliency planning of interdependent electric power systems and emergency services. We investigate the effect of the level of empathy, cooperation, coordination, flexibility, and experience of individuals on their mental well-being. Furthermore, we explore the impact of the information that is provided by emergency services and the impact of the availability of electric energy on the physical, mental, and social well-being of individuals. For our simulations, we use a stochastic, multi-agent-based numerical framework that is reported in the companion paper for estimating the social well-being of a community when facing natural disasters such as hurricanes, floods, earthquakes, and tsunamis. The performance of the proposed method is assessed by measuring community resilience for a multitude of effects in the context of two case studies. These effects are analyzed for Gaussian social random characteristics. Each case study considers nine agents, namely, three areas of three communities each, yielding a total of six communities. The results show that a high level of cooperation can positively change individual behavior. In addition, the relationship among the individuals of a community is so vital that the society with less population and more empathy may be more resilient than the community with more population and less empathy.