OpenRaiser/Intern-Atlas huggingface dataset:
The Naked Sun: Malicious Cooperation Between Benign-Looking Processes
Venue: arXiv cs.CR 2019
Year: 2019

Recent progress in machine learning has generated promising results in behavioral malware detection. Behavioral modeling identifies malicious processes via features derived by their runtime behavior. Behavioral features hold great promise as they are intrinsically related to the functioning of each malware, and are therefore considered difficult to evade. Indeed, while a significant amount of results exists on evasion of static malware features, evasion of dynamic features has seen limited work. This paper thoroughly examines the robustness of behavioral malware detectors to evasion, focusing particularly on anti-ransomware evasion. We choose ransomware as its behavior tends to differ significantly from that of benign processes, making it a low-hanging fruit for behavioral detection (and a difficult candidate for evasion). Our analysis identifies a set of novel attacks that distribute the overall malware workload across a small set of cooperating processes to avoid the generation of significant behavioral features. Our most effective attack decreases the accuracy of a state-of-the-art classifier from 98.6% to 0% using only 18 cooperating processes. Furthermore, we show our attacks to be effective against commercial ransomware detectors even in a black-box setting.
Map Enhanced Route Travel Time Prediction using Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Travel time estimation is a fundamental problem in transportation science with extensive literature. The study of these techniques has intensified due to availability of many publicly available large trip datasets. Recently developed deep learning based models have improved the generality and performance and have focused on estimating times for individual sub-trajectories and aggregating them to predict the travel time of the entire trajectory. However, these techniques ignore the road network information. In this work, we propose and study techniques for incorporating road networks along with historical trips' data into travel time prediction. We incorporate both node embeddings as well as road distance into the existing model. Experiments on large real-world benchmark datasets suggest improved performance, especially when the train data is small. As expected, the proposed method performs better than the baseline when there is a larger difference between road distance and Vincenty distance between start and end points.
Distributed MST: A Smoothed Analysis
Venue: arXiv cs.DS 2019
Year: 2019

We study smoothed analysis of distributed graph algorithms, focusing on the fundamental minimum spanning tree (MST) problem. With the goal of studying the time complexity of distributed MST as a function of the "perturbation" of the input graph, we posit a {\em smoothing model} that is parameterized by a smoothing parameter $0 \leq ε(n) \leq 1$ which controls the amount of {\em random} edges that can be added to an input graph $G$ per round. Informally, $ε(n)$ is the probability (typically a small function of $n$, e.g., $n^{-\frac{1}{4}}$) that a random edge can be added to a node per round. The added random edges, once they are added, can be used (only) for communication. We show upper and lower bounds on the time complexity of distributed MST in the above smoothing model. We present a distributed algorithm that, with high probability,\footnote{Throughout, with high probability (whp) means with probability at least $1 - n^{-c}$, for some fixed, positive constant $c$.} computes an MST and runs in $\tilde{O}(\min\{\frac{1}{\sqrt{ε(n)}} 2^{O(\sqrt{\log n})}, D + \sqrt{n}\})$ rounds\footnote{The notation $\tilde{O}$ hides a $\polylog(n)$ factor and $\tildeΩ$ hides a $\frac{1}{\polylog{(n)}}$ factor, where $n$ is the number of nodes of the graph.} where $ε$ is the smoothing parameter, $D$ is the network diameter and $n$ is the network size. To complement our upper bound, we also show a lower bound of $\tildeΩ(\min\{\frac{1}{\sqrt{ε(n)}}, D+\sqrt{n}\})$. We note that the upper and lower bounds essentially match except for a multiplicative $2^{O(\sqrt{\log n})} \polylog(n)$ factor. Our work can be considered as a first step in understanding the smoothed complexity of distributed graph algorithms.
Optimizing Semi-Stream CACHEJOIN for Near-Real-Time Data Warehousing
Venue: arXiv cs.DB 2019
Year: 2019

Streaming data join is a critical process in the field of near-real-time data warehousing. For this purpose, an adaptive semi-stream join algorithm called CACHEJOIN (Cache Join) focusing non-uniform stream data is provided in the literature. However, this algorithm cannot exploit the memory and CPU resources optimally and consequently it leaves its service rate suboptimal due to sequential execution of both of its phases, called stream-probing (SP) phase and disk-probing (DP) phase. By integrating the advantages of CACHEJOIN, in this paper we present two modifications in it. First is called P-CACHEJOIN (Parallel Cache Join) that enables the parallel processing of two phases in CACHEJOIN. This increases number of joined stream records and therefore improves throughput considerably. Second is called OP-CACHEJOIN (Optimized Parallel Cache Join) that implements a parallel loading of stored data into memory while the DP phase is executing. We present the performance analysis of both of our approaches with existing CACHEJOIN empirically using synthetic skewed dataset.
SubCharacter Chinese-English Neural Machine Translation with Wubi encoding
Venue: arXiv cs.CL 2019
Year: 2019

Neural machine translation (NMT) is one of the best methods for understanding the differences in semantic rules between two languages. Especially for Indo-European languages, subword-level models have achieved impressive results. However, when the translation task involves Chinese, semantic granularity remains at the word and character level, so there is still need more fine-grained translation model of Chinese. In this paper, we introduce a simple and effective method for Chinese translation at the sub-character level. Our approach uses the Wubi method to translate Chinese into English; byte-pair encoding (BPE) is then applied. Our method for Chinese-English translation eliminates the need for a complicated word segmentation algorithm during preprocessing. Furthermore, our method allows for sub-character-level neural translation based on recurrent neural network (RNN) architecture, without preprocessing. The empirical results show that for Chinese-English translation tasks, our sub-character-level model has a comparable BLEU score to the subword model, despite having a much smaller vocabulary. Additionally, the small vocabulary is highly advantageous for NMT model compression.
This dataset does not exist: training models from generated images
Venue: arXiv cs.CV 2019
Year: 2019

Current generative networks are increasingly proficient in generating high-resolution realistic images. These generative networks, especially the conditional ones, can potentially become a great tool for providing new image datasets. This naturally brings the question: Can we train a classifier only on the generated data? This potential availability of nearly unlimited amounts of training data challenges standard practices for training machine learning models, which have been crafted across the years for limited and fixed size datasets. In this work we investigate this question and its related challenges. We identify ways to improve significantly the performance over naive training on randomly generated images with regular heuristics. We propose three standalone techniques that can be applied at different stages of the pipeline, i.e., data generation, training on generated data, and deploying on real data. We evaluate our proposed approaches on a subset of the ImageNet dataset and show encouraging results compared to classifiers trained on real images.
SENSE: Semantically Enhanced Node Sequence Embedding
Venue: arXiv cs.LG 2019
Year: 2019

Effectively capturing graph node sequences in the form of vector embeddings is critical to many applications. We achieve this by (i) first learning vector embeddings of single graph nodes and (ii) then composing them to compactly represent node sequences. Specifically, we propose SENSE-S (Semantically Enhanced Node Sequence Embedding - for Single nodes), a skip-gram based novel embedding mechanism, for single graph nodes that co-learns graph structure as well as their textual descriptions. We demonstrate that SENSE-S vectors increase the accuracy of multi-label classification tasks by up to 50% and link-prediction tasks by up to 78% under a variety of scenarios using real datasets. Based on SENSE-S, we next propose generic SENSE to compute composite vectors that represent a sequence of nodes, where preserving the node order is important. We prove that this approach is efficient in embedding node sequences, and our experiments on real data confirm its high accuracy in node order decoding.
Probing Contextualized Sentence Representations with Visual Awareness
Venue: arXiv cs.CL 2019
Year: 2019

We present a universal framework to model contextualized sentence representations with visual awareness that is motivated to overcome the shortcomings of the multimodal parallel data with manual annotations. For each sentence, we first retrieve a diversity of images from a shared cross-modal embedding space, which is pre-trained on a large-scale of text-image pairs. Then, the texts and images are respectively encoded by transformer encoder and convolutional neural network. The two sequences of representations are further fused by a simple and effective attention layer. The architecture can be easily applied to text-only natural language processing tasks without manually annotating multimodal parallel corpora. We apply the proposed method on three tasks, including neural machine translation, natural language inference and sequence labeling and experimental results verify the effectiveness.
Adaptive Kernel Value Caching for SVM Training
Venue: arXiv cs.LG 2019
Year: 2019

Support Vector Machines (SVMs) can solve structured multi-output learning problems such as multi-label classification, multiclass classification and vector regression. SVM training is expensive especially for large and high dimensional datasets. The bottleneck of the SVM training often lies in the kernel value computation. In many real-world problems, the same kernel values are used in many iterations during the training, which makes the caching of kernel values potentially useful. The majority of the existing studies simply adopt the LRU (least recently used) replacement strategy for caching kernel values. However, as we analyze in this paper, the LRU strategy generally achieves high hit ratio near the final stage of the training, but does not work well in the whole training process. Therefore, we propose a new caching strategy called EFU (less frequently used) which replaces the less frequently used kernel values that enhances LFU (least frequently used). Our experimental results show that EFU often has 20\% higher hit ratio than LRU in the training with the Gaussian kernel. To further optimize the strategy, we propose a caching strategy called HCST (hybrid caching for the SVM training), which has a novel mechanism to automatically adapt the better caching strategy in the different stages of the training. We have integrated the caching strategy into ThunderSVM, a recent SVM library on many-core processors. Our experiments show that HCST adaptively achieves high hit ratios with little runtime overhead among different problems including multi-label classification, multiclass classification and regression problems. Compared with other existing caching strategies, HCST achieves 20\% more reduction in training time on average.
Deep geometric knowledge distillation with graphs
Venue: arXiv cs.LG 2019
Year: 2019

In most cases deep learning architectures are trained disregarding the amount of operations and energy consumption. However, some applications, like embedded systems, can be resource-constrained during inference. A popular approach to reduce the size of a deep learning architecture consists in distilling knowledge from a bigger network (teacher) to a smaller one (student). Directly training the student to mimic the teacher representation can be effective, but it requires that both share the same latent space dimensions. In this work, we focus instead on relative knowledge distillation (RKD), which considers the geometry of the respective latent spaces, allowing for dimension-agnostic transfer of knowledge. Specifically we introduce a graph-based RKD method, in which graphs are used to capture the geometry of latent spaces. Using classical computer vision benchmarks, we demonstrate the ability of the proposed method to efficiently distillate knowledge from the teacher to the student, leading to better accuracy for the same budget as compared to existing RKD alternatives.
What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning
Venue: arXiv cs.CL 2019
Year: 2019

Pretrained transformer-based language models have achieved state of the art across countless tasks in natural language processing. These models are highly expressive, comprising at least a hundred million parameters and a dozen layers. Recent evidence suggests that only a few of the final layers need to be fine-tuned for high quality on downstream tasks. Naturally, a subsequent research question is, "how many of the last layers do we need to fine-tune?" In this paper, we precisely answer this question. We examine two recent pretrained language models, BERT and RoBERTa, across standard tasks in textual entailment, semantic similarity, sentiment analysis, and linguistic acceptability. We vary the number of final layers that are fine-tuned, then study the resulting change in task-specific effectiveness. We show that only a fourth of the final layers need to be fine-tuned to achieve 90% of the original quality. Surprisingly, we also find that fine-tuning all layers does not always help.
Newmark algorithm for dynamic analysis with Maxwell chain model
Venue: arXiv cs.CE 2019
Year: 2019

This paper investigates a time-stepping procedure of the Newmark type for dynamic analyses of viscoelastic structures characterized by a generalized Maxwell model. We depart from a scheme developed for a three-parameter model by Hatada et al. in 2000, which we extend to a generic Maxwell chain and demonstrate that the resulting algorithm can be derived from a suitably discretized Hamilton variational principle. This variational structure manifests itself in an excellent stability and a low artificial damping of the integrator, as we confirm with a mass-spring-dashpot example. After a straightforward generalization to distributed systems, the integrator may find use in, e.g., fracture simulations of laminated glass units, once combined with variationally-based fracture models.
The Effect of Erasure Coding on the Burstiness of Packet Loss
Venue: arXiv cs.IT 2019
Year: 2019

The perceived quality of real-time media delivered over IP networks depends on both the rate and burstiness of packet loss. In this paper, we develop a new mathematical model for the residual burstiness of loss under erasure coding. We derive the expected number of consecutive losses in a burst as a function of erasure coding parameters and the network loss probability assuming a Bernoulli model for network losses.
How Language-Neutral is Multilingual BERT?
Venue: arXiv cs.CL 2019
Year: 2019

Multilingual BERT (mBERT) provides sentence representations for 104 languages, which are useful for many multi-lingual tasks. Previous work probed the cross-linguality of mBERT using zero-shot transfer learning on morphological and syntactic tasks. We instead focus on the semantic properties of mBERT. We show that mBERT representations can be split into a language-specific component and a language-neutral component, and that the language-neutral component is sufficiently general in terms of modeling semantics to allow high-accuracy word-alignment and sentence retrieval but is not yet good enough for the more difficult task of MT quality estimation. Our work presents interesting challenges which must be solved to build better language-neutral representations, particularly for tasks requiring linguistic transfer of semantics.
CPM Training Waveforms With Autocorrelation Sidelobes Close To Zero
Venue: arXiv cs.IT 2019
Year: 2019

Continuous phase modulation (CPM) plays an important role in wireless communications due to its constant envelope signal property and tight spectrum confinement capability. Although CPM has been studied for many years, CPM training waveforms having autocorrelations with zero sidelobes have not been reported before, to the best of our knowledge. Existing works on CPM system design mostly assume that the channel fading coefficients are either perfectly known at the receiver or estimated using random CPM training waveforms. In this work, we propose a novel class of CPM training waveforms displaying autocorrelation sidelobes close to zero. The key idea of our construction is to apply differential encoding to Golay complementary pair having perfect aperiodic autocorrelation sum properties.
Optimal Z-complementary Code Set From Generalized Reed-Muller Codes
Venue: arXiv cs.IT 2019
Year: 2019

Z-complementary code set (ZCCS), an extension of perfect complementary codes (CCs), refers to a set of two-dimensional matrices having zero correlation zone properties. ZCCS can be used in various multi-channel systems to support, for example, quasi-synchronous interference-free multicarrier code-division multiple access communication and optimal channel estimation in multiple-input multiple-output systems. Traditional constructions of ZCCS heavily rely on a series of sequence operations which may not be feasible for rapid hardware generation particularly for long ZCCSs. In this paper, we propose a direct construction of ZCCS using second-order Reed-Muller codes with efficient graphical representation. Our proposed construction, valid for any number of isolated vertices present in the graph, is capable of generating optimal ZCCS meeting the set size upper bound.
LiDAR Enhanced Structure-from-Motion
Venue: arXiv cs.RO 2019
Year: 2019

Although Structure-from-Motion (SfM) as a maturing technique has been widely used in many applications, state-of-the-art SfM algorithms are still not robust enough in certain situations. For example, images for inspection purposes are often taken in close distance to obtain detailed textures, which will result in less overlap between images and thus decrease the accuracy of estimated motion. In this paper, we propose a LiDAR-enhanced SfM pipeline that jointly processes data from a rotating LiDAR and a stereo camera pair to estimate sensor motions. We show that incorporating LiDAR helps to effectively reject falsely matched images and significantly improve the model consistency in large-scale environments. Experiments are conducted in different environments to test the performance of the proposed pipeline and comparison results with the state-of-the-art SfM algorithms are reported.
Low-Level Linguistic Controls for Style Transfer and Content Preservation
Venue: arXiv cs.CL 2019
Year: 2019

Despite the success of style transfer in image processing, it has seen limited progress in natural language generation. Part of the problem is that content is not as easily decoupled from style in the text domain. Curiously, in the field of stylometry, content does not figure prominently in practical methods of discriminating stylistic elements, such as authorship and genre. Rather, syntax and function words are the most salient features. Drawing on this work, we model style as a suite of low-level linguistic controls, such as frequency of pronouns, prepositions, and subordinate clause constructions. We train a neural encoder-decoder model to reconstruct reference sentences given only content words and the setting of the controls. We perform style transfer by keeping the content words fixed while adjusting the controls to be indicative of another style. In experiments, we show that the model reliably responds to the linguistic controls and perform both automatic and manual evaluations on style transfer. We find we can fool a style classifier 84% of the time, and that our model produces highly diverse and stylistically distinctive outputs. This work introduces a formal, extendable model of style that can add control to any neural text generation system.
Towards the Avoidance of Counterfeit Memory: Identifying the DRAM Origin
Venue: arXiv cs.CR 2019
Year: 2019

Due to the globalization in the semiconductor supply chain, counterfeit dynamic random-access memory (DRAM) chips/modules have been spreading worldwide at an alarming rate. Deploying counterfeit DRAM modules into an electronic system can have severe consequences on security and reliability domains because of their sub-standard quality, poor performance, and shorter life span. Besides, studies suggest that a counterfeit DRAM can be more vulnerable to sophisticated attacks. However, detecting counterfeit DRAMs is very challenging because of their nature and ability to pass the initial testing. In this paper, we propose a technique to identify the DRAM origin (i.e., the origin of the manufacturer and the specification of individual DRAM) to detect and prevent counterfeit DRAM modules. A silicon evaluation shows that the proposed method reliably identifies off-the-shelf DRAM modules from three major manufacturers.
Virtual Cell Clustering with Optimal Resource Allocation to Maximize Capacity
Venue: arXiv cs.IT 2019
Year: 2019

This work proposes a new resource allocation optimization and network management framework for wireless networks using neighborhood-based optimization rather than fully centralized or fully decentralized methods. We propose hierarchical clustering with a minimax linkage criterion for the formation of the virtual cells. Once the virtual cells are formed, we consider two cooperation models: the interference coordination model and the coordinated multi-point decoding model. In the first model base stations in a virtual cell decode their signals independently, but allocate the communication resources cooperatively. In the second model base stations in the same virtual cell allocate the communication resources and decode their signals cooperatively. We address the resource allocation problem for each of these cooperation models. For the interference coordination model this problem is an NP-hard mixed-integer optimization problem whereas for the coordinated multi-point decoding model it is convex. Our numerical results indicate that proper design of the neighborhood-based optimization leads to significant gains in sum rate over fully decentralized optimization, yet may also have a significant sum rate penalty compared to fully centralized optimization. In particular, neighborhood-based optimization has a significant sum rate penalty compared to fully centralized optimization in the coordinated multi-point model, but not the interference coordination model.
Fully-dynamic Planarity Testing in Polylogarithmic Time
Venue: arXiv cs.DS 2019
Year: 2019

Given a dynamic graph subject to insertions and deletions of edges, a natural question is whether the graph presently admits a planar embedding. We give a deterministic fully-dynamic algorithm for general graphs, running in amortized $O(\log^3 n)$ time per edge insertion or deletion, that maintains a bit indicating whether or not the graph is presently planar. This is an exponential improvement over the previous best algorithm [Eppstein, Galil, Italiano, Spencer, 1996] which spends amortized $O(\sqrt{n})$ time per update.