OpenRaiser/Intern-Atlas huggingface dataset:
Expanding the Text Classification Toolbox with Cross-Lingual Embeddings
Venue: arXiv cs.CL 2019
Year: 2019

Most work in text classification and Natural Language Processing (NLP) focuses on English or a handful of other languages that have text corpora of hundreds of millions of words. This is creating a new version of the digital divide: the artificial intelligence (AI) divide. Transfer-based approaches, such as Cross-Lingual Text Classification (CLTC) - the task of categorizing texts written in different languages into a common taxonomy, are a promising solution to the emerging AI divide. Recent work on CLTC has focused on demonstrating the benefits of using bilingual word embeddings as features, relegating the CLTC problem to a mere benchmark based on a simple averaged perceptron. In this paper, we explore more extensively and systematically two flavors of the CLTC problem: news topic classification and textual churn intent detection (TCID) in social media. In particular, we test the hypothesis that embeddings with context are more effective, by multi-tasking the learning of multilingual word embeddings and text classification; we explore neural architectures for CLTC; and we move from bi- to multi-lingual word embeddings. For all architectures, types of word embeddings and datasets, we notice a consistent gain trend in favor of multilingual joint training, especially for low-resourced languages.
Physics-Based Freely Scalable Continuum Deformation for UAS Traffic Coordination
Venue: arXiv cs.SY 2019
Year: 2019

This paper develops a novel physics-inspired traffic coordination approach and applies it to Unmanned Aircraft System (UAS) traffic management. We extend available physics-inspired approaches previously applied to 1-D traffic flow on highways and urban streets to support models of traffic coordination in higher dimension airspace for cases where no predefined paths exist. The paper considers airspace as a finite control volume while UAS coordination, treated as continuum deformation, is controlled at the airspace boundaries. By partitioning airspace into planned and unplanned spaces, the paper models nominal coordination in the planned airspace as the solution of a partial differential equation with spatiotemporal parameters. This paper also improves resilience to vehicle failures with a resilient boundary control algorithm to update the geometry of the planned space when UAS problems threaten safe coordination in existing navigable airspace channels. To support UAS coordination at the microscopic level, we propose clustering vehicles based on vehicle performance limits. UAS clusters, with each UAS treated as a particle of a virtual rigid body, use leader-follower containment to acquire the macroscopic desired trajectory.
Fast Compressed Power Spectrum Estimation: Towards A Practical Solution for Wideband Spectrum Sensing
Venue: arXiv cs.IT 2019
Year: 2019

There has been a growing interest in wideband spectrum sensing due to its applications in cognitive radios and electronic surveillance. To overcome the sampling rate bottleneck for wideband spectrum sensing, in this paper, we study the problem of compressed power spectrum estimation whose objective is to reconstruct the power spectrum of a wide-sense stationary signal based on sub-Nyquist samples. By exploring the sampling structure inherent in the multicoset sampling scheme, we develop a computationally efficient method for power spectrum reconstruction. An important advantage of our proposed method over existing compressed power spectrum estimation methods is that our proposed method, whose primary computational task consists of fast Fourier transform (FFT), has a very low computational complexity. Such a merit makes it possible to efficiently implement the proposed algorithm in a practical field-programmable gate array (FPGA)-based system for real-time wideband spectrum sensing. Our proposed method also provides a new perspective on the power spectrum recovery condition, which leads to a result similar to what was reported in prior works. Simulation results are presented to show the computational efficiency and the effectiveness of the proposed method.
KPTransfer: improved performance and faster convergence from keypoint subset-wise domain transfer in human pose estimation
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we present a novel approach called KPTransfer for improving modeling performance for keypoint detection deep neural networks via domain transfer between different keypoint subsets. This approach is motivated by the notion that rich contextual knowledge can be transferred between different keypoint subsets representing separate domains. In particular, the proposed method takes into account various keypoint subsets/domains by sequentially adding and removing keypoints. Contextual knowledge is transferred between two separate domains via domain transfer. Experiments to demonstrate the efficacy of the proposed KPTransfer approach were performed for the task of human pose estimation on the MPII dataset, with comparisons against random initialization and frozen weight extraction configurations. Experimental results demonstrate the efficacy of performing domain transfer between two different joint subsets resulting in a PCKh improvement of up to 1.1 over random initialization on joints such as wrists and knee in certain joint splits with an overall PCKh improvement of 0.5. Domain transfer from a different set of joints not only results in improved accuracy but also results in faster convergence because of mutual co-adaptations of weights resulting from the contextual knowledge of the pose from a different set of joints.
Full-Duplex MIMO-OFDM Communication with Self-Energy Recycling
Venue: arXiv cs.IT 2019
Year: 2019

This paper focuses on energy recycling in full-duplex (FD) relaying multiple-input-multiple-output orthogonal frequency division multiplexing (OFDM) communication. The loop self-interference (SI) due to full-duplexing is seen as an opportunity for the energy-constrained relay node to replenish its energy requirement through wireless power transfer. In forwarding the source information to the destination, the FD relay can simultaneously harvest energy from the source wireless transmission and also through energy recycling from its own transmission. The objective is to maximize the overall spectral efficiency by designing the optimal power allocation over OFDM sub-carriers and transmit antennas. Due to a large number of sub-carriers, this design problem poses a large-scale nonconvex optimization problem involving a few thousand variables of power allocation, which is very computationally challenging. A new path-following algorithm is proposed, which converges to an optimal solution. This algorithm is very efficient since it is based on \textit{closed-form} calculations. Numerical results for a practical simulation setting show promising results by achieving high spectral efficiency.
Relation extraction between the clinical entities based on the shortest dependency path based LSTM
Venue: arXiv cs.CL 2019
Year: 2019

Owing to the exponential rise in the electronic medical records, information extraction in this domain is becoming an important area of research in recent years. Relation extraction between the medical concepts such as medical problem, treatment, and test etc. is also one of the most important tasks in this area. In this paper, we present an efficient relation extraction system based on the shortest dependency path (SDP) generated from the dependency parsed tree of the sentence. Instead of relying on many handcrafted features and the whole sequence of tokens present in a sentence, our system relies only on the SDP between the target entities. For every pair of entities, the system takes only the words in the SDP, their dependency labels, Part-of-Speech information and the types of the entities as the input. We develop a dependency parser for extracting dependency information. We perform our experiments on the benchmark i2b2 dataset for clinical relation extraction challenge 2010. Experimental results show that our system outperforms the existing systems.
MUSCO: Multi-Stage Compression of neural networks
Venue: arXiv cs.LG 2019
Year: 2019

The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.
Residual Non-local Attention Networks for Image Restoration
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a residual non-local attention network for high-quality image restoration. Without considering the uneven distribution of information in the corrupted images, previous methods are restricted by local convolutional operation and equal treatment of spatial- and channel-wise features. To address this issue, we design local and non-local attention blocks to extract features that capture the long-range dependencies between pixels and pay more attention to the challenging parts. Specifically, we design trunk branch and (non-)local mask branch in each (non-)local attention block. The trunk branch is used to extract hierarchical features. Local and non-local mask branches aim to adaptively rescale these hierarchical features with mixed attentions. The local mask branch concentrates on more local structures with convolutional operations, while non-local attention considers more about long-range dependencies in the whole feature map. Furthermore, we propose residual local and non-local attention learning to train the very deep network, which further enhance the representation ability of the network. Our proposed method can be generalized for various image restoration applications, such as image denoising, demosaicing, compression artifacts reduction, and super-resolution. Experiments demonstrate that our method obtains comparable or better results compared with recently leading methods quantitatively and visually.
Training Data Independent Image Registration With GANs Using Transfer Learning And Segmentation Information
Venue: arXiv cs.CV 2019
Year: 2019

Registration is an important task in automated medical image analysis. Although deep learning (DL) based image registration methods out perform time consuming conventional approaches, they are heavily dependent on training data and do not generalize well for new images types. We present a DL based approach that can register an image pair which is different from the training images. This is achieved by training generative adversarial networks (GANs) in combination with segmentation information and transfer learning. Experiments on chest Xray and brain MR images show that our method gives better registration performance over conventional methods.
Predicting Multiple Demographic Attributes with Task Specific Embedding Transformation and Attention Network
Venue: arXiv cs.LG 2019
Year: 2019

Most companies utilize demographic information to develop their strategy in a market. However, such information is not available to most retail companies. Several studies have been conducted to predict the demographic attributes of users from their transaction histories, but they have some limitations. First, they focused on parameter sharing to predict all attributes but capturing task-specific features is also important in multi-task learning. Second, they assumed that all transactions are equally important in predicting demographic attributes. However, some transactions are more useful than others for predicting a certain attribute. Furthermore, decision making process of models cannot be interpreted as they work in a black-box manner. To address the limitations, we propose an Embedding Transformation Network with Attention (ETNA) model which shares representations at the bottom of the model structure and transforms them to task-specific representations using a simple linear transformation method. In addition, we can obtain more informative transactions for predicting certain attributes using the attention mechanism. The experimental results show that our model outperforms the previous models on all tasks. In our qualitative analysis, we show the visualization of attention weights, which provides business managers with some useful insights.
DenseBody: Directly Regressing Dense 3D Human Pose and Shape From a Single Color Image
Venue: arXiv cs.CV 2019
Year: 2019

Recovering 3D human body shape and pose from 2D images is a challenging task due to high complexity and flexibility of human body, and relatively less 3D labeled data. Previous methods addressing these issues typically rely on predicting intermediate results such as body part segmentation, 2D/3D joints, silhouette mask to decompose the problem into multiple sub-tasks in order to utilize more 2D labels. Most previous works incorporated parametric body shape model in their methods and predict parameters in low-dimensional space to represent human body. In this paper, we propose to directly regress the 3D human mesh from a single color image using Convolutional Neural Network(CNN). We use an efficient representation of 3D human shape and pose which can be predicted through an encoder-decoder neural network. The proposed method achieves state-of-the-art performance on several 3D human body datasets including Human3.6M, SURREAL and UP-3D with even faster running speed.
Down-Scaling with Learned Kernels in Multi-Scale Deep Neural Networks for Non-Uniform Single Image Deblurring
Venue: arXiv cs.CV 2019
Year: 2019

Multi-scale approach has been used for blind image / video deblurring problems to yield excellent performance for both conventional and recent deep-learning-based state-of-the-art methods. Bicubic down-sampling is a typical choice for multi-scale approach to reduce spatial dimension after filtering with a fixed kernel. However, this fixed kernel may be sub-optimal since it may destroy important information for reliable deblurring such as strong edges. We propose convolutional neural network (CNN)-based down-scale methods for multi-scale deep-learning-based non-uniform single image deblurring. We argue that our CNN-based down-scaling effectively reduces the spatial dimension of the original image, while learned kernels with multiple channels may well-preserve necessary details for deblurring tasks. For each scale, we adopt to use RCAN (Residual Channel Attention Networks) as a backbone network to further improve performance. Our proposed method yielded state-of-the-art performance on GoPro dataset by large margin. Our proposed method was able to achieve 2.59dB higher PSNR than the current state-of-the-art method by Tao. Our proposed CNN-based down-scaling was the key factor for this excellent performance since the performance of our network without it was decreased by 1.98dB. The same networks trained with GoPro set were also evaluated on large-scale Su dataset and our proposed method yielded 1.15dB better PSNR than the Tao's method. Qualitative comparisons on Lai dataset also confirmed the superior performance of our proposed method over other state-of-the-art methods.
Effect of payload size on goodput when message segmentations occur for wireless networks: Case of packet corruptions recovered by stop-and-wait protocol
Venue: arXiv cs.PF 2019
Year: 2019

This paper investigates the effect of payload size on goodput for wireless networks where packets created from a message through a segmentation function are lost due to bit errors and they are recovered by a stop-and-wait protocol. To achieve this, we derive the exact analytical form of goodput using the analytical form of a packet-size distribution, given a message-size distribution and a payload size. In previous work, the packet sizes are assumed to be constant, which are payload size plus header size, although actual segmented packets are not constant in size. Hence, this constant packet-size assumption may be not justified for goodput analysis. From numerical results, we show that the constant packet-size assumption is not justified under low bit-error rates. Furthermore, we indicate that the curves of goodput are concave in payload size under high bit-error rates. In addition, we show that the larger mean bit-error burst length yields less concave curves of goodput.
Capturing the symptoms of malicious code in electronic documents by file's entropy signal combined with Machine learning
Venue: arXiv cs.CY 2019
Year: 2019

Abstract-Email cyber-attacks based on malicious documents have become the popular techniques in today's sophisticated attacks. In the past, persistent efforts have been made to detect such attacks. But there are still some common defects in the existing methods including unable to capture unknown attacks, high overhead of resource and time, and just can be used to detect specific formats of documents. In this study, a new Framework named ESRMD (Entropy signal Reflects the Malicious document) is proposed, which can detect malicious document based on the entropy distribution of the file. In essence, ESRMD is a machine learning classifier. What makes it distinctive is that it extracts global and structural entropy features from the entropy of the malicious documents rather than the structural data or metadata of the file, enduing it the ability to deal with various document formats and against the parser-confusion and obfuscated attacks. In order to assess the validity of the model, we conducted extensive experiments on a collected dataset with 10381 samples in it, which contains malware (51.47%) and benign (48.53%) samples. The results show that our model can achieve a good performance on the true positive rate, precision and ROC with the value of 96.00%, 96.69% and 99.2% respectively. We also compared ESRMD with some leading antivirus engines and prevalent tools. The results showed that our framework can achieve a better performance compared with these engines and tools.
Fixed Region Beamforming using Frequency Diverse Subarray for Secure Mmwave Wireless Communications
Venue: arXiv cs.IT 2019
Year: 2019

Millimeter-wave (mmWave) using conventional phased array (CPA) enables highly directional and fixed angular beamforming (FAB), therefore enhancing physical layer security (PLS) in the angular domain. However, as the eavesdropper is located in the direction pointed by the mainlobe of the information-carrying beam, information leakage is inevitable and FAB cannot guarantee PLS performance. To address this threat, we propose a novel fixed region beamforming (FRB) by employing a frequency diverse subarray (FDSA) architecture to enhance the PLS performance for mmWave directional communications. In particular, we carefully introduce multiple frequency offset increments (FOIs) across subarrays to achieve a sophisticated beampattern synthesis that ensures a confined information transmission only within the desired angle-range region (DARR) in close vicinity of the target user. More specifically, we formulate the secrecy rate maximization problem with FRB over possible subarray FOIs, and consider two cases of interests, i.e., without/with the location information of eavesdropping, both turn out to be NP-hard. For the unknown eavesdropping location case, we propose a seeker optimization algorithm to minimize the maximum sidelobe peak of the beampattern outside the DARR. As for the known eavesdropping location case, a block coordinate descend linear approximation algorithm is proposed to minimize the sidelobe level in the eavesdropping region. By using the proposed FRB, the mainlobes of all subarrays are constructively superimposed in the DARR while the sidelobes are destructively overlayed outside the DARR. Therefore, FRB takes prominent effect on confining information transmission within the DARR. Numerical simulations demonstrate that the proposed FDSA-based FRB can provide superior PLS performance over the CPA-based FAB.
Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
Venue: arXiv cs.LG 2019
Year: 2019

We propose a novel approach to train a multi-modal policy from mixed demonstrations without their behavior labels. We develop a method to discover the latent factors of variation in the demonstrations. Specifically, our method is based on the variational autoencoder with a categorical latent variable. The encoder infers discrete latent factors corresponding to different behaviors from demonstrations. The decoder, as a policy, performs the behaviors accordingly. Once learned, the policy is able to reproduce a specific behavior by simply conditioning on a categorical vector. We evaluate our method on three different tasks, including a challenging task with high-dimensional visual inputs. Experimental results show that our approach is better than various baseline methods and competitive with a multi-modal policy trained by ground truth behavior labels.
Local Orthogonal Decomposition for Maximum Inner Product Search
Venue: arXiv cs.LG 2019
Year: 2019

Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the original IVFADC as well as all of its variants, after residuals are computed, the second production quantization step is completely independent of the first vector quantization step. In this work, we seek to exploit the connection between these two steps when we perform non-exhaustive search. More specifically, we decompose a residual vector locally into two orthogonal components and perform uniform quantization and multiscale quantization to each component respectively. The proposed method, called local orthogonal decomposition, combined with multiscale quantization consistently achieves higher recall than previous methods under the same bitrates. We conduct comprehensive experiments on large scale datasets as well as detailed ablation tests, demonstrating effectiveness of our method.
Symmetry Reduction for the Local Mu-Calculus
Venue: arXiv cs.LO 2019
Year: 2019

Model checking large networks of processes is challenging due to state explosion. In many cases, individual processes are isomorphic, but there is insufficient global symmetry to simplify model checking. This work considers the verification of local properties, those defined over the neighborhood of a process. Considerably generalizing earlier results on invariance, it is shown that all local mu-calculus properties, including safety and liveness properties, are preserved by neighborhood symmetries. Hence, it suffices to check them locally over a set of representative process neighborhoods. In general, local verification approximates verification over the global state space; however, if process interactions are outward-facing, the relationship is shown to be exact. For many network topologies, even those with little global symmetry, analysis with representatives provides a significant, even exponential, reduction in the cost of verification. Moreover, it is shown that for network families generated from building-block patterns, neighborhood symmetries are easily determined, and verification over the entire family reduces to verification over a finite set of representative process neighborhoods.