OpenRaiser/Intern-Atlas huggingface dataset:
Deep Trajectory for Recognition of Human Behaviours
Venue: arXiv cs.CV 2019
Year: 2019

Identifying human actions in complex scenes is widely considered as a challenging research problem due to the unpredictable behaviors and variation of appearances and postures. For extracting variations in motion and postures, trajectories provide meaningful way. However, simple trajectories are normally represented by vector of spatial coordinates. In order to identify human actions, we must exploit structural relationship between different trajectories. In this paper, we propose a method that divides the video into N number of segments and then for each segment we extract trajectories. We then compute trajectory descriptor for each segment which capture the structural relationship among different trajectories in the video segment. For trajectory descriptor, we project all extracted trajectories on the canvas. This will result in texture image which can store the relative motion and structural relationship among the trajectories. We then train Convolution Neural Network (CNN) to capture and learn the representation from dense trajectories. . Experimental results shows that our proposed method out performs state of the art methods by 90.01% on benchmark data set.
On the Global Minimizers of Real Robust Phase Retrieval with Sparse Noise
Venue: arXiv cs.IT 2019
Year: 2019

We study a class of real robust phase retrieval problems under a Gaussian assumption on the coding matrix when the received signal is sparsely corrupted by noise. The goal is to establish conditions on the sparsity under which the input vector can be exactly recovered. The recovery problem is formulated as the minimization of the $\ell_1$ norm of the residual. The main contribution is a robust phase retrieval counterpart to the seminal paper by Candes and Tao on compressed sensing ($\ell_1$ regression) [Decoding by linear programming. IEEE Transactions on Information Theory, 51(12):4203-4215, 2005]. Our analysis depends on a key new property on the coding matrix which we call the {Absolute Range Property} (ARP). This property is an analogue to the Null Space Property (NSP) in compressed sensing. When the residuals are computed using squared magnitudes, we show that ARP follows from a standard Restricted Isometry Property (RIP). However, when the residuals are computed using absolute magnitudes, a new and very different kind of RIP or growth property is required. We conclude by showing that the robust phase retrieval objectives are sharp with respect to their minimizers with high probability.
Emotion Classification in Response to Tactile Enhanced Multimedia using Frequency Domain Features of Brain Signals
Venue: arXiv cs.HC 2019
Year: 2019

Tactile enhanced multimedia is generated by synchronizing traditional multimedia clips, to generate hot and cold air effect, with an electric heater and a fan. This objective is to give viewers a more realistic and immersing feel of the multimedia content. The response to this enhanced multimedia content (mulsemedia) is evaluated in terms of the appreciation/emotion by using human brain signals. We observe and record electroencephalography (EEG) data using a commercially available four channel MUSE headband. A total of 21 participants voluntarily participated in this study for EEG recordings. We extract frequency domain features from five different bands of each EEG channel. Four emotions namely: happy, relaxed, sad, and angry are classified using a support vector machine in response to the tactile enhanced multimedia. An increased accuracy of 76:19% is achieved when compared to 63:41% by using the time domain features. Our results show that the selected frequency domain features could be better suited for emotion classification in mulsemedia studies.
Regula Sub-rosa: Latent Backdoor Attacks on Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Recent work has proposed the concept of backdoor attacks on deep neural networks (DNNs), where misbehaviors are hidden inside "normal" models, only to be triggered by very specific inputs. In practice, however, these attacks are difficult to perform and highly constrained by sharing of models through transfer learning. Adversaries have a small window during which they must compromise the student model before it is deployed. In this paper, we describe a significantly more powerful variant of the backdoor attack, latent backdoors, where hidden rules can be embedded in a single "Teacher" model, and automatically inherited by all "Student" models through the transfer learning process. We show that latent backdoors can be quite effective in a variety of application contexts, and validate its practicality through real-world attacks against traffic sign recognition, iris identification of lab volunteers, and facial recognition of public figures (politicians). Finally, we evaluate 4 potential defenses, and find that only one is effective in disrupting latent backdoors, but might incur a cost in classification accuracy as tradeoff.
On constructions and properties of $(n,m)$-functions with maximal number of bent components
Venue: arXiv cs.IT 2019
Year: 2019

For any positive integers $n=2k$ and $m$ such that $m\geq k$, in this paper we show the maximal number of bent components of any $(n,m)$-functions is equal to $2^{m}-2^{m-k}$, and for those attaining the equality, their algebraic degree is at most $k$. It is easily seen that all $(n,m)$-functions of the form $G(x)=(F(x),0)$ with $F(x)$ being any vectorial bent $(n,k)$-function, have the maximum number of bent components. Those simple functions $G$ are called trivial in this paper. We show that for a power $(n,n)$-function, it has such large number of bent components if and only if it is trivial under a mild condition. We also consider the $(n,n)$-function of the form $F^{i}(x)=x^{2^{i}}h({\rm Tr}^{n}_{e}(x))$, where $h: \mathbb{F}_{2^{e}} \rightarrow \mathbb{F}_{2^{e}}$, and show that $F^{i}$ has such large number if and only if $e=k$, and $h$ is a permutation over $\mathbb{F}_{2^{k}}$. It proves that all the previously known nontrivial such functions are subclasses of the functions $F^{i}$. Based on the Maiorana-McFarland class, we present constructions of large numbers of $(n,m)$-functions with maximal number of bent components for any integer $m$ in bivariate representation. We also determine the differential spectrum and Walsh spectrum of the constructed functions. It is found that our constructions can also provide new plateaued vectorial functions.
SuperCaptioning: Image Captioning Using Two-dimensional Word Embedding
Venue: arXiv cs.CL 2019
Year: 2019

Language and vision are processed as two different modal in current work for image captioning. However, recent work on Super Characters method shows the effectiveness of two-dimensional word embedding, which converts text classification problem into image classification problem. In this paper, we propose the SuperCaptioning method, which borrows the idea of two-dimensional word embedding from Super Characters method, and processes the information of language and vision together in one single CNN model. The experimental results on Flickr30k data shows the proposed method gives high quality image captions. An interactive demo is ready to show at the workshop.
Bandwidth-Efficient Transaction Relay for Bitcoin
Venue: arXiv cs.CR 2019
Year: 2019

Bitcoin is a top-ranked cryptocurrency that has experienced huge growth and survived numerous attacks. The protocols making up Bitcoin must therefore accommodate the growth of the network and ensure security. Security of the Bitcoin network depends on connectivity between the nodes. Higher connectivity yields better security. In this paper we make two observations: (1) current connectivity in the Bitcoin network is too low for optimal security; (2) at the same time, increasing connectivity will substantially increase the bandwidth used by the transaction dissemination protocol, making it prohibitively expensive to operate a Bitcoin node. Half of the total bandwidth needed to operate a Bitcoin node is currently used to just announce transactions. Unlike block relay, transaction dissemination has received little attention in prior work. We propose a new transaction dissemination protocol, Erlay, that not only reduces the bandwidth consumption by 40% assuming current connectivity, but also keeps the bandwidth use almost constant as the connectivity increases. In contrast, the existing protocol increases the bandwidth consumption linearly with the number of connections. By allowing more connections at a small cost, Erlay improves the security of the Bitcoin network. And, as we demonstrate, Erlay also hardens the network against attacks that attempt to learn the origin node of a transaction. Erlay is currently being investigated by the Bitcoin community for future use with the Bitcoin protocol.
SaaS CloudQual: A Quality Model for Evaluating Software as a Service on the Cloud Computing Environment
Venue: arXiv cs.SE 2019
Year: 2019

Abstract The cloud computing is a key computing approach adopted by many organizations in order to share resources. It provides Everything As-A-Service (XaaS). Software-As-A-Service is an important resource on the cloud computing environment. Without installing any software locally, service user can use software as a utility. And enjoy the benefits of SaaS model. Hence SaaS usage is increased drastically, the demand for selecting quality is also increased. This paper presents a novel quality model intended for evaluating software as a service (SaaS), depending on the key features of Software as a service. Because SaaS key features are playing critical role in the quality and differentiating from conventional software quality.
Efficient Object Annotation via Speaking and Pointing
Venue: arXiv cs.CV 2019
Year: 2019

Deep neural networks deliver state-of-the-art visual recognition, but they rely on large datasets, which are time-consuming to annotate. These datasets are typically annotated in two stages: (1) determining the presence of object classes at the image level and (2) marking the spatial extent for all objects of these classes. In this work we use speech, together with mouse inputs, to speed up this process. We first improve stage one, by letting annotators indicate object class presence via speech. We then combine the two stages: annotators draw an object bounding box via the mouse and simultaneously provide its class label via speech. Using speech has distinct advantages over relying on mouse inputs alone. First, it is fast and allows for direct access to the class name, by simply saying it. Second, annotators can simultaneously speak and mark an object location. Finally, speech-based interfaces can be kept extremely simple, hence using them requires less mouse movement compared to existing approaches. Through extensive experiments on the COCO and ILSVRC datasets we show that our approach yields high-quality annotations at significant speed gains. Stage one takes 2.3x - 14.9x less annotation time than existing methods based on a hierarchical organization of the classes to be annotated. Moreover, when combining the two stages, we find that object class labels come for free: annotating them at the same time as bounding boxes has zero additional cost. On COCO, this makes the overall process 1.9x faster than the two-stage approach.
Reconstructing faces from voices
Venue: arXiv cs.SD 2019
Year: 2019

Voice profiling aims at inferring various human parameters from their speech, e.g. gender, age, etc. In this paper, we address the challenge posed by a subtask of voice profiling - reconstructing someone's face from their voice. The task is designed to answer the question: given an audio clip spoken by an unseen person, can we picture a face that has as many common elements, or associations as possible with the speaker, in terms of identity? To address this problem, we propose a simple but effective computational framework based on generative adversarial networks (GANs). The network learns to generate faces from voices by matching the identities of generated faces to those of the speakers, on a training set. We evaluate the performance of the network by leveraging a closely related task - cross-modal matching. The results show that our model is able to generate faces that match several biometric characteristics of the speaker, and results in matching accuracies that are much better than chance.
Reasoning on Grasp-Action Affordances
Venue: arXiv cs.RO 2019
Year: 2019

Artificial intelligence is essential to succeed in challenging activities that involve dynamic environments, such as object manipulation tasks in indoor scenes. Most of the state-of-the-art literature explores robotic grasping methods by focusing exclusively on attributes of the target object. When it comes to human perceptual learning approaches, these physical qualities are not only inferred from the object, but also from the characteristics of the surroundings. This work proposes a method that includes environmental context to reason on an object affordance to then deduce its grasping regions. This affordance is reasoned using a ranked association of visual semantic attributes harvested in a knowledge base graph representation. The framework is assessed using standard learning evaluation metrics and the zero-shot affordance prediction scenario. The resulting grasping areas are compared with unseen labelled data to asses their accuracy matching percentage. The outcome of this evaluation suggest the autonomy capabilities of the proposed method for object interaction applications in indoor environments.
Dynamic Epistemic Logic with ASP Updates: Application to Conditional Planning
Venue: arXiv cs.AI 2019
Year: 2019

Dynamic Epistemic Logic (DEL) is a family of multimodal logics that has proved to be very successful for epistemic reasoning in planning tasks. In this logic, the agent's knowledge is captured by modal epistemic operators whereas the system evolution is described in terms of (some subset of) dynamic logic modalities in which actions are usually represented as semantic objects called event models. In this paper, we study a variant of DEL, that wecall DEL[ASP], where actions are syntactically described by using an Answer Set Programming (ASP) representation instead of event models. This representation directly inherits high level expressive features like indirect effects, qualifications, state constraints, defaults, or recursive fluents that are common in ASP descriptions of action domains. Besides, we illustrate how this approach can be applied for obtaining conditional plans in single-agent, partially observable domains where knowledge acquisition may be represented as indirect effects of actions.
Beyond Visual Semantics: Exploring the Role of Scene Text in Image Understanding
Venue: arXiv cs.CV 2019
Year: 2019

Images with visual and scene text content are ubiquitous in everyday life. However, current image interpretation systems are mostly limited to using only the visual features, neglecting to leverage the scene text content. In this paper, we propose to jointly use scene text and visual channels for robust semantic interpretation of images. We do not only extract and encode visual and scene text cues, but also model their interplay to generate a contextual joint embedding with richer semantics. The contextual embedding thus generated is applied to retrieval and classification tasks on multimedia images, with scene text content, to demonstrate its effectiveness. In the retrieval framework, we augment our learned text-visual semantic representation with scene text cues, to mitigate vocabulary misses that may have occurred during the semantic embedding. To deal with irrelevant or erroneous recognition of scene text, we also apply query-based attention to our text channel. We show how the multi-channel approach, involving visual semantics and scene text, improves upon state of the art.
A Reference Architecture for Blockchain-based Peer-to-Peer IoT Applications
Venue: arXiv cs.SE 2019
Year: 2019

The advent of Blockchain and Distributed Ledger Technologies enable IoT and smart city application developers to conceive new types of applications and solutions for identity management, trust, and data monetization. However, architecting blockchain-based IoT applications remain challenging due to the heterogeneous nature of blockchain platforms and lack of guidelines on how to interface existing components in the IoT ecosystem with the emerging Blockchain technology. This article explains the characteristics of blockchain and IoT technologies and presents a general reference architecture that can be used to develop many blockchain-based peer-to-peer IoT applications.
Adversarial Distillation for Ordered Top-k Attacks
Venue: arXiv cs.LG 2019
Year: 2019

Deep Neural Networks (DNNs) are vulnerable to adversarial attacks, especially white-box targeted attacks. One scheme of learning attacks is to design a proper adversarial objective function that leads to the imperceptible perturbation for any test image (e.g., the Carlini-Wagner (C&W) method). Most methods address targeted attacks in the Top-1 manner. In this paper, we propose to learn ordered Top-k attacks (k>= 1) for image classification tasks, that is to enforce the Top-k predicted labels of an adversarial example to be the k (randomly) selected and ordered labels (the ground-truth label is exclusive). To this end, we present an adversarial distillation framework: First, we compute an adversarial probability distribution for any given ordered Top-k targeted labels with respect to the ground-truth of a test image. Then, we learn adversarial examples by minimizing the Kullback-Leibler (KL) divergence together with the perturbation energy penalty, similar in spirit to the network distillation method. We explore how to leverage label semantic similarities in computing the targeted distributions, leading to knowledge-oriented attacks. In experiments, we thoroughly test Top-1 and Top-5 attacks in the ImageNet-1000 validation dataset using two popular DNNs trained with clean ImageNet-1000 train dataset, ResNet-50 and DenseNet-121. For both models, our proposed adversarial distillation approach outperforms the C&W method in the Top-1 setting, as well as other baseline methods. Our approach shows significant improvement in the Top-5 setting against a strong modified C&W method.
Efficient Neural Task Adaptation by Maximum Entropy Initialization
Venue: arXiv cs.LG 2019
Year: 2019

Transferring knowledge from one neural network to another has been shown to be helpful for learning tasks with few training examples. Prevailing fine-tuning methods could potentially contaminate pre-trained features by comparably high energy random noise. This noise is mainly delivered from a careless replacement of task-specific parameters. We analyze theoretically such knowledge contamination for classification tasks and propose a practical and easy to apply method to trap and minimize the contaminant. In our approach, the entropy of the output estimates gets maximized initially and the first back-propagated error is stalled at the output of the last layer. Our proposed method not only outperforms the traditional fine-tuning, but also significantly speeds up the convergence of the learner. It is robust to randomness and independent of the choice of architecture. Overall, our experiments show that the power of transfer learning has been substantially underestimated so far.
Algorithmic and geometric aspects of data depth with focus on $β$-skeleton depth
Venue: arXiv cs.CG 2019
Year: 2019

The statistical rank tests play important roles in univariate non-parametric data analysis. If one attempts to generalize the rank tests to a multivariate case, the problem of defining a multivariate order will occur. It is not clear how to define a multivariate order or statistical rank in a meaningful way. One approach to overcome this problem is to use the notion of data depth which measures the centrality of a point with respect to a given data set. In other words, a data depth can be applied to indicate how deep a point is located with respect to a given data set. Using data depth, a multivariate order can be defined by ordering the data points according to their depth values. Various notions of data depth have been introduced over the last decades. In this thesis, we discuss three depth functions: two well-known depth functions halfspace depth and simplicial depth, and one recently defined depth function named as $β$-skeleton depth, $β\geq 1$. The $β$-skeleton depth is equivalent to the previously defined spherical depth and lens depth when $β=1$ and $β=2$, respectively. Our main focus in this thesis is to explore the geometric and algorithmic aspects of $β$-skeleton depth.
Disentangling Style and Content in Anime Illustrations
Venue: arXiv cs.CV 2019
Year: 2019

Existing methods for AI-generated artworks still struggle with generating high-quality stylized content, where high-level semantics are preserved, or separating fine-grained styles from various artists. We propose a novel Generative Adversarial Disentanglement Network which can disentangle two complementary factors of variations when only one of them is labelled in general, and fully decompose complex anime illustrations into style and content in particular. Training such model is challenging, since given a style, various content data may exist but not the other way round. Our approach is divided into two stages, one that encodes an input image into a style independent content, and one based on a dual-conditional generator. We demonstrate the ability to generate high-fidelity anime portraits with a fixed content and a large variety of styles from over a thousand artists, and vice versa, using a single end-to-end network and with applications in style transfer. We show this unique capability as well as superior output to the current state-of-the-art.
HadaNets: Flexible Quantization Strategies for Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

On-board processing elements on UAVs are currently inadequate for training and inference of Deep Neural Networks. This is largely due to the energy consumption of memory accesses in such a network. HadaNets introduce a flexible train-from-scratch tensor quantization scheme by pairing a full precision tensor to a binary tensor in the form of a Hadamard product. Unlike wider reduced precision neural network models, we preserve the train-time parameter count, thus out-performing XNOR-Nets without a train-time memory penalty. Such training routines could see great utility in semi-supervised online learning tasks. Our method also offers advantages in model compression, as we reduce the model size of ResNet-18 by 7.43 times with respect to a full precision model without utilizing any other compression techniques. We also demonstrate a 'Hadamard Binary Matrix Multiply' kernel, which delivers a 10-fold increase in performance over full precision matrix multiplication with a similarly optimized kernel.
Secure Wireless Communication via Intelligent Reflecting Surface
Venue: arXiv cs.IT 2019
Year: 2019

An intelligent reflecting surface (IRS) can adaptively adjust the phase shifts of its reflecting units to strengthen the desired signal and/or suppress the undesired signal. In this letter, we investigate an IRS-aided secure wireless communication system where a multi-antenna access point (AP) sends confidential messages to a single-antenna user in the presence of a single-antenna eavesdropper. In particular, we consider the challenging scenario where the eavesdropping channel is stronger than the legitimate communication channel and they are also highly correlated in space. We maximize the secrecy rate of the legitimate communication link by jointly designing the AP's transmit beamforming and the IRS's reflect beamforming. While the resultant optimization problem is difficult to solve, we propose an efficient algorithm to obtain high-quality suboptimal solution for it by applying the alternating optimization and semidefinite relaxation methods. Simulation results show that the proposed design significantly improves the secrecy communication rate for the considered setup over the case without using the IRS, and outperforms a heuristic scheme.