OpenRaiser/Intern-Atlas huggingface dataset:
Shape Reconstruction by Learning Differentiable Surface Representations
Venue: arXiv cs.CV 2019
Year: 2019

Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no control over the deformations of the surface patches that form the ensemble and thus fail to prevent them from either overlapping or collapsing into single points or lines. As a consequence, computing shape properties such as surface normals and curvatures becomes difficult and unreliable. In this paper, we show that we can exploit the inherent differentiability of deep networks to leverage differential surface properties during training so as to prevent patch collapse and strongly reduce patch overlap. Furthermore, this lets us reliably compute quantities such as surface normals and curvatures. We will demonstrate on several tasks that this yields more accurate surface reconstructions than the state-of-the-art methods in terms of normals estimation and amount of collapsed and overlapped patches.
Runtime Analysis of Quantum Programs: A Formal Approach
Venue: arXiv cs.LO 2019
Year: 2019

In this abstract we study the resource consumption of quantum programs. Specifically, we focus on the expected runtime of programs and, inspired by recent methods for probabilistic programs, we develop a calculus à la weakest precondition to formally and systematically derive the (exact) expected runtime of quantum programs. Notably, the calculus admits a notion of loop runtime invariant that can be readily used to derive upper bounds of their runtime. Finally, we show the applicability of our calculus analyzing the runtime of (a simplified version of) the BB84 quantum key distribution protocol.
JSLess: A Tale of a Fileless Javascript Memory-Resident Malware
Venue: arXiv cs.CR 2019
Year: 2019

New computing paradigms, modern feature-rich programming languages and off-the-shelf software libraries enabled the development of new sophisticated malware families. Evidence of this phenomena is the recent growth of fileless malware attacks. Fileless malware or memory resident malware is an example of an Advanced Volatile Threat (AVT). In a fileless malware attack, the malware writes itself directly onto the main memory (RAM) of the compromised device without leaving any trace on the compromised device's file system. For this reason, fileless malware presents a difficult challenge for traditional malware detection tools and in particular signature-based detection. Moreover, fileless malware forensics and reverse engineering are nearly impossible using traditional methods. The majority of fileless malware attacks in the wild take advantage of MS PowerShell, however, fileless malware are not limited to MS PowerShell. In this paper, we designed and implemented a fileless malware by taking advantage of new features in Javascript and HTML5. The proposed fileless malware could infect any device that supports Javascript and HTML5. It serves as a proof-of-concept (PoC) to demonstrate the threats of fileless malware in web applications. We used the proposed fileless malware to evaluate existing methods and techniques for malware detection in web applications. We tested the proposed fileless malware with several free and commercial malware detection tools that apply both static and dynamic analysis. The proposed fileless malware bypassed all the anti-malware detection tools included in our study. In our analysis, we discussed the limitations of existing approaches/tools and suggested possible detection and mitigation techniques.
Calibrated Domain-Invariant Learning for Highly Generalizable Large Scale Re-Identification
Venue: arXiv cs.CV 2019
Year: 2019

Many real-world applications, such as city-scale traffic monitoring and control, requires large-scale re-identification. However, previous ReID methods often failed to address two limitations in existing ReID benchmarks, i.e., low spatiotemporal coverage and sample imbalance. Notwithstanding their demonstrated success in every single benchmark, they have difficulties in generalizing to unseen environments. As a result, these methods are less applicable in a large-scale setting due to poor generalization. In seek for a highly generalizable large-scale ReID method, we present an adversarial domain invariant feature learning framework (ADIN) that explicitly learns to separate identity-related features from challenging variations, where for the first time "free" annotations in ReID data such as video timestamp and camera index are utilized. Furthermore, we find that the imbalance of nuisance classes jeopardizes the adversarial training, and for mitigation we propose a calibrated adversarial loss that is attentive to nuisance distribution. Experiments on existing large-scale person vehicle ReID datasets demonstrate that ADIN learns more robust and generalizable representations, as evidenced by its outstanding direct transfer performance across datasets, which is a criterion that can better measure the generalizability of large-scale ReID methods/
CAWA: An Attention-Network for Credit Attribution
Venue: arXiv cs.CL 2019
Year: 2019

Credit attribution is the task of associating individual parts in a document with their most appropriate class labels. It is an important task with applications to information retrieval and text summarization. When labeled training data is available, traditional approaches for sequence tagging can be used for credit attribution. However, generating such labeled datasets is expensive and time-consuming. In this paper, we present "Credit Attribution With Attention (CAWA)", a neural-network-based approach, that instead of using sentence-level labeled data, uses the set of class labels that are associated with an entire document as a source of distant-supervision. CAWA combines an attention mechanism with a multilabel classifier into an end-to-end learning framework to perform credit attribution. CAWA labels the individual sentences from the input document using the resultant attention-weights. CAWA improves upon the state-of-the-art credit attribution approach by not constraining a sentence to belong to just one class, but modeling each sentence as a distribution over all classes, leading to better modeling of semantically-similar classes. Experiments on the credit attribution task on a variety of datasets show that the sentence class labels generated by CAWA outperform the competing approaches. Additionally, on the multilabel text classification task, CAWA performs better than the competing credit attribution approaches.
Mandala: A Smart Contract Programming Language
Venue: arXiv cs.PL 2019
Year: 2019

Smart contracts on a blockchain behave precisely as specified by their code. A vulnerability in this code can lead to unexpected behaviour, which is hard to fix because a blockchain does not allow to change smart contract code after its deployment. Such vulnerabilities have led to several incidents. In the aftermath of such an event, a hard-fork between Ethereum and Ethereum classic was the result. This thesis proposes to develop a new smart contract programming language with the primary focus on safety, auditability, and the intention to prevent as many of the known categories of vulnerabilities by design as possible. The programming language's code is validated during deployment and afterwards isolated from other smart contracts running on the same blockchain to enforce compile-time guarantees during runtime. The designed programming language does evaluate new concepts and paradigms rarely used in non-smart contract environments for their potential benefit in a smart contract environment.
A Note on Computational Complexity of Kill-all Go
Venue: arXiv cs.CC 2019
Year: 2019

Kill-all Go is a variant of Go in which Black tries to capture all white stones, while White tries to survive. We consider computational complexity of Kill-all Go with two rulesets, Chinese rules and Japanese rules. We prove that: (i) Kill-all Go with Chinese rules is PSPACE-hard, and (ii) Kill-all Go with Japanese rules is EXPTIME-complete.
Occluded Pedestrian Detection with Visible IoU and Box Sign Predictor
Venue: arXiv cs.CV 2019
Year: 2019

Training a robust classifier and an accurate box regressor are difficult for occluded pedestrian detection. Traditionally adopted Intersection over Union (IoU) measurement does not consider the occluded region of the object and leads to improper training samples. To address such issue, a modification called visible IoU is proposed in this paper to explicitly incorporate the visible ratio in selecting samples. Then a newly designed box sign predictor is placed in parallel with box regressor to separately predict the moving direction of training samples. It leads to higher localization accuracy by introducing sign prediction loss during training and sign refining in testing. Following these novelties, we obtain state-of-the-art performance on CityPersons benchmark for occluded pedestrian detection.
Relevance-Promoting Language Model for Short-Text Conversation
Venue: arXiv cs.CL 2019
Year: 2019

Despite the effectiveness of sequence-to-sequence framework on the task of Short-Text Conversation (STC), the issue of under-exploitation of training data (i.e., the supervision signals from query text is \textit{ignored}) still remains unresolved. Also, the adopted \textit{maximization}-based decoding strategies, inclined to generating the generic responses or responses with repetition, are unsuited to the STC task. In this paper, we propose to formulate the STC task as a language modeling problem and tailor-make a training strategy to adapt a language model for response generation. To enhance generation performance, we design a relevance-promoting transformer language model, which performs additional supervised source attention after the self-attention to increase the importance of informative query tokens in calculating the token-level representation. The model further refines the query representation with relevance clues inferred from its multiple references during training. In testing, we adopt a \textit{randomization-over-maximization} strategy to reduce the generation of generic responses. Experimental results on a large Chinese STC dataset demonstrate the superiority of the proposed model on relevance metrics and diversity metrics.\footnote{Code available at https://ai.tencent.com/ailab/nlp/dialogue/.
Word-Class Embeddings for Multiclass Text Classification
Venue: arXiv cs.LG 2019
Year: 2019

Pre-trained word embeddings encode general word semantics and lexical regularities of natural language, and have proven useful across many NLP tasks, including word sense disambiguation, machine translation, and sentiment analysis, to name a few. In supervised tasks such as multiclass text classification (the focus of this article) it seems appealing to enhance word representations with ad-hoc embeddings that encode task-specific information. We propose (supervised) word-class embeddings (WCEs), and show that, when concatenated to (unsupervised) pre-trained word embeddings, they substantially facilitate the training of deep-learning models in multiclass classification by topic. We show empirical evidence that WCEs yield a consistent improvement in multiclass classification accuracy, using four popular neural architectures and six widely used and publicly available datasets for multiclass text classification. Our code that implements WCEs is publicly available at https://github.com/AlexMoreo/word-class-embeddings
Transaction Confirmation Time Prediction in Ethereum Blockchain Using Machine Learning
Venue: arXiv cs.CR 2019
Year: 2019

Blockchain offers a decentralized, immutable, transparent system of records. It offers a peer-to-peer network of nodes with no centralised governing entity making it unhackable and therefore, more secure than the traditional paper-based or centralised system of records like banks etc. While there are certain advantages to the paper-based recording approach, it does not work well with digital relationships where the data is in constant flux. Unlike traditional channels, governed by centralized entities, blockchain offers its users a certain level of anonymity by providing capabilities to interact without disclosing their personal identities and allows them to build trust without a third-party governing entity. Due to the aforementioned characteristics of blockchain, more and more users around the globe are inclined towards making a digital transaction via blockchain than via rudimentary channels. Therefore, there is a dire need for us to gain insight on how these transactions are processed by the blockchain and how much time it may take for a peer to confirm a transaction and add it to the blockchain network. This paper presents a novel approach that would allow one to estimate the time, in block time or otherwise, it would take for a mining node to accept and confirm a transaction to a block using machine learning. The paper also aims to compare the predictive accuracy of two machine learning regression models- Random Forest Regressor and Multilayer Perceptron against previously proposed statistical regression model under a set evaluation criterion. The objective is to determine whether machine learning offers a more accurate predictive model than conventional statistical models. The proposed model results in improved accuracy in prediction.
Distortion and Faults in Machine Learning Software
Venue: arXiv cs.LG 2019
Year: 2019

Machine learning software, deep neural networks (DNN) software in particular, discerns valuable information from a large dataset, a set of data. Outcomes of such DNN programs are dependent on the quality of both learning programs and datasets. Unfortunately, the quality of datasets is difficult to be defined, because they are just samples. The quality assurance of DNN software is difficult, because resultant trained machine learning models are unknown prior to its development, and the validation is conducted indirectly in terms of prediction performance. This paper introduces a hypothesis that faults in the learning programs manifest themselves as distortions in trained machine learning models. Relative distortion degrees measured with appropriate observer functions may indicate that there are some hidden faults. The proposal is demonstrated with example cases of the MNIST dataset.
Hybrid Text Feature Modeling for Disease Group Prediction using Unstructured Physician Notes
Venue: arXiv cs.CL 2019
Year: 2019

Existing Clinical Decision Support Systems (CDSSs) largely depend on the availability of structured patient data and Electronic Health Records (EHRs) to aid caregivers. However, in case of hospitals in developing countries, structured patient data formats are not widely adopted, where medical professionals still rely on clinical notes in the form of unstructured text. Such unstructured clinical notes recorded by medical personnel can also be a potential source of rich patient-specific information which can be leveraged to build CDSSs, even for hospitals in developing countries. If such unstructured clinical text can be used, the manual and time-consuming process of EHR generation will no longer be required, with huge person-hours and cost savings. In this paper, we propose a generic ICD9 disease group prediction CDSS built on unstructured physician notes modeled using hybrid word embeddings. These word embeddings are used to train a deep neural network for effectively predicting ICD9 disease groups. Experimental evaluation showed that the proposed approach outperformed the state-of-the-art disease group prediction model built on structured EHRs by 15% in terms of AUROC and 40% in terms of AUPRC, thus proving our hypothesis and eliminating dependency on availability of structured patient data.
Emergent Structures and Lifetime Structure Evolution in Artificial Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Motivated by the flexibility of biological neural networks whose connectivity structure changes significantly during their lifetime, we introduce the Unstructured Recursive Network (URN) and demonstrate that it can exhibit similar flexibility during training via gradient descent. We show empirically that many of the different neural network structures commonly used in practice today (including fully connected, locally connected and residual networks of different depths and widths) can emerge dynamically from the same URN. These different structures can be derived using gradient descent on a single general loss function where the structure of the data and the relative strengths of various regulator terms determine the structure of the emergent network. We show that this loss function and the regulators arise naturally when considering the symmetries of the network as well as the geometric properties of the input data.
Words Avoiding Reversed Factors, Revisited
Venue: arXiv cs.FL 2019
Year: 2019

In 2005, Rampersad and the second author proved a number of theorems about infinite words x with the property that if w is any sufficiently long finite factor of x, then its reversal w^R is not a factor of x. In this note we revisit these results, reproving them in more generality, using machine computations only. Two different techniques are presented.
Edge-Guided Occlusion Fading Reduction for a Light-Weighted Self-Supervised Monocular Depth Estimation
Venue: arXiv cs.CV 2019
Year: 2019

Self-supervised monocular depth estimation methods generally suffer the occlusion fading issue due to the lack of supervision by the per pixel ground truth. Although a post-processing method was proposed by Godard et. al. to reduce the occlusion fading, the compensated results have a severe halo effect. In this paper, we propose a novel Edge-Guided post-processing to reduce the occlusion fading issue for self-supervised monocular depth estimation. We further introduce Atrous Spatial Pyramid Pooling (ASPP) into the network to reduce the computational costs and improve the inference performance. The proposed ASPP-based network is lighter, faster, and better than current commonly used depth estimation networks. This light-weight network only needs 8.1 million parameters and can achieve up to 40 frames per second for $256\times512$ input in the inference stage using a single nVIDIA GTX1080 GPU. The proposed network also outperforms the current state-of-the-art on the KITTI benchmarks. The ASPP-based network and Edge-Guided post-processing produce better results either quantitatively and qualitatively than the competitors.
Network Embedding: An Overview
Venue: arXiv cs.LG 2019
Year: 2019

Networks are one of the most powerful structures for modeling problems in the real world. Downstream machine learning tasks defined on networks have the potential to solve a variety of problems. With link prediction, for instance, one can predict whether two persons will become friends on a social network. Many machine learning algorithms, however, require that each input example is a real vector. Network embedding encompasses various methods for unsupervised, and sometimes supervised, learning of feature representations of nodes and links in a network. Typically, embedding methods are based on the assumption that the similarity between nodes in the network should be reflected in the learned feature representations. In this paper, we review significant contributions to network embedding in the last decade. In particular, we look at four methods: Spectral Clustering, DeepWalk, Large-scale Information Network Embedding (LINE), and node2vec. We describe each method and list its advantages and shortcomings. In addition, we give examples of real-world machine learning problems on networks in which the embedding is critical in order to maximize the predictive performance of the machine learning task. Finally, we take a look at research trends and state-of-the art methods in the research on network embedding.
A geometric characterization of minimal codes and their asymptotic performance
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we give a geometric characterization of minimal linear codes. In particular, we relate minimal linear codes to cutting blocking sets, introduced in a recent paper by Bonini and Borello. Using this characterization, we derive some bounds on the length and the distance of minimal codes, according to their dimension and the underlying field size. Furthermore, we show that the family of minimal codes is asymptotically good. Finally, we provide some geometrical constructions of minimal codes.
Defending Against Adversarial Machine Learning
Venue: arXiv cs.CR 2019
Year: 2019

An Adversarial System to attack and an Authorship Attribution System (AAS) to defend itself against the attacks are analyzed. Defending a system against attacks from an adversarial machine learner can be done by randomly switching between models for the system, by detecting and reacting to changes in the distribution of normal inputs, or by using other methods. Adversarial machine learning is used to identify a system that is being used to map system inputs to outputs. Three types of machine learners are using for the model that is being attacked. The machine learners that are used to model the system being attacked are a Radial Basis Function Support Vector Machine, a Linear Support Vector Machine, and a Feedforward Neural Network. The feature masks are evolved using accuracy as the fitness measure. The system defends itself against adversarial machine learning attacks by identifying inputs that do not match the probability distribution of normal inputs. The system also defends itself against adversarial attacks by randomly switching between the feature masks being used to map system inputs to outputs.
Blockchains: a Systematic Multivocal Literature Review
Venue: arXiv cs.CR 2019
Year: 2019

Blockchain technology has gained tremendous popularity both in practice and academia. The goal of this article is to develop a coherent overview of the state of the art in blockchain technology, using a systematic(i.e.,protocol-based, replicable), multivocal (i.e., featuring both white and grey literature alike) literature review, to (1) define blockchain technology (2) elaborate on its architecture options and (3) trade-offs, as well as understanding (4) the current applications and challenges, as evident from the state of the art. We derive a systematic definition of blockchain technology, based on a formal concept analysis. Further on, we flesh out an overview of blockchain technology elaborated by means of Grounded-Theory.
A Utilization Model for Optimization of Checkpoint Intervals in Distributed Stream Processing Systems
Venue: arXiv cs.DC 2019
Year: 2019

State-of-the-art distributed stream processing systems such as Apache Flink and Storm have recently included checkpointing to provide fault-tolerance for stateful applications. This is a necessary eventuality as these systems head into the Exascale regime, and is evidently more efficient than replication as state size grows. However current systems use a nominal value for the checkpoint interval, indicative of assuming roughly 1 failure every 19 days, that does not take into account the salient aspects of the checkpoint process, nor the system scale, which can readily lead to inefficient system operation. To address this shortcoming, we provide a rigorous derivation of utilization -- the fraction of total time available for the system to do useful work -- that incorporates checkpoint interval, failure rate, checkpoint cost, failure detection and restart cost, depth of the system topology and message delay. Our model yields an elegant expression for utilization and provides an optimal checkpoint interval given these parameters, interestingly showing it to be dependent only on checkpoint cost and failure rate. We confirm the accuracy and efficacy of our model through experiments with Apache Flink, where we obtain improvements in system utilization for every case, especially as the system size increases. Our model provides a solid theoretical basis for the analysis and optimization of more elaborate checkpointing approaches.