OpenRaiser/Intern-Atlas huggingface dataset:
Advance gender prediction tool of first names and its use in analysing gender disparity in Computer Science in the UK, Malaysia and China
Venue: arXiv cs.CY 2019
Year: 2019

Global gender disparity in science is an unsolved problem. Predicting gender has an important role in analysing the gender gap through online data. We study this problem within the UK, Malaysia and China. We enhance the accuracy of an existing gender prediction tools of names that can predict the sex of Chinese characters and English characters simultaneously and with more precision. During our research, we found that there is no free gender forecasting tool to predict an arbitrary number of names. We addressed this shortcoming by providing a tool that can predict an arbitrary number of names with free requests. We demonstrate our tool through a number of experimental results. We show that this tool is better than other gender prediction tools of names for analysing social problems with big data. In our approach, lists of data can be dynamically processed and the results of the data can be displayed with a dynamic graph. We present experiments of using this tool to analyse the gender disparity in computer science in the UK, Malaysia and China.
UCAM Biomedical translation at WMT19: Transfer learning multi-domain ensembles
Venue: arXiv cs.CL 2019
Year: 2019

The 2019 WMT Biomedical translation task involved translating Medline abstracts. We approached this using transfer learning to obtain a series of strong neural models on distinct domains, and combining them into multi-domain ensembles. We further experiment with an adaptive language-model ensemble weighting scheme. Our submission achieved the best submitted results on both directions of English-Spanish.
Extending Eigentrust with the Max-Plus Algebra
Venue: arXiv cs.MA 2019
Year: 2019

Eigentrust is a simple and widely used algorithm, which quantifies trust based on the repeated application of an update matrix to a vector of initial trust values. In some cases, however, this procedure is rendered uninformative. Here, we characterise such situations and trace their origin to the algebraic conditions guaranteeing the convergence of the Power Method. We overcome the identified limitations by extending Eigentrust's core ideas into the Max-Plus Algebra. The empirical evaluation of our max-plus approach demonstrates improvements over Eigentrust.
Modeling and Interpreting Real-world Human Risk Decision Making with Inverse Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

We model human decision-making behaviors in a risk-taking task using inverse reinforcement learning (IRL) for the purposes of understanding real human decision making under risk. To the best of our knowledge, this is the first work applying IRL to reveal the implicit reward function in human risk-taking decision making and to interpret risk-prone and risk-averse decision-making policies. We hypothesize that the state history (e.g. rewards and decisions in previous trials) are related to the human reward function, which leads to risk-averse and risk-prone decisions. We design features that reflect these factors in the reward function of IRL and learn the corresponding weight that is interpretable as the importance of features. The results confirm the sub-optimal risk-related decisions of human-driven by the personalized reward function. In particular, the risk-prone person tends to decide based on the current pump number, while the risk-averse person relies on burst information from the previous trial and the average end status. Our results demonstrate that IRL is an effective tool to model human decision-making behavior, as well as to help interpret the human psychological process in risk decision-making.
A Variational Autoencoder for Probabilistic Non-Negative Matrix Factorisation
Venue: arXiv cs.LG 2019
Year: 2019

We introduce and demonstrate the variational autoencoder (VAE) for probabilistic non-negative matrix factorisation (PAE-NMF). We design a network which can perform non-negative matrix factorisation (NMF) and add in aspects of a VAE to make the coefficients of the latent space probabilistic. By restricting the weights in the final layer of the network to be non-negative and using the non-negative Weibull distribution we produce a probabilistic form of NMF which allows us to generate new data and find a probability distribution that effectively links the latent and input variables. We demonstrate the effectiveness of PAE-NMF on three heterogeneous datasets: images, financial time series and genomic.
Scalable Neural Architecture Search for 3D Medical Image Segmentation
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, a neural architecture search (NAS) framework is proposed for 3D medical image segmentation, to automatically optimize a neural architecture from a large design space. Our NAS framework searches the structure of each layer including neural connectivities and operation types in both of the encoder and decoder. Since optimizing over a large discrete architecture space is difficult due to high-resolution 3D medical images, a novel stochastic sampling algorithm based on a continuous relaxation is also proposed for scalable gradient based optimization. On the 3D medical image segmentation tasks with a benchmark dataset, an automatically designed architecture by the proposed NAS framework outperforms the human-designed 3D U-Net, and moreover this optimized architecture is well suited to be transferred for different tasks.
Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking
Venue: arXiv cs.CR 2019
Year: 2019

The rise of machine learning as a service and model sharing platforms has raised the need of traitor-tracing the models and proof of authorship. Watermarking technique is the main component of existing methods for protecting copyright of models. In this paper, we show that distillation, a widely used transformation technique, is a quite effective attack to remove watermark embedded by existing algorithms. The fragility is due to the fact that distillation does not retain the watermark embedded in the model that is redundant and independent to the main learning task. We design ingrain in response to the destructive distillation. It regularizes a neural network with an ingrainer model, which contains the watermark, and forces the model to also represent the knowledge of the ingrainer. Our extensive evaluations show that ingrain is more robust to distillation attack and its robustness against other widely used transformation techniques is comparable to existing methods.
Exact Crossing Number Parameterized by Vertex Cover
Venue: arXiv cs.DM 2019
Year: 2019

We prove that the exact crossing number of a graph can be efficiently computed for simple graphs having bounded vertex cover. In more precise words, Crossing Number is in FPT when parameterized by the vertex cover size. This is a notable advance since we know only very few nontrivial examples of graph classes with unbounded and yet efficiently computable crossing number. Our result can be viewed as a strengthening of a previous result of Lokshtanov [arXiv, 2015] that Optimal Linear Arrangement is in FPT when parameterized by the vertex cover size, and we use a similar approach of reducing the problem to a tractable instance of Integer Quadratic Programming as in Lokshtanov's paper.
Towards Compact and Robust Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks have achieved impressive performance in many applications but their large number of parameters lead to significant computational and storage overheads. Several recent works attempt to mitigate these overheads by designing compact networks using pruning of connections. However, we observe that most of the existing strategies to design compact networks fail to preserve network robustness against adversarial examples. In this work, we rigorously study the extension of network pruning strategies to preserve both benign accuracy and robustness of a network. Starting with a formal definition of the pruning procedure, including pre-training, weights pruning, and fine-tuning, we propose a new pruning method that can create compact networks while preserving both benign accuracy and robustness. Our method is based on two main insights: (1) we ensure that the training objectives of the pre-training and fine-tuning steps match the training objective of the desired robust model (e.g., adversarial robustness/verifiable robustness), and (2) we keep the pruning strategy agnostic to pre-training and fine-tuning objectives. We evaluate our method on four different networks on the CIFAR-10 dataset and measure benign accuracy, empirical robust accuracy, and verifiable robust accuracy. We demonstrate that our pruning method can preserve on average 93\% benign accuracy, 92.5\% empirical robust accuracy, and 85.0\% verifiable robust accuracy while compressing the tested network by 10$\times$.
Do we need two forms of feedback in the Rate Control Protocol (RCP)?
Venue: arXiv cs.NI 2019
Year: 2019

There is considerable interest in the networking community in explicit congestion control as it may allow the design of a fair, stable, low loss, low delay, and high utilization network. The Rate Control Protocol (RCP) is an example of such a congestion control protocol. The current design of RCP suggests that it should employ two forms of feedback; i.e. rate mismatch and queue size, in order to manage its flow control algorithms. An outstanding design question in RCP is whether the presence of queue size feedback is useful or not, given feedback based on rate mismatch. In this paper, we address this question using tools from control and bifurcation theory. We linearize the actual non-linear system and analyze the local asymptotic stability, robust stability and rate of convergence of both the design choices, i.e., with and without queue size feedback. But such analyses do not offer clear design recommendations on whether the queue feedback is useful or not. This motivates a bifurcation-theoretic analysis where we have to take non-linear terms into consideration, which helps to learn additional dynamical properties of the RCP system. In particular, we proceed to analyze two non-linear properties, namely, the type of Hopf bifurcation and the asymptotic stability of the bifurcating limit cycles. Analytical results reveal that the presence of queue feedback in RCP can induce a sub-critical Hopf bifurcation, which can lead to undesirable system behavior. Whereas, in the absence of queue feedback, the Hopf bifurcation is always super-critical where the bifurcating limit cycles are stable and of small amplitude. The analysis is corroborated by numerical computations and some packet-level simulations as well. Based on our work, the suggestion for RCP is to only include feedback based on rate mismatch in the design of the protocol.
Soft Subdivision Motion Planning for Complex Planar Robots
Venue: arXiv cs.CG 2019
Year: 2019

The design and implementation of theoretically-sound robot motion planning algorithms is challenging. Within the framework of resolution-exact algorithms, it is possible to exploit soft predicates for collision detection. The design of soft predicates is a balancing act between easily implementable predicates and their accuracy/effectivity. In this paper, we focus on the class of planar polygonal rigid robots with arbitrarily complex geometry. We exploit the remarkable decomposability property of soft collision-detection predicates of such robots. We introduce a general technique to produce such a decomposition. If the robot is an m-gon, the complexity of this approach scales linearly in m. This contrasts with the O(m^3) complexity known for exact planners. It follows that we can now routinely produce soft predicates for any rigid polygonal robot. This results in resolution-exact planners for such robots within the general Soft Subdivision Search (SSS) framework. This is a significant advancement in the theory of sound and complete planners for planar robots. We implemented such decomposed predicates in our open-source Core Library. The experiments show that our algorithms are effective, perform in real time on non-trivial environments, and can outperform many sampling-based methods.
A Congruence-based Perspective on Automata Minimization Algorithms
Venue: arXiv cs.FL 2019
Year: 2019

In this work we use a framework of finite-state automata constructions based on equivalences over words to provide new insights on the relation between well-known methods for computing the minimal deterministic automaton of a language.
Correspondence-Free Region Localization for Partial Shape Similarity via Hamiltonian Spectrum Alignment
Venue: arXiv cs.CG 2019
Year: 2019

We consider the problem of localizing relevant subsets of non-rigid geometric shapes given only a partial 3D query as the input. Such problems arise in several challenging tasks in 3D vision and graphics, including partial shape similarity, retrieval, and non-rigid correspondence. We phrase the problem as one of alignment between short sequences of eigenvalues of basic differential operators, which are constructed upon a scalar function defined on the 3D surfaces. Our method therefore seeks for a scalar function that entails this alignment. Differently from existing approaches, we do not require solving for a correspondence between the query and the target, therefore greatly simplifying the optimization process; our core technique is also descriptor-free, as it is driven by the geometry of the two objects as encoded in their operator spectra. We further show that our spectral alignment algorithm provides a remarkably simple alternative to the recent shape-from-spectrum reconstruction approaches. For both applications, we demonstrate improvement over the state-of-the-art either in terms of accuracy or computational cost.
Epistemic Risk-Sensitive Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

We develop a framework for interacting with uncertain environments in reinforcement learning (RL) by leveraging preferences in the form of utility functions. We claim that there is value in considering different risk measures during learning. In this framework, the preference for risk can be tuned by variation of the parameter $β$ and the resulting behavior can be risk-averse, risk-neutral or risk-taking depending on the parameter choice. We evaluate our framework for learning problems with model uncertainty. We measure and control for \emph{epistemic} risk using dynamic programming (DP) and policy gradient-based algorithms. The risk-averse behavior is then compared with the behavior of the optimal risk-neutral policy in environments with epistemic risk.
Problems with the EFG formalism: a solution attempt using observations
Venue: arXiv cs.GT 2019
Year: 2019

We argue that the extensive-form game (EFG) model isn't powerful enough to express all important aspects of imperfect information games, such as those related to decomposition and online game solving. We present a principled attempt to fix the formalism by considering information partitions that correspond to observations. We show that EFGs cannot be "fixed" without additional knowledge about the original task, and show how to extend the EFG model under this assumption. However, during our investigation, we ultimately concluded that a better solution is to abandon EFGs entirely and instead adopt a model based on partially observable stochastic games. The primary contribution of the text thus lies in exposing the problems with EFGs and presenting a detailed study on introducing imperfect information by adding observations into an underlying perfect-information model.
A Performance Study of the 2D Ising Model on GPUs
Venue: arXiv cs.DC 2019
Year: 2019

The simulation of the two-dimensional Ising model is used as a benchmark to show the computational capabilities of Graphic Processing Units (GPUs). The rich programming environment now available on GPUs and flexible hardware capabilities allowed us to quickly experiment with several implementation ideas: a simple stencil-based algorithm, recasting the stencil operations into matrix multiplies to take advantage of Tensor Cores available on NVIDIA GPUs, and a highly optimized multi-spin coding approach. Using the managed memory API available in CUDA allows for simple and efficient distribution of these implementations across a multi-GPU NVIDIA DGX-2 server. We show that even a basic GPU implementation can outperform current results published on TPUs and that the optimized multi-GPU implementation can simulate very large lattices faster than custom FPGA solutions.
On the Computational Power of RNNs
Venue: arXiv cs.CL 2019
Year: 2019

Recent neural network architectures such as the basic recurrent neural network (RNN) and Gated Recurrent Unit (GRU) have gained prominence as end-to-end learning architectures for natural language processing tasks. But what is the computational power of such systems? We prove that finite precision RNNs with one hidden layer and ReLU activation and finite precision GRUs are exactly as computationally powerful as deterministic finite automata. Allowing arbitrary precision, we prove that RNNs with one hidden layer and ReLU activation are at least as computationally powerful as pushdown automata. If we also allow infinite precision, infinite edge weights, and nonlinear output activation functions, we prove that GRUs are at least as computationally powerful as pushdown automata. All results are shown constructively.
Release early, release often, and watch your users' emotions
Venue: arXiv cs.SE 2019
Year: 2019

App stores are highly competitive markets, sometimes offering dozens of apps for a single use case. Unexpected app changes such as a feature removal might incite even loyal users to explore alternative apps. Sentiment analysis tools can help monitor users' emotions expressed, e.g., in app reviews or tweets. We found that these emotions include four recurring patterns corresponding to the app releases. Based on these patterns and online reports about popular apps, we derived five release lessons to assist app vendors maintain positive emotions and gain competitive advantages.
Principled Frameworks for Evaluating Ethics in NLP Systems
Venue: arXiv cs.CL 2019
Year: 2019

We critique recent work on ethics in natural language processing. Those discussions have focused on data collection, experimental design, and interventions in modeling. But we argue that we ought to first understand the frameworks of ethics that are being used to evaluate the fairness and justice of algorithmic systems. Here, we begin that discussion by outlining deontological ethics, and envision a research agenda prioritized by it.
High-Performance Deep Learning via a Single Building Block
Venue: arXiv cs.LG 2019
Year: 2019

Deep learning (DL) is one of the most prominent branches of machine learning. Due to the immense computational cost of DL workloads, industry and academia have developed DL libraries with highly-specialized kernels for each workload/architecture, leading to numerous, complex code-bases that strive for performance, yet they are hard to maintain and do not generalize. In this work, we introduce the batch-reduce GEMM kernel and show how the most popular DL algorithms can be formulated with this kernel as the basic building-block. Consequently, the DL library-development degenerates to mere (potentially automatic) tuning of loops around this sole optimized kernel. By exploiting our new kernel we implement Recurrent Neural Networks, Convolution Neural Networks and Multilayer Perceptron training and inference primitives in just 3K lines of high-level code. Our primitives outperform vendor-optimized libraries on multi-node CPU clusters, and we also provide proof-of-concept CNN kernels targeting GPUs. Finally, we demonstrate that the batch-reduce GEMM kernel within a tensor compiler yields high-performance CNN primitives, further amplifying the viability of our approach.
Efficient Neural Network Approaches for Leather Defect Classification
Venue: arXiv cs.CV 2019
Year: 2019

Genuine leather, such as the hides of cows, crocodiles, lizards and goats usually contain natural and artificial defects, like holes, fly bites, tick marks, veining, cuts, wrinkles and others. A traditional solution to identify the defects is by manual defect inspection, which involves skilled experts. It is time consuming and may incur a high error rate and results in low productivity. This paper presents a series of automatic image processing processes to perform the classification of leather defects by adopting deep learning approaches. Particularly, the leather images are first partitioned into small patches,then it undergoes a pre-processing technique, namely the Canny edge detection to enhance defect visualization. Next, artificial neural network (ANN) and convolutional neural network (CNN) are employed to extract the rich image features. The best classification result achieved is 80.3 %, evaluated on a data set that consists of 2000 samples. In addition, the performance metrics such as confusion matrix and Receiver Operating Characteristic (ROC) are reported to demonstrate the efficiency of the method proposed.
Txilm: Lossy Block Compression with Salted Short Hashing
Venue: arXiv cs.CR 2019
Year: 2019

Current blockchains are restricted by the low throughput. Aimed at this problem, we propose Txilm, a protocol that compresses the size of transaction presentation in each block to save the bandwidth of the network. In this protocol, a block carries short hashes of TXIDs instead of complete transactions. Combined with the sorted transactions based on TXIDs, Txilm realizes 80 times of data size reduction compared with the original blockchains. We also evaluate the probability of hash collisions, and provide methods of resolving such collisions. Finally, we design strategies to protect against potential attacks on Txilm.
Designing, 3D Printing of a Quadruped Robot and Choice of Materials for Fabrication
Venue: arXiv cs.RO 2019
Year: 2019

Purpose-This paper is based on design of a quadruped robot and manufacturing it with 3-D printer followed by its detailed analysis. It focuses on the advantages of additive manufacturing rather than conventional manufacturing techniques and also highlights its limitations. The consequences of choice of different materials for 3-D printing are evaluated in this report. Design/methodology/Approach- The parts were designed in CAD software and made into Stereo lithography files, which were fed into the 3-D printing software. They were printed with the selection of ABS material. Low torque servos were employed in the beginning of the assembly and were controlled with the Arduino Uno micro controller. Findings- The versatility of legged robots require materials with high strength to weight ratio, where 3003 aluminium alloy sheets proved to be a better choice than conventional ABS. The ABS legs buckled under load and proved to be an inferior material choice for fabrication. The aluminium sheet fabricated legs not only imparted structural stability to the robot but also allowed in selection of more powerful servos for added strength.The bot could now achieve significant level of stability. Research Limitations/Implications- The power supply used for powering the bot was of SMPS type of power supply which made it less mobile. Due to the huge demand of current by the high torque servos used later, the power supply became a huge limitation which can be overcome by applying Lithium ion batteries. Practical Implications- The bot can be used for SLAM and autonomous navigation in areas where it is almost impossible for humans to access into. Originality/Value- This paper shows a concrete study on efficacy of quick CAD designing and rapid fabrication.