OpenRaiser/Intern-Atlas huggingface dataset:
Almost Optimal Tensor Sketch
Venue: arXiv cs.DS 2019
Year: 2019

We construct a matrix $M\in R^{m\otimes d^c}$ with just $m=O(c\,λ\,\varepsilon^{-2}\text{poly}\log1/\varepsilonδ)$ rows, which preserves the norm $\|Mx\|_2=(1\pm\varepsilon)\|x\|_2$ of all $x$ in any given $λ$ dimensional subspace of $ R^d$ with probability at least $1-δ$. This matrix can be applied to tensors $x^{(1)}\otimes\dots\otimes x^{(c)}\in R^{d^c}$ in $O(c\, m \min\{d,m\})$ time -- hence the name "Tensor Sketch". (Here $x\otimes y = \text{asvec}(xy^T) = [x_1y_1, x_1y_2,\dots,x_1y_m,x_2y_1,\dots,x_ny_m]\in R^{nm}$.) This improves upon earlier Tensor Sketch constructions by Pagh and Pham~[TOCT 2013, SIGKDD 2013] and Avron et al.~[NIPS 2014] which require $m=Ω(3^cλ^2δ^{-1})$ rows for the same guarantees. The factors of $λ$, $\varepsilon^{-2}$ and $\log1/δ$ can all be shown to be necessary making our sketch optimal up to log factors. With another construction we get $λ$ times more rows $m=\tilde O(c\,λ^2\,\varepsilon^{-2}(\log1/δ)^3)$, but the matrix can be applied to any vector $x^{(1)}\otimes\dots\otimes x^{(c)}\in R^{d^c}$ in just $\tilde O(c\, (d+m))$ time. This matches the application time of Tensor Sketch while still improving the exponential dependencies in $c$ and $\log1/δ$. Technically, we show two main lemmas: (1) For many Johnson Lindenstrauss (JL) constructions, if $Q,Q'\in R^{m\times d}$ are independent JL matrices, the element-wise product $Qx \circ Q'y$ equals $M(x\otimes y)$ for some $M\in R^{m\times d^2}$ which is itself a JL matrix. (2) If $M^{(i)}\in R^{m\times md}$ are independent JL matrices, then $M^{(1)}(x \otimes (M^{(2)}y \otimes \dots)) = M(x\otimes y\otimes \dots)$ for some $M\in R^{m\times d^c}$ which is itself a JL matrix. Combining these two results give an efficient sketch for tensors of any size.
Agent-based model for tumour-analysis using Python+Mesa
Venue: arXiv cs.MA 2019
Year: 2019

The potential power provided and possibilities presented by computation graphs has steered most of the available modeling techniques to re-implementing, utilization and including the complex nature of System Biology (SB). To model the dynamics of cellular population, we need to study a plethora of scenarios ranging from cell differentiation to tumor growth and etcetera. Test and verification of a model in research means running the model multiple times with different or in some cases identical parameters, to see how the model interacts and if some of the outputs would change regarding different parameters. In this paper, we will describe the development and implementation of a new agent-based model using Python. The model can be executed using a development environment (based on Mesa, and extremely simplified for convenience) with different parameters. The result is collecting large sets of data, which will allow an in-depth analysis in the microenvironment of the tumor by the means of network analysis.
About Fibonacci trees III: multiple Fibonacci trees
Venue: arXiv cs.DM 2019
Year: 2019

In this third paper, we revisit the question to which extent the properties of the trees associated to the tilings $\{p,4\}$ of the hyperbolic plane are still true if we consider a finitely generated tree by the same rules but rooted at a black node? What happens if, considering the same distinction between black and white nodes but changing the place of the black son in the rules. What happens if we change the representation of the numbers by another set of digits? We tackle all of these questions in the paper. The present paper is an extension of the previous papers arXiv:1904.12135 and arXiv:1907.04677.
ICSrange: A Simulation-based Cyber Range Platform for Industrial Control Systems
Venue: arXiv cs.CR 2019
Year: 2019

Maintenance staff of Industrial Control Systems (ICS) is generally not aware about information technologies, and even less about cyber security problems. The scary impact of cyber attacks in the industrial world calls for tools to train defensive skills and test effective security measures. Cyber range offers this opportunity, but current research is lacking cost-effective solutions verticalized for the industrial domain. This work proposes ICSrange, a simulation-based cyber range platform for Industrial Control Systems. ICSrange adopts Commercial-Off-The-Shelf (COTS) technologies to virtualize an enterprise network connected to Industrial Control Systems. ICSrange is the outcome of a preliminary study intended to investigate challenges and opportunities to build a configurable and extensible cyber range with simulated industrial processes. Literature shows that testbeds based on realistic mock-ups are effectively employed to develop complex exploits like Advanced Persistent Threats (APTs), hence motivating their usage to train and test security in ICS. We prove the effectiveness of ICSrange through the execution of a multi-staged attack that breaches an enterprise network and progressively intrudes a simulated ICS with water tanks. The attack mimics lateral movements as observed in APTs.
Using Weaker Consistency Models with Monitoring and Recovery for Improving Performance of Key-Value Stores
Venue: arXiv cs.DC 2019
Year: 2019

Consistency properties provided by most key-value stores can be classified into sequential consistency and eventual consistency. The former is easier to program with but suffers from lower performance whereas the latter suffers from potential anomalies while providing higher performance. We focus on the problem of what a designer should do if he/she has an algorithm that works correctly with sequential consistency but is faced with an underlying key-value store that provides a weaker consistency. We propose a detect-rollback based approach: The designer identifies a correctness predicate, say $P$, and continues to run the protocol, as our system monitors $P$. If $P$ is violated (because of weaker consistency), the system rolls back and resumes the computation at a state where $P$ holds. We evaluate this approach with graph-based applications running on the Voldemort key-value store. Our experiments with deployment on Amazon AWS EC2 instances shows that using eventual consistency with monitoring can provide a $50\%$ -- $80\%$ increase in throughput when compared with sequential consistency. We also observe that the overhead of the monitoring itself was low (typically less than $4\%$) and the latency of detecting violations was small. In particular, in a scenario designed to intentionally cause a large number of violations, more than $99.9\%$ of violations were detected in less than 50 milliseconds in regional networks, and in less than 3 seconds in global networks. We find that for some applications, frequent rollback can cause the program using eventual consistency to effectively \textit{stall}. We propose alternate mechanisms for dealing with re-occurring rollbacks. Overall, for applications considered in this paper, we find that even with rollback, eventual consistency provides better performance than using sequential consistency.
Outage Analysis of Cooperative NOMA Using Maximum Ratio Combining at Intersections
Venue: arXiv cs.IT 2019
Year: 2019

The paper investigates the improvement of using maximum ratio combining (MRC) in cooperative vehicular communications (VCs) transmission schemes considering non-orthogonal multiple access scheme (NOMA) at intersections. The transmission occurs between a source and two destination nodes with a help of a relay. The transmission is subject to interference originated from vehicles that are located on the roads. Closed form outage probability expressions are obtained. We compare the performance of MRC cooperative NOMA with a classical cooperative NOMA, and show that implementing MRC in cooperative NOMA transmission offers a significant improvement over the classical cooperative NOMA in terms of outage probability. We also compare the performance of MRC cooperative NOMA with MRC cooperative orthogonal multiple access (OMA), and we show that NOMA has a better performance than OMA. Finally, we show that the outage probability increases when the nodes come closer to the intersection, and that using MRC considering NOMA improves the performance in this context. The analysis is verified with Monte Carlo simulations.
Quasi-Newton Optimization Methods For Deep Learning Applications
Venue: arXiv cs.LG 2019
Year: 2019

Deep learning algorithms often require solving a highly non-linear and nonconvex unconstrained optimization problem. Methods for solving optimization problems in large-scale machine learning, such as deep learning and deep reinforcement learning (RL), are generally restricted to the class of first-order algorithms, like stochastic gradient descent (SGD). While SGD iterates are inexpensive to compute, they have slow theoretical convergence rates. Furthermore, they require exhaustive trial-and-error to fine-tune many learning parameters. Using second-order curvature information to find search directions can help with more robust convergence for non-convex optimization problems. However, computing Hessian matrices for large-scale problems is not computationally practical. Alternatively, quasi-Newton methods construct an approximate of the Hessian matrix to build a quadratic model of the objective function. Quasi-Newton methods, like SGD, require only first-order gradient information, but they can result in superlinear convergence, which makes them attractive alternatives to SGD. The limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) approach is one of the most popular quasi-Newton methods that construct positive definite Hessian approximations. In this chapter, we propose efficient optimization methods based on L-BFGS quasi-Newton methods using line search and trust-region strategies. Our methods bridge the disparity between first- and second-order methods by using gradient information to calculate low-rank updates to Hessian approximations. We provide formal convergence analysis of these methods as well as empirical results on deep learning applications, such as image classification tasks and deep reinforcement learning on a set of ATARI 2600 video games. Our results show a robust convergence with preferred generalization characteristics as well as fast training time.
ModiPick: SLA-aware Accuracy Optimization For Mobile Deep Inference
Venue: arXiv cs.PF 2019
Year: 2019

Mobile applications are increasingly leveraging complex deep learning models to deliver features, e.g., image recognition, that require high prediction accuracy. Such models can be both computation and memory-intensive, even for newer mobile devices, and are therefore commonly hosted in powerful remote servers. However, current cloud-based inference services employ static model selection approach that can be suboptimal for satisfying application SLAs (service level agreements), as they fail to account for inherent dynamic mobile environment. We introduce a cloud-based technique called ModiPick that dynamically selects the most appropriate model for each inference request, and adapts its selection to match different SLAs and execution time budgets that are caused by variable mobile environments. The key idea of ModiPick is to make inference speed and accuracy trade-offs at runtime with a pool of managed deep learning models. As such, ModiPick masks unpredictable inference time budgets and therefore meets SLA targets, while improving accuracy within mobile network constraints. We evaluate ModiPick through experiments based on prototype systems and through simulations. We show that ModiPick achieves comparable inference accuracy to a greedy approach while improving SLA adherence by up to 88.5%.
Towards Precise Robotic Grasping by Probabilistic Post-grasp Displacement Estimation
Venue: arXiv cs.RO 2019
Year: 2019

Precise robotic grasping is important for many industrial applications, such as assembly and palletizing, where the location of the object needs to be controlled and known. However, achieving precise grasps is challenging due to noise in sensing and control, as well as unknown object properties. We propose a method to plan robotic grasps that are both robust and precise by training two convolutional neural networks - one to predict the robustness of a grasp and another to predict a distribution of post-grasp object displacements. Our networks are trained with depth images in simulation on a dataset of over 1000 industrial parts and were successfully deployed on a real robot without having to be further fine-tuned. The proposed displacement estimator achieves a mean prediction errors of 0.68cm and 3.42deg on novel objects in real world experiments.
Bidding Strategies with Gender Nondiscrimination: Constraints for Online Ad Auctions
Venue: arXiv cs.GT 2019
Year: 2019

Interactions between bids to show ads online can lead to an advertiser's ad being shown to more men than women even when the advertiser does not target towards men. We design bidding strategies that advertisers can use to avoid such emergent discrimination without having to modify the auction mechanism. We mathematically analyze the strategies to determine the additional cost to the advertiser for avoiding discrimination, proving our strategies to be optimal in some settings. We use simulations to understand other settings.
An Outage Probability Analysis of Full-Duplex NOMA in UAV Communications
Venue: arXiv cs.IT 2019
Year: 2019

As unmanned aerial vehicles (UAVs) are expected to play a significant role in fifth generation (5G) networks, addressing spectrum scarcity in UAV communications remains a pressing issue. In this regard, the feasibility of full-duplex non-orthogonal multiple access (FD-NOMA) UAV communications to improve spectrum utilization is investigated in this paper. Specifically, closed-form outage probability expressions are presented for FD-NOMA, half-duplex non-orthogonal multiple access (HD-NOMA), and half-duplex orthogonal multiple access (HD-OMA) schemes over Rician shadowed fading channels. Extensive analysis revealed that the bottleneck of performance in FD-NOMA is at the downlink UAVs. Also, FD-NOMA exhibits lower outage probability at the ground station (GS) and downlink UAVs than HD-NOMA and HD-OMA under low transmit power regimes. At high transmit power regimes, FD-NOMA is limited by residual SI and inter-UAV interference at the downlink UAVs and FD-GS, respectively. The impact of shadowing is also shown to affect the reliability of FD-NOMA and HD-OMA at the downlink UAVs.
Automated Let's Play Commentary
Venue: arXiv cs.CL 2019
Year: 2019

Let's Plays of video games represent a relatively unexplored area for experimental AI in games. In this short paper, we discuss an approach to generate automated commentary for Let's Play videos, drawing on convolutional deep neural networks. We focus on Let's Plays of the popular game Minecraft. We compare our approach and a prior approach and demonstrate the generation of automated, artificial commentary.
Efficient Optimal Planning in non-FIFO Time-Dependent Flow Fields
Venue: arXiv cs.RO 2019
Year: 2019

We propose an algorithm for solving the time-dependent shortest path problem in flow fields where the FIFO (first-in-first-out) assumption is violated. This problem variant is important for autonomous vehicles in the ocean, for example, that cannot arbitrarily hover in a fixed position and that are strongly influenced by time-varying ocean currents. Although polynomial-time solutions are available for discrete-time problems, the continuous-time non-FIFO case is NP-hard with no known relevant special cases. Our main result is to show that this problem can be solved in polynomial time if the edge travel time functions are piecewise-constant, agreeing with existing worst-case bounds for FIFO problems with restricted slopes. We present a minimum-time algorithm for graphs that allows for paths with finite-length cycles, and then embed this algorithm within an asymptotically optimal sampling-based framework to find time-optimal paths in flows. The algorithm relies on an efficient data structure to represent and manipulate piecewise-constant functions and is straightforward to implement. We illustrate the behaviour of the algorithm in an example based on a common ocean vortex model in addition to simpler graph-based examples.
Source Dependency-Aware Transformer with Supervised Self-Attention
Venue: arXiv cs.CL 2019
Year: 2019

Recently, Transformer has achieved the state-of-the-art performance on many machine translation tasks. However, without syntax knowledge explicitly considered in the encoder, incorrect context information that violates the syntax structure may be integrated into source hidden states, leading to erroneous translations. In this paper, we propose a novel method to incorporate source dependencies into the Transformer. Specifically, we adopt the source dependency tree and define two matrices to represent the dependency relations. Based on the matrices, two heads in the multi-head self-attention module are trained in a supervised manner and two extra cross entropy losses are introduced into the training objective function. Under this training objective, the model is trained to learn the source dependency relations directly. Without requiring pre-parsed input during inference, our model can generate better translations with the dependency-aware context information. Experiments on bi-directional Chinese-to-English, English-to-Japanese and English-to-German translation tasks show that our proposed method can significantly improve the Transformer baseline.
Detector With Focus: Normalizing Gradient In Image Pyramid
Venue: arXiv cs.CV 2019
Year: 2019

An image pyramid can extend many object detection algorithms to solve detection on multiple scales. However, interpolation during the resampling process of an image pyramid causes gradient variation, which is the difference of the gradients between the original image and the scaled images. Our key insight is that the increased variance of gradients makes the classifiers have difficulty in correctly assigning categories. We prove the existence of the gradient variation by formulating the ratio of gradient expectations between an original image and scaled images, then propose a simple and novel gradient normalization method to eliminate the effect of this variation. The proposed normalization method reduce the variance in an image pyramid and allow the classifier to focus on a smaller coverage. We show the improvement in three different visual recognition problems: pedestrian detection, pose estimation, and object detection. The method is generally applicable to many vision algorithms based on an image pyramid with gradients.
Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Venue: arXiv cs.HC 2019
Year: 2019

The rapid advancement of artificial intelligence (AI) is changing our lives in many ways. One application domain is data science. New techniques in automating the creation of AI, known as AutoAI or AutoML, aim to automate the work practices of data scientists. AutoAI systems are capable of autonomously ingesting and pre-processing data, engineering new features, and creating and scoring models based on a target objectives (e.g. accuracy or run-time efficiency). Though not yet widely adopted, we are interested in understanding how AutoAI will impact the practice of data science. We conducted interviews with 20 data scientists who work at a large, multinational technology company and practice data science in various business settings. Our goal is to understand their current work practices and how these practices might change with AutoAI. Reactions were mixed: while informants expressed concerns about the trend of automating their jobs, they also strongly felt it was inevitable. Despite these concerns, they remained optimistic about their future job security due to a view that the future of data science work will be a collaboration between humans and AI systems, in which both automation and human expertise are indispensable.
FraudJudger: Real-World Data Oriented Fraud Detection on Digital Payment Platforms
Venue: arXiv cs.CR 2019
Year: 2019

Automated fraud behaviors detection on electronic payment platforms is a tough problem. Fraud users often exploit the vulnerability of payment platforms and the carelessness of users to defraud money, steal passwords, do money laundering, etc, which causes enormous losses to digital payment platforms and users. There are many challenges for fraud detection in practice. Traditional fraud detection methods require a large-scale manually labeled dataset, which is hard to obtain in reality. Manually labeled data cost tremendous human efforts. Besides, the continuous and rapid evolution of fraud users makes it hard to find new fraud patterns based on existing detection rules. In our work, we propose a real-world data oriented detection paradigm which can detect fraud users and upgrade its detection ability automatically. Based on the new paradigm, we design a novel fraud detection model, FraudJudger, to analyze users behaviors on digital payment platforms and detect fraud users with fewer labeled data in training. FraudJudger can learn the latent representations of users from unlabeled data with the help of Adversarial Autoencoder (AAE). Furthermore, FraudJudger can find new fraud patterns from unknown users by cluster analysis. Our experiment is based on a real-world electronic payment dataset. Comparing with other well-known fraud detection methods, FraudJudger can achieve better detection performance with only 10% labeled data.
Neural Style-Preserving Visual Dubbing
Venue: arXiv cs.CV 2019
Year: 2019

Dubbing is a technique for translating video content from one language to another. However, state-of-the-art visual dubbing techniques directly copy facial expressions from source to target actors without considering identity-specific idiosyncrasies such as a unique type of smile. We present a style-preserving visual dubbing approach from single video inputs, which maintains the signature style of target actors when modifying facial expressions, including mouth motions, to match foreign languages. At the heart of our approach is the concept of motion style, in particular for facial expressions, i.e., the person-specific expression change that is yet another essential factor beyond visual accuracy in face editing applications. Our method is based on a recurrent generative adversarial network that captures the spatiotemporal co-activation of facial expressions, and enables generating and modifying the facial expressions of the target actor while preserving their style. We train our model with unsynchronized source and target videos in an unsupervised manner using cycle-consistency and mouth expression losses, and synthesize photorealistic video frames using a layered neural face renderer. Our approach generates temporally coherent results, and handles dynamic backgrounds. Our results show that our dubbing approach maintains the idiosyncratic style of the target actor better than previous approaches, even for widely differing source and target actors.