OpenRaiser/Intern-Atlas huggingface dataset:
Enabling High-Capacity, Latency-Tolerant, and Highly-Concurrent GPU Register Files via Software/Hardware Cooperation
Venue: arXiv cs.AR 2020
Year: 2020

Graphics Processing Units (GPUs) employ large register files to accommodate all active threads and accelerate context switching. Unfortunately, register files are a scalability bottleneck for future GPUs due to long access latency, high power consumption, and large silicon area provisioning. Prior work proposes hierarchical register file to reduce the register file power consumption by caching registers in a smaller register file cache. Unfortunately, this approach does not improve register access latency due to the low hit rate in the register file cache. In this paper, we propose the Latency-Tolerant Register File (LTRF) architecture to achieve low latency in a two-level hierarchical structure while keeping power consumption low. We observe that compile-time interval analysis enables us to divide GPU program execution into intervals with an accurate estimate of a warp's aggregate register working-set within each interval. The key idea of LTRF is to prefetch the estimated register working-set from the main register file to the register file cache under software control, at the beginning of each interval, and overlap the prefetch latency with the execution of other warps. We observe that register bank conflicts while prefetching the registers could greatly reduce the effectiveness of LTRF. Therefore, we devise a compile-time register renumbering technique to reduce the likelihood of register bank conflicts. Our experimental results show that LTRF enables high-capacity yet long-latency main GPU register files, paving the way for various optimizations. As an example optimization, we implement the main register file with emerging high-density high-latency memory technologies, enabling 8X larger capacity and improving overall GPU performance by 34%.
Secure Consensus Generation with Distributed DoH
Venue: arXiv cs.CR 2020
Year: 2020

Many applications and protocols depend on the ability to generate a pool of servers to conduct majority-based consensus mechanisms and often this is done by doing plain DNS queries. A recent off-path attack [1] against NTP and security enhanced NTP with Chronos [2] showed that relying on DNS for generating the pool of NTP servers introduces a weak link. In this work, we propose a secure, backward-compatible address pool generation method using distributed DNS-over-HTTPS (DoH) resolvers which is aimed to prevent such attacks against server pool generation.
On Properties and Optimization of Information-theoretic Privacy Watchdog
Venue: arXiv cs.IT 2020
Year: 2020

We study the problem of privacy preservation in data sharing, where $S$ is a sensitive variable to be protected and $X$ is a non-sensitive useful variable correlated with $S$. Variable $X$ is randomized into variable $Y$, which will be shared or released according to $p_{Y|X}(y|x)$. We measure privacy leakage by \emph{information privacy} (also known as \emph{log-lift} in the literature), which guarantees mutual information privacy and differential privacy (DP). Let $\Xepsc \subseteq \X$ contain elements n the alphabet of $X$ for which the absolute value of log-lift (abs-log-lift for short) is greater than a desired threshold $\eps$. When elements $x\in \Xepsc$ are randomized into $y\in \Y$, we derive the best upper bound on the abs-log-lift across the resultant pairs $(s,y)$. We then prove that this bound is achievable via an \emph{$X$-invariant} randomization $p(y|x) = R(y)$ for $x,y\in\Xepsc$. However, the utility measured by the mutual information $I(X;Y)$ is severely damaged in imposing a strict upper bound $\eps$ on the abs-log-lift. To remedy this and inspired by the probabilistic ($\eps$, $δ$)-DP, we propose a relaxed ($\eps$, $δ$)-log-lift framework. To achieve this relaxation, we introduce a greedy algorithm which exempts some elements in $\Xepsc$ from randomization, as long as their abs-log-lift is bounded by $\eps$ with probability $1-δ$. Numerical results demonstrate efficacy of this algorithm in achieving a better privacy-utility tradeoff.
The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification
Venue: arXiv cs.CL 2020
Year: 2020

Relation classification is one of the key topics in information extraction, which can be used to construct knowledge bases or to provide useful information for question answering. Current approaches for relation classification are mainly focused on the English language and require lots of training data with human annotations. Creating and annotating a large amount of training data for low-resource languages is impractical and expensive. To overcome this issue, we propose two cross-lingual relation classification models: a baseline model based on Multilingual BERT and a new multilingual pretraining setup, which significantly improves the baseline with distant supervision. For evaluation, we introduce a new public benchmark dataset for cross-lingual relation classification in English, French, German, Spanish, and Turkish, called RELX. We also provide the RELX-Distant dataset, which includes hundreds of thousands of sentences with relations from Wikipedia and Wikidata collected by distant supervision for these languages. Our code and data are available at: https://github.com/boun-tabi/RELX
SD-DefSLAM: Semi-Direct Monocular SLAM for Deformable and Intracorporeal Scenes
Venue: arXiv cs.CV 2020
Year: 2020

Conventional SLAM techniques strongly rely on scene rigidity to solve data association, ignoring dynamic parts of the scene. In this work we present Semi-Direct DefSLAM (SD-DefSLAM), a novel monocular deformable SLAM method able to map highly deforming environments, built on top of DefSLAM. To robustly solve data association in challenging deforming scenes, SD-DefSLAM combines direct and indirect methods: an enhanced illumination-invariant Lucas-Kanade tracker for data association, geometric Bundle Adjustment for pose and deformable map estimation, and bag-of-words based on feature descriptors for camera relocation. Dynamic objects are detected and segmented-out using a CNN trained for the specific application domain. We thoroughly evaluate our system in two public datasets. The mandala dataset is a SLAM benchmark with increasingly aggressive deformations. The Hamlyn dataset contains intracorporeal sequences that pose serious real-life challenges beyond deformation like weak texture, specular reflections, surgical tools and occlusions. Our results show that SD-DefSLAM outperforms DefSLAM in point tracking, reconstruction accuracy and scale drift thanks to the improvement in all the data association steps, being the first system able to robustly perform SLAM inside the human body.
LANNS: A Web-Scale Approximate Nearest Neighbor Lookup System
Venue: arXiv cs.IR 2020
Year: 2020

Nearest neighbor search (NNS) has a wide range of applications in information retrieval, computer vision, machine learning, databases, and other areas. Existing state-of-the-art algorithm for nearest neighbor search, Hierarchical Navigable Small World Networks(HNSW), is unable to scale to large datasets of 100M records in high dimensions. In this paper, we propose LANNS, an end-to-end platform for Approximate Nearest Neighbor Search, which scales for web-scale datasets. Library for Large Scale Approximate Nearest Neighbor Search (LANNS) is deployed in multiple production systems for identifying topK ($100 \leq topK \leq 200$) approximate nearest neighbors with a latency of a few milliseconds per query, high throughput of 2.5k Queries Per Second (QPS) on a single node, on large ($\sim$180M data points) high dimensional (50-2048 dimensional) datasets.
Maps for Learning Indexable Classes
Venue: arXiv cs.LG 2020
Year: 2020

We study learning of indexed families from positive data where a learner can freely choose a hypothesis space (with uniformly decidable membership) comprising at least the languages to be learned. This abstracts a very universal learning task which can be found in many areas, for example learning of (subsets of) regular languages or learning of natural languages. We are interested in various restrictions on learning, such as consistency, conservativeness or set-drivenness, exemplifying various natural learning restrictions. Building on previous results from the literature, we provide several maps (depictions of all pairwise relations) of various groups of learning criteria, including a map for monotonicity restrictions and similar criteria and a map for restrictions on data presentation. Furthermore, we consider, for various learning criteria, whether learners can be assumed consistent.
Mapping Monotonic Restrictions in Inductive Inference
Venue: arXiv cs.LG 2020
Year: 2020

In language learning in the limit we investigate computable devices (learners) learning formal languages. Through the years, many natural restrictions have been imposed on the studied learners. As such, monotonic restrictions always enjoyed particular attention as, although being a natural requirement, monotonic learners show significantly diverse behaviour when studied in different settings. A recent study thoroughly analysed the learning capabilities of strongly monotone learners imposed with memory restrictions and various additional requirements. The unveiled differences between explanatory and behaviourally correct such learners motivate our studies of monotone learners dealing with the same restrictions. We reveal differences and similarities between monotone learners and their strongly monotone counterpart when studied with various additional restrictions. In particular, we show that explanatory monotone learners, although known to be strictly stronger, do (almost) preserve the pairwise relation as seen in strongly monotone learning. Contrasting this similarity, we find substantial differences when studying behaviourally correct monotone learners. Most notably, we show that monotone learners, as opposed to their strongly monotone counterpart, do heavily rely on the order the information is given in, an unusual result for behaviourally correct learners.
A Nesterov's Accelerated quasi-Newton method for Global Routing using Deep Reinforcement Learning
Venue: arXiv cs.LG 2020
Year: 2020

Deep Q-learning method is one of the most popularly used deep reinforcement learning algorithms which uses deep neural networks to approximate the estimation of the action-value function. Training of the deep Q-network (DQN) is usually restricted to first order gradient based methods. This paper attempts to accelerate the training of deep Q-networks by introducing a second order Nesterov's accelerated quasi-Newton method. We evaluate the performance of the proposed method on deep reinforcement learning using double DQNs for global routing. The results show that the proposed method can obtain better routing solutions compared to the DQNs trained with first order Adam and RMSprop methods.
It's the Best Only When It Fits You Most: Finding Related Models for Serving Based on Dynamic Locality Sensitive Hashing
Venue: arXiv cs.IR 2020
Year: 2020

In recent, deep learning has become the most popular direction in machine learning and artificial intelligence. However, preparation of training data is often a bottleneck in the lifecycle of deploying a deep learning model for production or research. Reusing models for inferencing a dataset can greatly save the human costs required for training data creation. Although there exist a number of model sharing platform such as TensorFlow Hub, PyTorch Hub, DLHub, most of these systems require model uploaders to manually specify the details of each model and model downloaders to screen keyword search results for selecting a model. They are in lack of an automatic model searching tool. This paper proposes an end-to-end process of searching related models for serving based on the similarity of the target dataset and the training datasets of the available models. While there exist many similarity measurements, we study how to efficiently apply these metrics without pair-wise comparison and compare the effectiveness of these metrics. We find that our proposed adaptivity measurement which is based on Jensen-Shannon (JS) divergence, is an effective measurement, and its computation can be significantly accelerated by using the technique of locality sensitive hashing.
Hector: Using Untrusted Browsers to Provision Web Applications
Venue: arXiv cs.CR 2020
Year: 2020

Web applications are on the rise and rapidly evolve into more and more mature replacements for their native counterparts. This disruptive trend is mainly driven by the attainment of platform-independence and instant deployability. On top of this, web browsers offer the opportunity for seamless browser-to-browser communication for distributed interaction. In this paper, we present Hector, a novel web application framework that transforms web browsers into a distributed application-centric computing platform. Hector enables offloading application logic to users, thereby improving user experience with lower latencies while generating less costs for service providers. Following the programming paradigm of Function-as-a-Service, applications are decomposed into functions so they can be managed efficiently and deployed in a responsive, scalable and lightweight fashion. In case of client-side resource shortage or unresponsive clients, execution falls back to a traditional cloud-based infrastructure. Hector combines WebAssembly for multi-language computations at near-native speed, WebRTC for browser-to-browser communication and trusted execution as provided by the Intel Software Guard Extensions so browsers can trust each other's computations. We evaluate Hector by implementing a digital assistant as well as a recommendation system. Our evaluation shows that Hector achieves lower end-user latencies while generating less costs than traditional deployments. Additionally, we show that Hector scales linearly with increasing client numbers and can cope well with unresponsive clients.
COSEA: Convolutional Code Search with Layer-wise Attention
Venue: arXiv cs.SE 2020
Year: 2020

Semantic code search, which aims to retrieve code snippets relevant to a given natural language query, has attracted many research efforts with the purpose of accelerating software development. The huge amount of online publicly available code repositories has prompted the employment of deep learning techniques to build state-of-the-art code search models. Particularly, they leverage deep neural networks to embed codes and queries into a unified semantic vector space and then use the similarity between code's and query's vectors to approximate the semantic correlation between code and the query. However, most existing studies overlook the code's intrinsic structural logic, which indeed contains a wealth of semantic information, and fails to capture intrinsic features of codes. In this paper, we propose a new deep learning architecture, COSEA, which leverages convolutional neural networks with layer-wise attention to capture the valuable code's intrinsic structural logic. To further increase the learning efficiency of COSEA, we propose a variant of contrastive loss for training the code search model, where the ground-truth code should be distinguished from the most similar negative sample. We have implemented a prototype of COSEA. Extensive experiments over existing public datasets of Python and SQL have demonstrated that COSEA can achieve significant improvements over state-of-the-art methods on code search tasks.
Parametric non-interference in timed automata
Venue: arXiv cs.FL 2020
Year: 2020

We consider a notion of non-interference for timed automata (TAs) that allows to quantify the frequency of an attack; that is, we infer values of the minimal time between two consecutive actions of the attacker, so that (s)he disturbs the set of reachable locations. We also synthesize valuations for the timing constants of the TA (seen as parameters) guaranteeing non-interference. We show that this can reduce to reachability synthesis in parametric timed automata. We apply our method to a model of the Fischer mutual exclusion protocol and obtain preliminary results.
Proximal-ADMM Decoder for Nonbinary LDPC Codes
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we develop an efficient decoder via the proximal alternating direction method of multipliers (proximal-ADMM) technique for nonbinary linear block codes in the Galois field. Its main contents are as follows: first, exploiting the decomposition technique based on the three-variables check equation, we formulate the maximum likelihood (ML) decoding problem approximately to a non-convex quadratic program; second, an efficient algorithm based on the proximal-ADMM technique is proposed to solve the formulated QP problem. Exploiting the QP problem's inherent structures, its variables can be updated in parallel; third, we prove that the proposed decoding algorithm can converge to some stationary point of the formulated QP problem. Moreover, we also show, for nonbinary low-density parity-check (LDPC) codes, its computational complexity in each proximal-ADMM iteration scales linearly with block length and the size of the considered Galois field. Simulation results demonstrate that the proposed proximal-ADMM decoder outperforms state-of-the-art nonbinary LDPC decoders in terms of either error correction performance or computational complexity.
RONELD: Robust Neural Network Output Enhancement for Active Lane Detection
Venue: arXiv cs.CV 2020
Year: 2020

Accurate lane detection is critical for navigation in autonomous vehicles, particularly the active lane which demarcates the single road space that the vehicle is currently traveling on. Recent state-of-the-art lane detection algorithms utilize convolutional neural networks (CNNs) to train deep learning models on popular benchmarks such as TuSimple and CULane. While each of these models works particularly well on train and test inputs obtained from the same dataset, the performance drops significantly on unseen datasets of different environments. In this paper, we present a real-time robust neural network output enhancement for active lane detection (RONELD) method to identify, track, and optimize active lanes from deep learning probability map outputs. We first adaptively extract lane points from the probability map outputs, followed by detecting curved and straight lanes before using weighted least squares linear regression on straight lanes to fix broken lane edges resulting from fragmentation of edge maps in real images. Lastly, we hypothesize true active lanes through tracking preceding frames. Experimental results demonstrate an up to two-fold increase in accuracy using RONELD on cross-dataset validation tests.
Learning to Reconstruct and Segment 3D Objects
Venue: arXiv cs.CV 2020
Year: 2020

To endow machines with the ability to perceive the real-world in a three dimensional representation as we do as humans is a fundamental and long-standing topic in Artificial Intelligence. Given different types of visual inputs such as images or point clouds acquired by 2D/3D sensors, one important goal is to understand the geometric structure and semantics of the 3D environment. Traditional approaches usually leverage hand-crafted features to estimate the shape and semantics of objects or scenes. However, they are difficult to generalize to novel objects and scenarios, and struggle to overcome critical issues caused by visual occlusions. By contrast, we aim to understand scenes and the objects within them by learning general and robust representations using deep neural networks, trained on large-scale real-world 3D data. To achieve these aims, this thesis makes three core contributions from object-level 3D shape estimation from single or multiple views to scene-level semantic understanding.
A General, Implicit, Large-Strain FE$^2$ Framework for the Simulation of Dynamic Problems on Two Scales
Venue: arXiv cs.CE 2020
Year: 2020

In this paper we present a fully-coupled, two-scale homogenization method for dynamic loading in the spirit of FE$^2$ methods. The framework considers the balance of linear momentum including inertia at the microscale to capture possible dynamic effects arising from micro heterogeneities. A finite-strain formulation is adapted to account for geometrical nonlinearities enabling the study of e.g. plasticity or fiber pullout, which may be associated with large deformations. A consistent kinematic scale link is established as displacement constraint on the whole representative volume element. The consistent macroscopic material tangent moduli are derived including micro inertia in closed form. These can easily be calculated with a loop over all microscopic finite elements, only applying existing assembly and solving procedures. Thus, making it suitable for standard finite element program architectures. Numerical examples of a layered periodic material are presented and compared to direct numerical simulations to demonstrate the capability of the proposed framework.
On the Parallel Composition for True Concurrency
Venue: arXiv cs.LO 2020
Year: 2020

For insight into the parallel composition for true concurrency, we recall the axiomatization of the parallel composition modulo truly concurrent behavioral equivalences as the sidelights of truly concurrent process algebra APTC. We prove that: (1) There is a finite sound and complete axiomatization of the parallel composition modulo pomset, step and hp-bisimulations, without any auxiliary operators. (2) There does not exist a finite sound and complete axiomatization of the parallel composition modulo hhp-bisimulation, without any auxiliary operator. (3) There is a finite sound and complete axiomatization of the parallel composition modulo pomset, step, hp-. and hhp-bisimulations, with the auxiliary left parallel composition and communication merge.
A Survey of Machine Learning Techniques in Adversarial Image Forensics
Venue: arXiv cs.CR 2020
Year: 2020

Image forensic plays a crucial role in both criminal investigations (e.g., dissemination of fake images to spread racial hate or false narratives about specific ethnicity groups) and civil litigation (e.g., defamation). Increasingly, machine learning approaches are also utilized in image forensics. However, there are also a number of limitations and vulnerabilities associated with machine learning-based approaches, for example how to detect adversarial (image) examples, with real-world consequences (e.g., inadmissible evidence, or wrongful conviction). Therefore, with a focus on image forensics, this paper surveys techniques that can be used to enhance the robustness of machine learning-based binary manipulation detectors in various adversarial scenarios.