OpenRaiser/Intern-Atlas huggingface dataset:
$h_{PI}$: The Citation Index for Principal Investigators
Venue: arXiv cs.DL 2018
Year: 2018

A new citation index $h_{PI}$ for principal investigators (PIs) is defined in analogy to Hirsch's index $h$, but based on renormalized citations of a PI's papers. To this end, the authors of a paper are divided into two groups: PIs and non-PIs. A PI is defined as an assistant, associate or full professor at a university who supervises an individual research program. The citations for each paper of a certain PI are then divided by the number of PIs among the authors of that paper. Data are presented for a sample of 48 PIs who are senior faculty members of physics and physics-related engineering departments at a private research-oriented U.S. university, using the ISI Web of Science citations database. The main result is that individual rankings based on $h$ and $h_{PI}$ differ substantially. Also, to a good approximation across the sample of 48 PIs, one finds that $h_{PI} = h \,/ \sqrt{<N_{PI}>}$ where <$N_{PI}$> is the average number of principal investigators on the papers of a particular PI. In addition, $h_{PI} = \frac{1}{2} \sqrt{C_{tot}\,/<N_{PI}>}$, where $C_{tot}$ is the total number of citations. Approaches to broadening the scope of $h$ or $h_{PI}$ are discussed briefly, and a new metric for highly cited papers called $h_x$ is introduced which represents the average number of citations exceeding the minimum of $h^2$ in the $h$-core.
Bridging the Gap Between Layout Pattern Sampling and Hotspot Detection via Batch Active Learning
Venue: arXiv cs.LG 2018
Year: 2018

Layout hotpot detection is one of the main steps in modern VLSI design. A typical hotspot detection flow is extremely time consuming due to the computationally expensive mask optimization and lithographic simulation. Recent researches try to facilitate the procedure with a reduced flow including feature extraction, training set generation and hotspot detection, where feature extraction methods and hotspot detection engines are deeply studied. However, the performance of hotspot detectors relies highly on the quality of reference layout libraries which are costly to obtain and usually predetermined or randomly sampled in previous works. In this paper, we propose an active learning-based layout pattern sampling and hotspot detection flow, which simultaneously optimizes the machine learning model and the training set that aims to achieve similar or better hotspot detection performance with much smaller number of training instances. Experimental results show that our proposed method can significantly reduce lithography simulation overhead while attaining satisfactory detection accuracy on designs under both DUV and EUV lithography technologies.
Dynamic Sampling from Graphical Models
Venue: arXiv cs.DS 2018
Year: 2018

In this paper, we study the problem of sampling from a graphical model when the model itself is changing dynamically with time. This problem derives its interest from a variety of inference, learning, and sampling settings in machine learning, computer vision, statistical physics, and theoretical computer science. While the problem of sampling from a static graphical model has received considerable attention, theoretical works for its dynamic variants have been largely lacking. The main contribution of this paper is an algorithm that can sample dynamically from a broad class of graphical models over discrete random variables. Our algorithm is parallel and Las Vegas: it knows when to stop and it outputs samples from the exact distribution. We also provide sufficient conditions under which this algorithm runs in time proportional to the size of the update, on general graphical models as well as well-studied specific spin systems. In particular we obtain, for the Ising model (ferromagnetic or anti-ferromagnetic) and for the hardcore model the first dynamic sampling algorithms that can handle both edge and vertex updates (addition, deletion, change of functions), both efficient within regimes that are close to the respective uniqueness regimes, beyond which, even for the static and approximate sampling, no local algorithms were known or the problem itself is intractable. Our dynamic sampling algorithm relies on a local resampling algorithm and a new "equilibrium" property that is shown to be satisfied by our algorithm at each step, and enables us to prove its correctness. This equilibrium property is robust enough to guarantee the correctness of our algorithm, helps us improve bounds on fast convergence on specific models, and should be of independent interest.
BAM: Bottleneck Attention Module
Venue: arXiv cs.CV 2018
Year: 2018

Recent advances in deep neural networks have been developed via architecture search for stronger representational power. In this work, we focus on the effect of attention in general deep neural networks. We propose a simple and effective attention module, named Bottleneck Attention Module (BAM), that can be integrated with any feed-forward convolutional neural networks. Our module infers an attention map along two separate pathways, channel and spatial. We place our module at each bottleneck of models where the downsampling of feature maps occurs. Our module constructs a hierarchical attention at bottlenecks with a number of parameters and it is trainable in an end-to-end manner jointly with any feed-forward models. We validate our BAM through extensive experiments on CIFAR-100, ImageNet-1K, VOC 2007 and MS COCO benchmarks. Our experiments show consistent improvement in classification and detection performances with various models, demonstrating the wide applicability of BAM. The code and models will be publicly available.
A framework for remote sensing images processing using deep learning technique
Venue: arXiv cs.CV 2018
Year: 2018

Deep learning techniques are becoming increasingly important to solve a number of image processing tasks. Among common algorithms, Convolutional Neural Networks and Recurrent Neural Networks based systems achieve state of the art results on satellite and aerial imagery in many applications. While these approaches are subject to scientific interest, there is currently no operational and generic implementation available at user-level for the remote sensing community. In this paper, we presents a framework enabling the use of deep learning techniques with remote sensing images and geospatial data. Our solution takes roots in two extensively used open-source libraries, the remote sensing image processing library Orfeo ToolBox, and the high performance numerical computation library TensorFlow. It can apply deep nets without restriction on images size and is computationally efficient, regardless hardware configuration.
A Deep Learning Driven Active Framework for Segmentation of Large 3D Shape Collections
Venue: arXiv cs.CV 2018
Year: 2018

High-level shape understanding and technique evaluation on large repositories of 3D shapes often benefit from additional information known about the shapes. One example of such information is the semantic segmentation of a shape into functional or meaningful parts. Generating accurate segmentations with meaningful segment boundaries is, however, a costly process, typically requiring large amounts of user time to achieve high quality results. In this paper we present an active learning framework for large dataset segmentation, which iteratively provides the user with new predictions by training new models based on already segmented shapes. Our proposed pipeline consists of three novel components. First, we a propose a fast and relatively accurate feature-based deep learning model to provide dataset-wide segmentation predictions. Second, we propose an information theory measure to estimate the prediction quality and for ordering subsequent fast and meaningful shape selection. Our experiments show that such suggestive ordering helps reduce users time and effort, produce high quality predictions, and construct a model that generalizes well. Finally, we provide effective segmentation refinement features to help the user quickly correct any incorrect predictions. We show that our framework is more accurate and in general more efficient than state-of-the-art, for massive dataset segmentation with while also providing consistent segment boundaries.
Regularized Zero-Forcing Precoding Aided Adaptive Coding and Modulation for Large-Scale Antenna Array Based Air-to-Air Communications
Venue: arXiv cs.IT 2018
Year: 2018

We propose a regularized zero-forcing transmit precoding (RZF-TPC) aided and distance-based adaptive coding and modulation (ACM) scheme to support aeronautical communication applications, by exploiting the high spectral efficiency of large-scale antenna arrays and link adaption. Our RZF-TPC aided and distance-based ACM scheme switches its mode according to the distance between the communicating aircraft. We derive the closed-form asymptotic signal-to-interference-plus-noise ratio (SINR) expression of the RZF-TPC for the aeronautical channel, which is Rician, relying on a non-centered channel matrix that is dominated by the deterministic line-of-sight component. The effects of both realistic channel estimation errors and of the co-channel interference are considered in the derivation of this approximate closed-form SINR formula. Furthermore, we derive the analytical expression of the optimal regularization parameter that minimizes the mean square detection error. The achievable throughput expression based on our asymptotic approximate SINR formula is then utilized as the design metric for the proposed RZF-TPC aided and distance-based ACM scheme. Monte-Carlo simulation results are presented for validating our theoretical analysis as well as for investigating the impact of the key system parameters. The simulation results closely match the theoretical results. In the specific example that two communicating aircraft fly at a typical cruising speed of 920 km/h, heading in opposite direction over the distance up to 740 km taking a period of about 24 minutes, the RZF-TPC aided and distance-based ACM is capable of transmitting a total of 77 Gigabyte of data with the aid of 64 transmit antennas and 4 receive antennas, which is significantly higher than that of our previous eigen-beamforming transmit precoding aided and distance-based ACM benchmark.
Merlin: A Language Server for OCaml (Experience Report)
Venue: arXiv cs.PL 2018
Year: 2018

We report on the experience of developing Merlin, a language server for the OCaml programming language in development since 2013. Merlin is a daemon that connects to your favourite text editor and provides services that require a fine-grained understanding of the programming language syntax and static semantics: instant feedback on warnings and errors, autocompletion, `type of the code under the cursor', `go to definition', etc. Language servers need to handle incomplete and partially-incorrect programs, and try to be incremental to minimize recomputation after small editing actions. Merlin was built by carefully adapting the existing tools (the OCamllex lexer and Menhir parser generators) to better support incrementality, incompleteness and error handling. These extensions are elegant and general, as demonstrated by the interesting, unplanned uses that the OCaml community found for them. They could be adapted to other frontends -- in any language. Besides incrementality, we discuss the way Merlin communicates with editors, describe the design decisions that went into some demanding features and report on some of the non-apparent difficulties in building good editor support, emerging from expressive programming languages or frustrating tooling ecosystems. We expect this experience report to be of interest to authors of interactive language tooling for any programming language; many design choices may be reused, and some hard-won lessons can serve as warnings.
Encrypted Control System with Quantizer
Venue: arXiv cs.SY 2018
Year: 2018

This paper considers the design of encrypted control systems to secure data privacy when the control systems operate over a network. In particular, we propose to combine Paillier cryptosystem with a quantizer whose sensitivity changes with the evolution of the system. This allows the encrypted control system to balance between the cipher strength and processing time. Such an ability is essential for control systems that are expected to run real-time. It also allows the closed-loop system to achieve the asymptotic stability for linear systems. Extensions to event-triggered control and nonlinear control systems are also discussed.
Motivating the Rules of the Game for Adversarial Example Research
Venue: arXiv cs.LG 2018
Year: 2018

Advances in machine learning have led to broad deployment of systems with impressive performance on important problems. Nonetheless, these systems can be induced to make errors on data that are surprisingly similar to examples the learned system handles correctly. The existence of these errors raises a variety of questions about out-of-sample generalization and whether bad actors might use such examples to abuse deployed systems. As a result of these security concerns, there has been a flurry of recent papers proposing algorithms to defend against such malicious perturbations of correctly handled examples. It is unclear how such misclassifications represent a different kind of security problem than other errors, or even other attacker-produced examples that have no specific relationship to an uncorrupted input. In this paper, we argue that adversarial example defense papers have, to date, mostly considered abstract, toy games that do not relate to any specific security concern. Furthermore, defense papers have not yet precisely described all the abilities and limitations of attackers that would be relevant in practical security. Towards this end, we establish a taxonomy of motivations, constraints, and abilities for more plausible adversaries. Finally, we provide a series of recommendations outlining a path forward for future work to more clearly articulate the threat model and perform more meaningful evaluation.
BOLT: A Practical Binary Optimizer for Data Centers and Beyond
Venue: arXiv cs.PL 2018
Year: 2018

Performance optimization for large-scale applications has recently become more important as computation continues to move towards data centers. Data-center applications are generally very large and complex, which makes code layout an important optimization to improve their performance. This has motivated recent investigation of practical techniques to improve code layout at both compile time and link time. Although post-link optimizers had some success in the past, no recent work has explored their benefits in the context of modern data-center applications. In this paper, we present BOLT, a post-link optimizer built on top of the LLVM framework. Utilizing sample-based profiling, BOLT boosts the performance of real-world applications even for highly optimized binaries built with both feedback-driven optimizations (FDO) and link-time optimizations (LTO). We demonstrate that post-link performance improvements are complementary to conventional compiler optimizations, even when the latter are done at a whole-program level and in the presence of profile information. We evaluated BOLT on both Facebook data-center workloads and open-source compilers. For data-center applications, BOLT achieves up to 8.0% performance speedups on top of profile-guided function reordering and LTO. For the GCC and Clang compilers, our evaluation shows that BOLT speeds up their binaries by up to 20.4% on top of FDO and LTO, and up to 52.1% if the binaries are built without FDO and LTO.
Gradient Band-based Adversarial Training for Generalized Attack Immunity of A3C Path Finding
Venue: arXiv cs.LG 2018
Year: 2018

As adversarial attacks pose a serious threat to the security of AI system in practice, such attacks have been extensively studied in the context of computer vision applications. However, few attentions have been paid to the adversarial research on automatic path finding. In this paper, we show dominant adversarial examples are effective when targeting A3C path finding, and design a Common Dominant Adversarial Examples Generation Method (CDG) to generate dominant adversarial examples against any given map. In addition, we propose Gradient Band-based Adversarial Training, which trained with a single randomly choose dominant adversarial example without taking any modification, to realize the "1:N" attack immunity for generalized dominant adversarial examples. Extensive experimental results show that, the lowest generation precision for CDG algorithm is 91.91%, and the lowest immune precision for Gradient Band-based Adversarial Training is 93.89%, which can prove that our method can realize the generalized attack immunity of A3C path finding with a high confidence.
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
Venue: arXiv cs.LG 2018
Year: 2018

RMSProp and ADAM continue to be extremely popular algorithms for training neural nets but their theoretical convergence properties have remained unclear. Further, recent work has seemed to suggest that these algorithms have worse generalization properties when compared to carefully tuned stochastic gradient descent or its momentum variants. In this work, we make progress towards a deeper understanding of ADAM and RMSProp in two ways. First, we provide proofs that these adaptive gradient algorithms are guaranteed to reach criticality for smooth non-convex objectives, and we give bounds on the running time. Next we design experiments to empirically study the convergence and generalization properties of RMSProp and ADAM against Nesterov's Accelerated Gradient method on a variety of common autoencoder setups and on VGG-9 with CIFAR-10. Through these experiments we demonstrate the interesting sensitivity that ADAM has to its momentum parameter $β_1$. We show that at very high values of the momentum parameter ($β_1 = 0.99$) ADAM outperforms a carefully tuned NAG on most of our experiments, in terms of getting lower training and test losses. On the other hand, NAG can sometimes do better when ADAM's $β_1$ is set to the most commonly used value: $β_1 = 0.9$, indicating the importance of tuning the hyperparameters of ADAM to get better generalization performance. We also report experiments on different autoencoders to demonstrate that NAG has better abilities in terms of reducing the gradient norms, and it also produces iterates which exhibit an increasing trend for the minimum eigenvalue of the Hessian of the loss function at the iterates.
Robust Distributed Compression of Symmetrically Correlated Gaussian Sources
Venue: arXiv cs.IT 2018
Year: 2018

Consider a lossy compression system with $\ell$ distributed encoders and a centralized decoder. Each encoder compresses its observed source and forwards the compressed data to the decoder for joint reconstruction of the target signals under the mean squared error distortion constraint. It is assumed that the observed sources can be expressed as the sum of the target signals and the corruptive noises, which are generated independently from two symmetric multivariate Gaussian distributions. Depending on the parameters of such distributions, the rate-distortion limit of this system is characterized either completely or at least for sufficiently low distortions. The results are further extended to the robust distributed compression setting, where the outputs of a subset of encoders may also be used to produce a non-trivial reconstruction of the corresponding target signals. In particular, we obtain in the high-resolution regime a precise characterization of the minimum achievable reconstruction distortion based on the outputs of $k+1$ or more encoders when every $k$ out of all $\ell$ encoders are operated collectively in the same mode that is greedy in the sense of minimizing the distortion incurred by the reconstruction of the corresponding $k$ target signals with respect to the average rate of these $k$ encoders.
Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
Venue: arXiv cs.LG 2018
Year: 2018

While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few epochs during the main NAS and much larger numbers of epochs during a post-processing step is inefficient due to little correlation in the relative rankings for these two training regimes. To combat both of these problems, we propose to use a recent combination of Bayesian optimization and Hyperband for efficient joint neural architecture and hyperparameter search.
Structuring the Synthesis of Heap-Manipulating Programs - Extended Version
Venue: arXiv cs.PL 2018
Year: 2018

This paper describes a deductive approach to synthesizing imperative programs with pointers from declarative specifications expressed in Separation Logic. Our synthesis algorithm takes as input a pair of assertions---a pre- and a postcondition---which describe two states of the symbolic heap, and derives a program that transforms one state into the other, guided by the shape of the heap. The program synthesis algorithm rests on the novel framework of Synthetic Separation Logic (SSL), which generalises the classical notion of heap entailment $\mathcal{P} \vdash \mathcal{Q}$ to incorporate a possibility of transforming a heap satisfying an assertion $\mathcal{P}$ into a heap satisfying an assertion $\mathcal{Q}$. A synthesized program represents a proof term for a transforming entailment statement $\mathcal{P} \leadsto \mathcal{Q}$, and the synthesis procedure corresponds to a proof search. The derived programs are, thus, correct by construction, in the sense that they satisfy the ascribed pre/postconditions, and are accompanied by complete proof derivations, which can be checked independently. We have implemented a proof search engine for SSL in a form the program synthesizer called SuSLik. For efficiency, the engine exploits properties of SSL rules, such as invertibility and commutativity of rule applications on separate heaps, to prune the space of derivations it has to consider. We explain and showcase the use of SSL on characteristic examples, describe the design of SuSLik, and report on our experience of using it to synthesize a series of benchmark programs manipulating heap-based linked data structures.
Skeletal Movement to Color Map: A Novel Representation for 3D Action Recognition with Inception Residual Networks
Venue: arXiv cs.CV 2018
Year: 2018

We propose a novel skeleton-based representation for 3D action recognition in videos using Deep Convolutional Neural Networks (D-CNNs). Two key issues have been addressed: First, how to construct a robust representation that easily captures the spatial-temporal evolutions of motions from skeleton sequences. Second, how to design D-CNNs capable of learning discriminative features from the new representation in a effective manner. To address these tasks, a skeletonbased representation, namely, SPMF (Skeleton Pose-Motion Feature) is proposed. The SPMFs are built from two of the most important properties of a human action: postures and their motions. Therefore, they are able to effectively represent complex actions. For learning and recognition tasks, we design and optimize new D-CNNs based on the idea of Inception Residual networks to predict actions from SPMFs. Our method is evaluated on two challenging datasets including MSR Action3D and NTU-RGB+D. Experimental results indicated that the proposed method surpasses state-of-the-art methods whilst requiring less computation.