OpenRaiser/Intern-Atlas huggingface dataset:
Deep generative LDA
Venue: arXiv cs.LG 2020
Year: 2020

Linear discriminant analysis (LDA) is a popular tool for classification and dimension reduction. Limited by its linear form and the underlying Gaussian assumption, however, LDA is not applicable in situations where the data distribution is complex. Recently, we proposed a discriminative normalization flow (DNF) model. In this study, we reinterpret DNF as a deep generative LDA model, and study its properties in representing complex data. We conducted a simulation experiment and a speaker recognition experiment. The results show that DNF and its subspace version are much more powerful than the conventional LDA in modeling complex data and retrieving low-dimensional representations.
Polar Coded Repetition for Low-Capacity Channels
Venue: arXiv cs.IT 2020
Year: 2020

Constructing efficient low-rate error-correcting codes with low-complexity encoding and decoding have become increasingly important for applications involving ultra-low-power devices such as Internet-of-Things (IoT) networks. To this end, schemes based on concatenating the state-of-the-art codes at moderate rates with repetition codes have emerged as practical solutions deployed in various standards. In this paper, we propose a novel mechanism for concatenating outer polar codes with inner repetition codes which we refer to as polar coded repetition. More specifically, we propose to transmit a slightly modified polar codeword by deviating from Arikan's standard 2 x 2 Kernel in a certain number of polarization recursions at each repetition block. We show how this modification can improve the asymptotic achievable rate of the polar-repetition scheme, while ensuring that the overall encoding and decoding complexity is kept almost the same. The achievable rate is analyzed for the binary erasure channels (BEC).
Thinking About Causation: A Causal Language with Epistemic Operators
Venue: arXiv cs.AI 2020
Year: 2020

This paper proposes a formal framework for modeling the interaction of causal and (qualitative) epistemic reasoning. To this purpose, we extend the notion of a causal model with a representation of the epistemic state of an agent. On the side of the object language, we add operators to express knowledge and the act of observing new information. We provide a sound and complete axiomatization of the logic, and discuss the relation of this framework to causal team semantics.
How Does Performance Scale with Antenna Number for Extremely Large-Scale MIMO?
Venue: arXiv cs.IT 2020
Year: 2020

Extremely large-scale multiple-input multiple-output (XL-MIMO) communications correspond to systems whose antenna size is so large that conventional assumptions, such as uniform plane wave (UPW) impingement, are no longer valid. This paper studies the channel modelling and performance analysis of XL-MIMO communication based on the generic spherical wavefront propagation model. First, for the single-user uplink/downlink communication with the optimal maximum ratio combining/transmission (MRC/MRT), we rigorously derive a new closed-form expression for the resulting signal-to-noise ratio (SNR), which includes the conventional SNR expression based on UPW assumption as a special case. Our result shows that instead of scaling linearly with the base station (BS) antenna number $M$, the SNR with the more generic spherical wavefront model increases with $M$ with diminishing return, governed by a new parameter called angular span. One important finding from our derivation is the necessity to introduce a new distance criterion, termed critical distance, to complement the classical Rayleigh distance for separating the near- and far-field propagation regions. While Rayleigh distance is based on the phase difference across array elements and hence depends on the electrical size of the antenna, the critical distance cares about the amplitude/power difference and only depends on its physical size. We then extend the study to the multi-user XL-MIMO communication system, for which we demonstrate that inter-user interference (IUI) can be mitigated not just by angle separation, but also by distance separation along the same direction. This offers one new degree of freedom (DoF) for interference suppression with XL-MIMO.
Artificial Intelligence: Research Impact on Key Industries; the Upper-Rhine Artificial Intelligence Symposium (UR-AI 2020)
Venue: arXiv cs.AI 2020
Year: 2020

The TriRhenaTech alliance presents a collection of accepted papers of the cancelled tri-national 'Upper-Rhine Artificial Inteeligence Symposium' planned for 13th May 2020 in Karlsruhe. The TriRhenaTech alliance is a network of universities in the Upper-Rhine Trinational Metropolitan Region comprising of the German universities of applied sciences in Furtwangen, Kaiserslautern, Karlsruhe, and Offenburg, the Baden-Wuerttemberg Cooperative State University Loerrach, the French university network Alsace Tech (comprised of 14 'grandes écoles' in the fields of engineering, architecture and management) and the University of Applied Sciences and Arts Northwestern Switzerland. The alliance's common goal is to reinforce the transfer of knowledge, research, and technology, as well as the cross-border mobility of students.
Advanced Join Patterns for the Actor Model based on CEP Techniques
Venue: arXiv cs.PL 2020
Year: 2020

Context: Actor-based programming languages offer many essential features for developing modern distributed reactive systems. These systems exploit the actor model's isolation property to fulfill their performance and scalability demands. Unfortunately, the reliance of the model on isolation as its most fundamental property requires programmers to express complex interaction patterns between their actors to be expressed manually in terms of complex combinations of messages sent between the isolated actors. Inquiry: In the last three decades, several language design proposals have been introduced to reduce the complexity that emerges from describing said interaction and coordination of actors. We argue that none of these proposals is satisfactory in order to express the many complex interaction patterns between actors found in modern reactive distributed systems. Approach: We describe seven smart home automation scenarios (in which an actor represents every smart home appliance) to motivate the support by actor languages for five radically different types of message synchronization patterns, which are lacking in modern distributed actor-based languages. Fortunately, these five types of synchronisation patterns have been studied extensively by the Complex Event Processing (CEP) community. Our paper describes how such CEP patterns are elegantly added to an actor-based programming language. Knowledge: Based on our findings, we propose an extension of the single-message matching paradigm of contemporary actor-based languages in order to support a multiple-message matching way of thinking in the same way as proposed by CEP languages. Our proposal thus enriches the actor-model by ways of declaratively describing complex message combinations to which an actor can respond. Grounding: We base the problem-statement of the paper on an online poll in the home automation community that has motivated the real need for the CEP-based synchronisation operators between actors proposed in the paper. Furthermore, we implemented a DSL -- called Sparrow -- that supports said operators and we argue quantitatively (in terms of LOC and in terms of a reduction of the concerns that have to be handled by programmers) that the DSL outperforms existing approaches. Importance: This work aims to provide a set of synchronization operators that help actor-based languages to handle the complex interaction required by modern reactive distributed systems. To the best of our knowledge, our proposal is the first one to add advanced CEP synchronization operators to the -- relatively simplistic single-message based matching -- mechanisms of most actor-based languages.
Being Single Has Benefits. Instance Poisoning to Deceive Malware Classifiers
Venue: arXiv cs.CR 2020
Year: 2020

The performance of a machine learning-based malware classifier depends on the large and updated training set used to induce its model. In order to maintain an up-to-date training set, there is a need to continuously collect benign and malicious files from a wide range of sources, providing an exploitable target to attackers. In this study, we show how an attacker can launch a sophisticated and efficient poisoning attack targeting the dataset used to train a malware classifier. The attacker's ultimate goal is to ensure that the model induced by the poisoned dataset will be unable to detect the attacker's malware yet capable of detecting other malware. As opposed to other poisoning attacks in the malware detection domain, our attack does not focus on malware families but rather on specific malware instances that contain an implanted trigger, reducing the detection rate from 99.23% to 0% depending on the amount of poisoning. We evaluate our attack on the EMBER dataset with a state-of-the-art classifier and malware samples from VirusTotal for end-to-end validation of our work. We propose a comprehensive detection approach that could serve as a future sophisticated defense against this newly discovered severe threat.
Topic-Preserving Synthetic News Generation: An Adversarial Deep Reinforcement Learning Approach
Venue: arXiv cs.CL 2020
Year: 2020

Nowadays, there exist powerful language models such as OpenAI's GPT-2 that can generate readable text and can be fine-tuned to generate text for a specific domain. Considering GPT-2, it cannot directly generate synthetic news with respect to a given topic and the output of the language model cannot be explicitly controlled. In this paper, we study the novel problem of topic-preserving synthetic news generation. We propose a novel deep reinforcement learning-based method to control the output of GPT-2 with respect to a given news topic. When generating text using GPT-2, by default, the most probable word is selected from the vocabulary. Instead of selecting the best word each time from GPT-2's output, an RL agent tries to select words that optimize the matching of a given topic. In addition, using a fake news detector as an adversary, we investigate generating realistic news using our proposed method. In this paper, we consider realistic news as news that cannot be easily detected by a fake news classifier. Experimental results demonstrate the effectiveness of the proposed framework on generating topic-preserving news content than state-of-the-art baselines.
Patterns Count-Based Labels for Datasets
Venue: arXiv cs.DB 2020
Year: 2020

Counts of attribute-value combinations are central to the profiling of a dataset, particularly in determining fitness for use and in eliminating bias and unfairness. While counts of individual attribute values may be stored in some dataset profiles, there are too many combinations of attributes for it to be practical to store counts for each combination. In this paper, we develop the notion of storing a "label" of limited size that can be used to obtain good estimates for these counts. A label, in this paper, contains information regarding the count of selected patterns--attributes values combinations--in the data. We define an estimation function, that uses this label to estimate the count of every pattern. We present the problem of finding the optimal label given a bound on its size and propose a heuristic algorithm for generating optimal labels. We experimentally show the accuracy of count estimates derived from the resulting labels and the efficiency of our algorithm.
A New Neural Search and Insights Platform for Navigating and Organizing AI Research
Venue: arXiv cs.CL 2020
Year: 2020

To provide AI researchers with modern tools for dealing with the explosive growth of the research literature in their field, we introduce a new platform, AI Research Navigator, that combines classical keyword search with neural retrieval to discover and organize relevant literature. The system provides search at multiple levels of textual granularity, from sentences to aggregations across documents, both in natural language and through navigation in a domain-specific Knowledge Graph. We give an overview of the overall architecture of the system and of the components for document analysis, question answering, search, analytics, expert search, and recommendations.
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour
Venue: arXiv cs.LG 2020
Year: 2020

EfficientNets are a family of state-of-the-art image classification models based on efficiently scaled convolutional neural networks. Currently, EfficientNets can take on the order of days to train; for example, training an EfficientNet-B0 model takes 23 hours on a Cloud TPU v2-8 node. In this paper, we explore techniques to scale up the training of EfficientNets on TPU-v3 Pods with 2048 cores, motivated by speedups that can be achieved when training at such scales. We discuss optimizations required to scale training to a batch size of 65536 on 1024 TPU-v3 cores, such as selecting large batch optimizers and learning rate schedules as well as utilizing distributed evaluation and batch normalization techniques. Additionally, we present timing and performance benchmarks for EfficientNet models trained on the ImageNet dataset in order to analyze the behavior of EfficientNets at scale. With our optimizations, we are able to train EfficientNet on ImageNet to an accuracy of 83% in 1 hour and 4 minutes.
Resource-Aware Pareto-Optimal Automated Machine Learning Platform
Venue: arXiv cs.LG 2020
Year: 2020

In this study, we introduce a novel platform Resource-Aware AutoML (RA-AutoML) which enables flexible and generalized algorithms to build machine learning models subjected to multiple objectives, as well as resource and hard-ware constraints. RA-AutoML intelligently conducts Hyper-Parameter Search(HPS) as well as Neural Architecture Search (NAS) to build models optimizing predefined objectives. RA-AutoML is a versatile framework that allows user to prescribe many resource/hardware constraints along with objectives demanded by the problem at hand or business requirements. At its core, RA-AutoML relies on our in-house search-engine algorithm,MOBOGA, which combines a modified constraint-aware Bayesian Optimization and Genetic Algorithm to construct Pareto optimal candidates. Our experiments on CIFAR-10 dataset shows very good accuracy compared to results obtained by state-of-art neural network models, while subjected to resource constraints in the form of model size.
Low Latency Cross-Shard Transactions in Coded Blockchain
Venue: arXiv cs.CR 2020
Year: 2020

Although blockchain, the supporting technology of Bitcoin and various cryptocurrencies, has offered a potentially effective framework for numerous applications, it still suffers from the adverse affects of the impossibility triangle. Performance, security, and decentralization of blockchains normally do not scale simultaneously with the number of participants in the network. The recent introduction of error correcting codes in sharded blockchain by Li et al. partially settles this trilemma, boosting throughput without compromising security and decentralization. In this paper, we improve the coded sharding scheme in three ways. First, we propose a novel 2-Dimensional Sharding strategy, which inherently supports cross-shard transactions, alleviating the need for complicated inter-shard communication protocols. Second, we employ distributed storage techniques in the propagation of blocks, improving latency under restricted bandwidth. Finally, we incorporate polynomial cryptographic primitives of low degree, which brings coded blockchain techniques into the realm of feasible real-world parameters.
Evaluation of Inference Attack Models for Deep Learning on Medical Data
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning has attracted broad interest in healthcare and medical communities. However, there has been little research into the privacy issues created by deep networks trained for medical applications. Recently developed inference attack algorithms indicate that images and text records can be reconstructed by malicious parties that have the ability to query deep networks. This gives rise to the concern that medical images and electronic health records containing sensitive patient information are vulnerable to these attacks. This paper aims to attract interest from researchers in the medical deep learning community to this important problem. We evaluate two prominent inference attack models, namely, attribute inference attack and model inversion attack. We show that they can reconstruct real-world medical images and clinical reports with high fidelity. We then investigate how to protect patients' privacy using defense mechanisms, such as label perturbation and model perturbation. We provide a comparison of attack results between the original and the medical deep learning models with defenses. The experimental evaluations show that our proposed defense approaches can effectively reduce the potential privacy leakage of medical deep learning from the inference attacks.
Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution
Venue: arXiv cs.CV 2020
Year: 2020

3D human pose estimation (HPE) is crucial in many fields, such as human behavior analysis, augmented reality/virtual reality (AR/VR) applications, and self-driving industry. Videos that contain multiple potentially occluded people captured from freely moving monocular cameras are very common in real-world scenarios, while 3D HPE for such scenarios is quite challenging, partially because there is a lack of such data with accurate 3D ground truth labels in existing datasets. In this paper, we propose a temporal regression network with a gated convolution module to transform 2D joints to 3D and recover the missing occluded joints in the meantime. A simple yet effective localization approach is further conducted to transform the normalized pose to the global trajectory. To verify the effectiveness of our approach, we also collect a new moving camera multi-human (MMHuman) dataset that includes multiple people with heavy occlusion captured by moving cameras. The 3D ground truth joints are provided by accurate motion capture (MoCap) system. From the experiments on static-camera based Human3.6M data and our own collected moving-camera based data, we show that our proposed method outperforms most state-of-the-art 2D-to-3D pose estimation methods, especially for the scenarios with heavy occlusions.
A Novel Semi-Supervised Data-Driven Method for Chiller Fault Diagnosis with Unlabeled Data
Venue: arXiv cs.LG 2020
Year: 2020

In practical chiller systems, applying efficient fault diagnosis techniques can significantly reduce energy consumption and improve energy efficiency of buildings. The success of the existing methods for fault diagnosis of chillers relies on the condition that sufficient labeled data are available for training. However, label acquisition is laborious and costly in practice. Usually, the number of labeled data is limited and most data available are unlabeled. The existing methods cannot exploit the information contained in unlabeled data, which significantly limits the improvement of fault diagnosis performance in chiller systems. To make effective use of unlabeled data to further improve fault diagnosis performance and reduce the dependency on labeled data, we proposed a novel semi-supervised data-driven fault diagnosis method for chiller systems based on the semi-generative adversarial network, which incorporates both unlabeled and labeled data into learning process. The semi-generative adversarial network can learn the information of data distribution from unlabeled data and this information can help to significantly improve the diagnostic performance. Experimental results demonstrate the effectiveness of the proposed method. Under the scenario that there are only 80 labeled samples and 16000 unlabeled samples, the proposed method can improve the diagnostic accuracy to 84%, while the supervised baseline methods only reach the accuracy of 65% at most. Besides, the minimal required number of labeled samples can be reduced by about 60% with the proposed method when there are enough unlabeled samples.
Finding the Near Optimal Policy via Adaptive Reduced Regularization in MDPs
Venue: arXiv cs.LG 2020
Year: 2020

Regularized MDPs serve as a smooth version of original MDPs. However, biased optimal policy always exists for regularized MDPs. Instead of making the coefficientλof regularized term sufficiently small, we propose an adaptive reduction scheme for λ to approximate optimal policy of the original MDP. It is shown that the iteration complexity for obtaining anε-optimal policy could be reduced in comparison with setting sufficiently smallλ. In addition, there exists strong duality connection between the reduction method and solving the original MDP directly, from which we can derive more adaptive reduction method for certain algorithms.
GTOC X: Karmarkar's Gang's Approach and Results
Venue: arXiv cs.RO 2020
Year: 2020

This paper describes the methods used and the results obtained by team Karmarkar's Gang for the 10th edition of the Global Trajectory Optimization Competition. The methods used by our team are described. These methods involve: mothership targeting flyby to target high-value stars using a single impulse, fast ships targeting the edge of the galaxy and settler ships having a fast expansion, time-optimal three impulse transfer strategy to select the targets to get fast and wide spatial distribution. It is seen that there is a scope of improvement with respect to multi-star targeting for motherships. The results of the strategy are discussed.
Optimizing Waste Management Collection Routes in Urban Haiti: A Collaboration between DataKind and SOIL
Venue: arXiv cs.CY 2020
Year: 2020

Sustainable Organic Integrated Livelihoods (SOIL) is a research and development organization that aims to increase access to cost-effective household sanitation services in urban communities in Haiti. Each week, SOIL provides over 1,000 households with ecological sanitation toilets, then transports the waste to be transformed into rich compost. However, SOIL faces several challenges regarding the route optimization of their mixed fleet vehicle routing. This paper builds upon the authors' submission to Bloomberg's 2019 Data for Good Exchange (D4GX), presenting preliminary findings from a joint collaboration between DataKind, a data science nonprofit, and SOIL. This research showcases how optimization algorithms and open source tools (i.e., OpenStreetMap and Google OR-Tools) can help improve and reduce the costs of mixed-fleet routing problems, particularly in the context of developing countries. As a result of this work, SOIL is able to make improvement to their collection routes, which account for different road conditions and vehicle types. These improvements reduce operational costs and fuel use, which are essential to the service's expansion in the coming years.
Pose Estimation of Specular and Symmetrical Objects
Venue: arXiv cs.RO 2020
Year: 2020

In the robotic industry, specular and textureless metallic components are ubiquitous. The 6D pose estimation of such objects with only a monocular RGB camera is difficult because of the absence of rich texture features. Furthermore, the appearance of specularity heavily depends on the camera viewpoint and environmental light conditions making traditional methods, like template matching, fail. In the last 30 years, pose estimation of the specular object has been a consistent challenge, and most related works require massive knowledge modeling effort for light setups, environment, or the object surface. On the other hand, recent works exhibit the feasibility of 6D pose estimation on a monocular camera with convolutional neural networks(CNNs) however they mostly use opaque objects for evaluation. This paper provides a data-driven solution to estimate the 6D pose of specular objects for grasping them, proposes a cost function for handling symmetry, and demonstrates experimental results showing the system's feasibility.