OpenRaiser/Intern-Atlas huggingface dataset:
Knowledge Elicitation using Deep Metric Learning and Psychometric Testing
Venue: arXiv cs.LG 2020
Year: 2020

Knowledge present in a domain is well expressed as relationships between corresponding concepts. For example, in zoology, animal species form complex hierarchies; in genomics, the different (parts of) molecules are organized in groups and subgroups based on their functions; plants, molecules, and astronomical objects all form complex taxonomies. Nevertheless, when applying supervised machine learning (ML) in such domains, we commonly reduce the complex and rich knowledge to a fixed set of labels, and induce a model shows good generalization performance with respect to these labels. The main reason for such a reductionist approach is the difficulty in eliciting the domain knowledge from the experts. Developing a label structure with sufficient fidelity and providing comprehensive multi-label annotation can be exceedingly labor-intensive in many real-world applications. In this paper, we provide a method for efficient hierarchical knowledge elicitation (HKE) from experts working with high-dimensional data such as images or videos. Our method is based on psychometric testing and active deep metric learning. The developed models embed the high-dimensional data in a metric space where distances are semantically meaningful, and the data can be organized in a hierarchical structure. We provide empirical evidence with a series of experiments on a synthetically generated dataset of simple shapes, and Cifar 10 and Fashion-MNIST benchmarks that our method is indeed successful in uncovering hierarchical structures.
A Primal-Dual Solver for Large-Scale Tracking-by-Assignment
Venue: arXiv cs.CV 2020
Year: 2020

We propose a fast approximate solver for the combinatorial problem known as tracking-by-assignment, which we apply to cell tracking. The latter plays a key role in discovery in many life sciences, especially in cell and developmental biology. So far, in the most general setting this problem was addressed by off-the-shelf solvers like Gurobi, whose run time and memory requirements rapidly grow with the size of the input. In contrast, for our method this growth is nearly linear. Our contribution consists of a new (1) decomposable compact representation of the problem; (2) dual block-coordinate ascent method for optimizing the decomposition-based dual; and (3) primal heuristics that reconstructs a feasible integer solution based on the dual information. Compared to solving the problem with Gurobi, we observe an up to~60~times speed-up, while reducing the memory footprint significantly. We demonstrate the efficacy of our method on real-world tracking problems.
Incorporating Uncertain Segmentation Information into Chinese NER for Social Media Text
Venue: arXiv cs.CL 2020
Year: 2020

Chinese word segmentation is necessary to provide word-level information for Chinese named entity recognition (NER) systems. However, segmentation error propagation is a challenge for Chinese NER while processing colloquial data like social media text. In this paper, we propose a model (UIcwsNN) that specializes in identifying entities from Chinese social media text, especially by leveraging ambiguous information of word segmentation. Such uncertain information contains all the potential segmentation states of a sentence that provides a channel for the model to infer deep word-level characteristics. We propose a trilogy (i.e., candidate position embedding -> position selective attention -> adaptive word convolution) to encode uncertain word segmentation information and acquire appropriate word-level representation. Experiments results on the social media corpus show that our model alleviates the segmentation error cascading trouble effectively, and achieves a significant performance improvement of more than 2% over previous state-of-the-art methods.
A Machine Learning-based Approach to Build Zero False-Positive IPSs for Industrial IoT and CPS with a Case Study on Power Grids Security
Venue: arXiv cs.CR 2020
Year: 2020

Intrusion Prevention Systems (IPS), have long been the first layer of defense against malicious attacks. Most sensitive systems employ instances of them (e.g. Firewalls) to secure the network perimeter and filter out attacks or unwanted traffic. A firewall, similar to classifiers, has a boundary to decide which traffic sample is normal and which one is not. This boundary is defined by configuration and is managed by a set of rules which occasionally might also filter normal traffic by mistake. However, for some applications, any interruption of the normal operation is not tolerable e.g. in power plants, water distribution systems, gas or oil pipelines, etc. In this paper, we design a learning firewall that receives labelled samples and configures itself automatically by writing preventive rules in a conservative way that avoids false alarms. We design a new family of classifiers, called $\mathfrak{z}$-classifiers, that unlike the traditional ones which merely target accuracy, rely on zero false-positive as the metric for decision making. First, we analytically show why naive modification of current classifiers like SVM does not yield acceptable results and then, propose a generic iterative algorithm to accomplish this goal. We use the proposed classifier with CART at its heart to build a firewall for a Power Grid Monitoring System. To further evaluate the algorithm, we additionally test it on KDD CUP'99 dataset. The results confirm the effectiveness of our approach.
RTLola Cleared for Take-Off: Monitoring Autonomous Aircraft
Venue: arXiv cs.RO 2020
Year: 2020

The autonomous control of unmanned aircraft is a highly safety-critical domain with great economic potential in a wide range of application areas, including logistics, agriculture, civil engineering, and disaster recovery. We report on the development of a dynamic monitoring framework for the DLR ARTIS (Autonomous Rotorcraft Testbed for Intelligent Systems) family of unmanned aircraft based on the formal specification language RTLola. RTLola is a stream-based specification language for real-time properties. An RTLola specification of hazardous situations and system failures is statically analyzed in terms of consistency and resource usage and then automatically translated into an FPGA-based monitor. Our approach leads to highly efficient, parallelized monitors with formal guarantees on the noninterference of the monitor with the normal operation of the autonomous system.
C++ Modules in ROOT and Beyond
Venue: arXiv cs.DC 2020
Year: 2020

C++ Modules come in C++20 to fix the long-standing build scalability problems in the language. They provide an io-efficient, on-disk representation capable to reduce build times and peak memory usage. ROOT employs the C++ modules technology further in the ROOT dictionary system to improve its performance and reduce the memory footprint. ROOT with C++ Modules was released as a technology preview in fall 2018, after intensive development during the last few years. The current state is ready for production, however, there is still room for performance optimizations. In this talk, we show the roadmap for making the technology default in ROOT. We demonstrate a global module indexing optimization which allows reducing the memory footprint dramatically for many workflows. We will report user feedback on the migration to ROOT with C++ Modules.
Hi Sigma, do I have the Coronavirus?: Call for a New Artificial Intelligence Approach to Support Health Care Professionals Dealing With The COVID-19 Pandemic
Venue: arXiv cs.CY 2020
Year: 2020

Just like your phone can detect what song is playing in crowded spaces, we show that Artificial Intelligence transfer learning algorithms trained on cough phone recordings results in diagnostic tests for COVID-19. To gain adoption by the health care community, we plan to validate our results in a clinical trial and three other venues in Mexico, Spain and the USA . However, if we had data from other on-going clinical trials and volunteers, we may do much more. For example, for confirmed stay-at-home COVID-19 patients, a longitudinal audio test could be developed to determine contact-with-hospital recommendations, and for the most critical COVID-19 patients a success ratio forecast test, including patient clinical data, to prioritize ICU allocation. As a challenge to the engineering community and in the context of our clinical trial, the authors suggest distributing cough recordings daily, hoping other trials and crowdsourcing users will contribute more data. Previous approaches to complex AI tasks have either used a static dataset or were private efforts led by large corporations. All existing COVID-19 trials published also follow this paradigm. Instead, we suggest a novel open collective approach to large-scale real-time health care AI. We will be posting updates at https://opensigma.mit.edu. Our personal view is that our approach is the right one for large scale pandemics, and therefore is here to stay - will you join?
Topology-Aware Hashing for Effective Control Flow Graph Similarity Analysis
Venue: arXiv cs.CR 2020
Year: 2020

Control Flow Graph (CFG) similarity analysis is an essential technique for a variety of security analysis tasks, including malware detection and malware clustering. Even though various algorithms have been developed, existing CFG similarity analysis methods still suffer from limited efficiency, accuracy, and usability. In this paper, we propose a novel fuzzy hashing scheme called topology-aware hashing (TAH) for effective and efficient CFG similarity analysis. Given the CFGs constructed from program binaries, we extract blended n-gram graphical features of the CFGs, encode the graphical features into numeric vectors (called graph signatures), and then measure the graph similarity by comparing the graph signatures. We further employ a fuzzy hashing technique to convert the numeric graph signatures into smaller fixed-size fuzzy hash signatures for efficient similarity calculation. Our comprehensive evaluation demonstrates that TAH is more effective and efficient compared to existing CFG comparison techniques. To demonstrate the applicability of TAH to real-world security analysis tasks, we develop a binary similarity analysis tool based on TAH, and show that it outperforms existing similarity analysis tools while conducting malware clustering.
A Tailored NSGA-III Instantiation for Flexible Job Shop Scheduling
Venue: arXiv cs.NE 2020
Year: 2020

A customized multi-objective evolutionary algorithm (MOEA) is proposed for the multi-objective flexible job shop scheduling problem (FJSP). It uses smart initialization approaches to enrich the first generated population, and proposes various crossover operators to create a better diversity of offspring. Especially, the MIP-EGO configurator, which can tune algorithm parameters, is adopted to automatically tune operator probabilities. Furthermore, different local search strategies are employed to explore the neighborhood for better solutions. In general, the algorithm enhancement strategy can be integrated with any standard EMO algorithm. In this paper, it has been combined with NSGA-III to solve benchmark multi-objective FJSPs, whereas an off-the-shelf implementation of NSGA-III is not capable of solving the FJSP. The experimental results show excellent performance with less computing budget.
Towards Scalable Security in Interference Channels With Arbitrary Number of Users
Venue: arXiv cs.CR 2020
Year: 2020

In this paper, we present an achievable security scheme for an interference channel with arbitrary number of users. In this model, each receiver should be able to decode its intended message while it cannot decode any meaningful information regarding messages intended for other receivers. Our scheme achieves individual secure rates which scale linearly with log(SNR) and achieves sum secure rates which is within constant gap of sum secure capacity. To design the encoders at the transmitters side, we combine nested lattice coding, random i.i.d. codes, and cooperative jamming techniques. Asymmetric compute-and-forward framework is used to perform the decoding operation at the receivers. The novelty of our scheme is that it is the first asymptotically optimal achievable scheme for this security scenario which scales to arbitrary number of users and works for any finite-valued SNR. Also, our scheme achieves the upper bound sum secure degrees of freedom of $1$ without using external helpers.
A Text-based Deep Reinforcement Learning Framework for Interactive Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

Due to its nature of learning from dynamic interactions and planning for long-run performance, reinforcement learning (RL) recently has received much attention in interactive recommender systems (IRSs). IRSs usually face the large discrete action space problem, which makes most of the existing RL-based recommendation methods inefficient. Moreover, data sparsity is another challenging problem that most IRSs are confronted with. While the textual information like reviews and descriptions is less sensitive to sparsity, existing RL-based recommendation methods either neglect or are not suitable for incorporating textual information. To address these two problems, in this paper, we propose a Text-based Deep Deterministic Policy Gradient framework (TDDPG-Rec) for IRSs. Specifically, we leverage textual information to map items and users into a feature space, which greatly alleviates the sparsity problem. Moreover, we design an effective method to construct an action candidate set. By the policy vector dynamically learned from TDDPG-Rec that expresses the user's preference, we can select actions from the candidate set effectively. Through experiments on three public datasets, we demonstrate that TDDPG-Rec achieves state-of-the-art performance over several baselines in a time-efficient manner.
A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation
Venue: arXiv cs.CV 2020
Year: 2020

Action Units (AUs) are geometrically-based atomic facial muscle movements known to produce appearance changes at specific facial locations. Motivated by this observation we propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity. To this end, we propose a simple yet efficient approach based on Heatmap Regression that merges both problems into a single task. A Heatmap models whether an AU occurs or not at a given spatial location. To accommodate the joint modelling of AUs intensity, we propose variable size heatmaps, with their amplitude and size varying according to the labelled intensity. Using Heatmap Regression, we can inherit from the progress recently witnessed in facial landmark localisation. Building upon the similarities between both problems, we devise a transfer learning approach where we exploit the knowledge of a network trained on large-scale facial landmark datasets. In particular, we explore different alternatives for transfer learning through a) fine-tuning, b) adaptation layers, c) attention maps, and d) reparametrisation. Our approach effectively inherits the rich facial features produced by a strong face alignment network, with minimal extra computational cost. We empirically validate that our system sets a new state-of-the-art on three popular datasets, namely BP4D, DISFA, and FERA2017.
Co-eye: A Multi-resolution Symbolic Representation to TimeSeries Diversified Ensemble Classification
Venue: arXiv cs.LG 2020
Year: 2020

Time series classification (TSC) is a challenging task that attracted many researchers in the last few years. One main challenge in TSC is the diversity of domains where time series data come from. Thus, there is no "one model that fits all" in TSC. Some algorithms are very accurate in classifying a specific type of time series when the whole series is considered, while some only target the existence/non-existence of specific patterns/shapelets. Yet other techniques focus on the frequency of occurrences of discriminating patterns/features. This paper presents a new classification technique that addresses the inherent diversity problem in TSC using a nature-inspired method. The technique is stimulated by how flies look at the world through "compound eyes" that are made up of thousands of lenses, called ommatidia. Each ommatidium is an eye with its own lens, and thousands of them together create a broad field of vision. The developed technique similarly uses different lenses and representations to look at the time series, and then combines them for broader visibility. These lenses have been created through hyper-parameterisation of symbolic representations (Piecewise Aggregate and Fourier approximations). The algorithm builds a random forest for each lens, then performs soft dynamic voting for classifying new instances using the most confident eyes, i.e, forests. We evaluate the new technique, coined Co-eye, using the recently released extended version of UCR archive, containing more than 100 datasets across a wide range of domains. The results show the benefits of bringing together different perspectives reflecting on the accuracy and robustness of Co-eye in comparison to other state-of-the-art techniques.
Systematically designing better instance counting models on cell images with Neural Arithmetic Logic Units
Venue: arXiv cs.LG 2020
Year: 2020

The big problem for neural network models which are trained to count instances is that whenever test range goes high training range generalization error increases i.e. they are not good generalizers outside training range. Consider the case of automating cell counting process where more dense images with higher cell counts are commonly encountered as compared to images used in training data. By making better predictions for higher ranges of cell count we are aiming to create better generalization systems for cell counting. With architecture proposal of neural arithmetic logic units (NALU) for arithmetic operations, task of counting has become feasible for higher numeric ranges which were not included in training data with better accuracy. As a part of our study we used these units and different other activation functions for learning cell counting task with two different architectures namely Fully Convolutional Regression Network and U-Net. These numerically biased units are added in the form of residual concatenated layers to original architectures and a comparative experimental study is done with these newly proposed changes. This comparative study is described in terms of optimizing regression loss problem from these models trained with extensive data augmentation techniques. We were able to achieve better results in our experiments of cell counting tasks with introduction of these numerically biased units to already existing architectures in the form of residual layer concatenation connections. Our results confirm that above stated numerically biased units does help models to learn numeric quantities for better generalization results.
Daily growth rate of scientific production on Covid-19. Analysis in databases and open access repositories
Venue: arXiv cs.DL 2020
Year: 2020

The scientific community is facing one of its greatest challenges in solving a global health problem: COVID-19 pandemic. This situation has generated an unprecedented volume of publications. What is the volume, in terms of publications, of research on COVID-19? The general objective of this research work is to obtain a global vision of the daily growth of scientific production on COVID-19 in different databases (Dimensions, Web of Science Core Collection, Scopus-Elsevier, Pubmed and eight repositories). In relation to the results obtained, Dimensions indexes a total of 9435 publications (69% with peer review and 2677 preprints) well above Scopus (1568) and WoS (718). This is a classic biliometric phenomenon of exponential growth (R2 = 0.92). The global growth rate is 500 publications and the production doubles every 15 days. In the case of Pubmed the weekly growth is around 1000 publications. Of the eight repositories analysed, Pubmed Central, Medrxiv and SSRN are the leaders. Despite their enormous contribution, the journals continue to be the core of scientific communication. Finally, it has been established that three out of every four publications on the COVID-19 are available in open access. The information explosion demands a serious and coordinated response from information professionals, which places us at the centre of the information pandemic.
Scalability of High-Performance PDE Solvers
Venue: arXiv cs.PF 2020
Year: 2020

Performance tests and analyses are critical to effective HPC software development and are central components in the design and implementation of computational algorithms for achieving faster simulations on existing and future computing architectures for large-scale application problems. In this paper, we explore performance and space-time trade-offs for important compute-intensive kernels of large-scale numerical solvers for PDEs that govern a wide range of physical applications. We consider a sequence of PDE- motivated bake-off problems designed to establish best practices for efficient high-order simulations across a variety of codes and platforms. We measure peak performance (degrees of freedom per second) on a fixed number of nodes and identify effective code optimization strategies for each architecture. In addition to peak performance, we identify the minimum time to solution at 80% parallel efficiency. The performance analysis is based on spectral and p-type finite elements but is equally applicable to a broad spectrum of numerical PDE discretizations, including finite difference, finite volume, and h-type finite elements.
Model and Machine Learning based Caching and Routing Algorithms for Cache-enabled Networks
Venue: arXiv cs.NI 2020
Year: 2020

In-network caching is likely to become an integral part of various networked systems (e.g., 5G networks, LPWAN and IoT systems) in the near future. In this paper, we compare and contrast model-based and machine learning approaches for designing caching and routing strategies to improve cache network performance (e.g., delay, hit rate). We first outline the key principles used in the design of model-based strategies and discuss the analytical results and bounds obtained for these approaches. By conducting experiments on real-world traces and networks, we identify the interplay between content popularity skewness and request stream correlation as an important factor affecting cache performance. With respect to routing, we show that the main factors impacting performance are alternate path routing and content search. We then discuss the applicability of multiple machine learning models, specifically reinforcement learning, deep learning, transfer learning and probabilistic graphical models for the caching and routing problem.
A Human Evaluation of AMR-to-English Generation Systems
Venue: arXiv cs.CL 2020
Year: 2020

Most current state-of-the art systems for generating English text from Abstract Meaning Representation (AMR) have been evaluated only using automated metrics, such as BLEU, which are known to be problematic for natural language generation. In this work, we present the results of a new human evaluation which collects fluency and adequacy scores, as well as categorization of error types, for several recent AMR generation systems. We discuss the relative quality of these systems and how our results compare to those of automatic metrics, finding that while the metrics are mostly successful in ranking systems overall, collecting human judgments allows for more nuanced comparisons. We also analyze common errors made by these systems.