OpenRaiser/Intern-Atlas huggingface dataset:
e-TLD: Event-based Framework for Dynamic Object Tracking
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a long-term object tracking framework with a moving event camera under general tracking conditions. A first of its kind for these revolutionary cameras, the tracking framework uses a discriminative representation for the object with online learning, and detects and re-tracks the object when it comes back into the field-of-view. One of the key novelties is the use of an event-based local sliding window technique that tracks reliably in scenes with cluttered and textured background. In addition, Bayesian bootstrapping is used to assist real-time processing and boost the discriminative power of the object representation. On the other hand, when the object re-enters the field-of-view of the camera, a data-driven, global sliding window detector locates the object for subsequent tracking. Extensive experiments demonstrate the ability of the proposed framework to track and detect arbitrary objects of various shapes and sizes, including dynamic objects such as a human. This is a significant improvement compared to earlier works that simply track objects as long as they are visible under simpler background settings. Using the ground truth locations for five different objects under three motion settings, namely translation, rotation and 6-DOF, quantitative measurement is reported for the event-based tracking framework with critical insights on various performance issues. Finally, real-time implementation in C++ highlights tracking ability under scale, rotation, view-point and occlusion scenarios in a lab setting.
HL-Pow: A Learning-Based Power Modeling Framework for High-Level Synthesis
Venue: arXiv cs.AR 2020
Year: 2020

High-level synthesis (HLS) enables designers to customize hardware designs efficiently. However, it is still challenging to foresee the correlation between power consumption and HLS-based applications at an early design stage. To overcome this problem, we introduce HL-Pow, a power modeling framework for FPGA HLS based on state-of-the-art machine learning techniques. HL-Pow incorporates an automated feature construction flow to efficiently identify and extract features that exert a major influence on power consumption, simply based upon HLS results, and a modeling flow that can build an accurate and generic power model applicable to a variety of designs with HLS. By using HL-Pow, the power evaluation process for FPGA designs can be significantly expedited because the power inference of HL-Pow is established on HLS instead of the time-consuming register-transfer level (RTL) implementation flow. Experimental results demonstrate that HL-Pow can achieve accurate power modeling that is only 4.67% (24.02 mW) away from onboard power measurement. To further facilitate power-oriented optimizations, we describe a novel design space exploration (DSE) algorithm built on top of HL-Pow to trade off between latency and power consumption. This algorithm can reach a close approximation of the real Pareto frontier while only requiring running HLS flow for 20% of design points in the entire design space.
GAIT: Gradient Adjusted Unsupervised Image-to-Image Translation
Venue: arXiv cs.CV 2020
Year: 2020

Image-to-image translation (IIT) has made much progress recently with the development of adversarial learning. In most of the recent work, an adversarial loss is utilized to match the distributions of the translated and target image sets. However, this may create artifacts if two domains have different marginal distributions, for example, in uniform areas. In this work, we propose an unsupervised IIT method that preserves the uniform regions after the translation. The gradient adjustment loss, which is the L2 norm between the Sobel response of the target image and the adjusted Sobel response of the source images, is utilized. The proposed method is validated on the jellyfish-to-Haeckel dataset, which is prepared to demonstrate the mentioned problem, which contains images with different background distributions. We demonstrate that our method obtained a performance gain compared to the baseline method qualitatively and quantitatively, showing the effectiveness of the proposed method.
Deep Learning to Detect Bacterial Colonies for the Production of Vaccines
Venue: arXiv cs.CV 2020
Year: 2020

During the development of vaccines, bacterial colony forming units (CFUs) are counted in order to quantify the yield in the fermentation process. This manual task is time-consuming and error-prone. In this work we test multiple segmentation algorithms based on the U-Net CNN architecture and show that these offer robust, automated CFU counting. We show that the multiclass generalisation with a bespoke loss function allows distinguishing virulent and avirulent colonies with acceptable accuracy. While many possibilities are left to explore, our results show the potential of deep learning for separating and classifying bacterial colonies.
A Study of Opacity Ranges for Transparent Overlays in 3D Landscapes
Venue: arXiv cs.GR 2020
Year: 2020

{When visualizing data in a realistically rendered 3D virtual environment, it is often important to represent not only the 3D scene but also overlaid information about additional, abstract data. These overlays must be usefully visible, i.e. be readable enough to convey the information they represent, but remain unobtrusive to avoid cluttering the view. We take a step toward establishing guidelines for designing such overlays by studying the relationship between three different patterns (filled, striped and dotted patterns), two pattern densities, the presence or not of a solid outline, two types of background (blank and with trees), and the opacity of the overlay. For each combination of factors, participants set the faintest and the strongest acceptable opacity values. Results from this first study suggest that i) ranges of acceptable opacities are around 20-70%, that ii) ranges can be extended by 5% by using an outline, and that iii) ranges shift based on features like pattern and density.
Real-time 3D Facial Tracking via Cascaded Compositional Learning
Venue: arXiv cs.CV 2020
Year: 2020

We propose to learn a cascade of globally-optimized modular boosted ferns (GoMBF) to solve multi-modal facial motion regression for real-time 3D facial tracking from a monocular RGB camera. GoMBF is a deep composition of multiple regression models with each is a boosted ferns initially trained to predict partial motion parameters of the same modality, and then concatenated together via a global optimization step to form a singular strong boosted ferns that can effectively handle the whole regression target. It can explicitly cope with the modality variety in output variables, while manifesting increased fitting power and a faster learning speed comparing against the conventional boosted ferns. By further cascading a sequence of GoMBFs (GoMBF-Cascade) to regress facial motion parameters, we achieve competitive tracking performance on a variety of in-the-wild videos comparing to the state-of-the-art methods, which require much more training data or have higher computational complexity. It provides a robust and highly elegant solution to real-time 3D facial tracking using a small set of training data and hence makes it more practical in real-world applications.
Analysis of open source license selection for the GitHub programming community
Venue: arXiv cs.SE 2020
Year: 2020

Developers usually select different open source licenses to restrain the conditions of using open source software, in order to protect intellectual property rights effectively and maintain the long-term development of the software. However, the open source community has a wide variety of licenses available, developers generally find it difficult to understand the differences between different open source license. And existing open source license selection tools require developers to understand the terms of the open source license and identify their business needs, which makes it hard for developers to make the right choice. Although academia has extensive research to the open source license, but there is no systematic analysis on the actual difficulties of the developers to choose the open source license, thus lacking a clear understanding, for this reason, the purpose of this paper is to understand the difficulties faced by open source developers in choosing open source licenses, analyze the components of open source license and the affecting factors of open source license selection, and to provide references for developers to choose open source licenses.
zkay v0.2: Practical Data Privacy for Smart Contracts
Venue: arXiv cs.PL 2020
Year: 2020

Recent work introduces zkay, a system for specifying and enforcing data privacy in smart contracts. While the original prototype implementation of zkay (v0.1) demonstrates the feasibility of the approach, its proof-of-concept implementation suffers from severe limitations such as insecure encryption and lack of important language features. In this report, we present zkay v0.2, which addresses its predecessor's limitations. The new implementation significantly improves security, usability, modularity, and performance of the system. In particular, zkay v0.2 supports state-of-the-art asymmetric and hybrid encryption, introduces many new language features (such as function calls, private control flow, and extended type support), allows for different zk-SNARKs backends, and reduces both compilation time and on-chain costs.
ASTRAL: Adversarial Trained LSTM-CNN for Named Entity Recognition
Venue: arXiv cs.CL 2020
Year: 2020

Named Entity Recognition (NER) is a challenging task that extracts named entities from unstructured text data, including news, articles, social comments, etc. The NER system has been studied for decades. Recently, the development of Deep Neural Networks and the progress of pre-trained word embedding have become a driving force for NER. Under such circumstances, how to make full use of the information extracted by word embedding requires more in-depth research. In this paper, we propose an Adversarial Trained LSTM-CNN (ASTRAL) system to improve the current NER method from both the model structure and the training process. In order to make use of the spatial information between adjacent words, Gated-CNN is introduced to fuse the information of adjacent words. Besides, a specific Adversarial training method is proposed to deal with the overfitting problem in NER. We add perturbation to variables in the network during the training process, making the variables more diverse, improving the generalization and robustness of the model. Our model is evaluated on three benchmarks, CoNLL-03, OntoNotes 5.0, and WNUT-17, achieving state-of-the-art results. Ablation study and case study also show that our system can converge faster and is less prone to overfitting.
Java Cryptography Uses in the Wild
Venue: arXiv cs.CR 2020
Year: 2020

[Background] Previous research has shown that developers commonly misuse cryptography APIs. [Aim] We have conducted an exploratory study to find out how crypto APIs are used in open-source Java projects, what types of misuses exist, and why developers make such mistakes. [Method] We used a static analysis tool to analyze hundreds of open-source Java projects that rely on Java Cryptography Architecture, and manually inspected half of the analysis results to assess the tool results. We also contacted the maintainers of these projects by creating an issue on the GitHub repository of each project, and discussed the misuses with developers. [Results] We learned that 85% of Cryptography APIs are misused, however, not every misuse has severe consequences. Developer feedback showed that security caveats in the documentation of crypto APIs are rare, developers may overlook misuses that originate in third-party code, and the context where a Crypto API is used should be taken into account. [Conclusion] We conclude that using Crypto APIs is still problematic for developers but blindly blaming them for such misuses may lead to erroneous conclusions.
Magma: A Ground-Truth Fuzzing Benchmark
Venue: arXiv cs.CR 2020
Year: 2020

High scalability and low running costs have made fuzz testing the de facto standard for discovering software bugs. Fuzzing techniques are constantly being improved in a race to build the ultimate bug-finding tool. However, while fuzzing excels at finding bugs in the wild, evaluating and comparing fuzzer performance is challenging due to the lack of metrics and benchmarks. For example, crash count, perhaps the most commonly-used performance metric, is inaccurate due to imperfections in deduplication techniques. Additionally, the lack of a unified set of targets results in ad hoc evaluations that hinder fair comparison. We tackle these problems by developing Magma, a ground-truth fuzzing benchmark that enables uniform fuzzer evaluation and comparison. By introducing real bugs into real software, Magma allows for the realistic evaluation of fuzzers against a broad set of targets. By instrumenting these bugs, Magma also enables the collection of bug-centric performance metrics independent of the fuzzer. Magma is an open benchmark consisting of seven targets that perform a variety of input manipulations and complex computations, presenting a challenge to state-of-the-art fuzzers. We evaluate seven widely-used mutation-based fuzzers (AFL, AFLFast, AFL++, FairFuzz, MOpt-AFL, honggfuzz, and SymCC-AFL) against Magma over 200,000 CPU-hours. Based on the number of bugs reached, triggered, and detected, we draw conclusions about the fuzzers' exploration and detection capabilities. This provides insight into fuzzer performance evaluation, highlighting the importance of ground truth in performing more accurate and meaningful evaluations.
Too good to be true? Predicting author profiles from abusive language
Venue: arXiv cs.CL 2020
Year: 2020

The problem of online threats and abuse could potentially be mitigated with a computational approach, where sources of abuse are better understood or identified through author profiling. However, abusive language constitutes a specific domain of language for which it has not yet been tested whether differences emerge based on a text author's personality, age, or gender. This study examines statistical relationships between author demographics and abusive vs normal language, and performs prediction experiments for personality, age, and gender. Although some statistical relationships were established between author characteristics and language use, these patterns did not translate to high prediction performance. Personality traits were predicted within 15% of their actual value, age was predicted with an error margin of 10 years, and gender was classified correctly in 70% of the cases. These results are poor when compared to previous research on author profiling, therefore we urge caution in applying this within the context of abusive language and threat assessment.
Lifelong Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Recent advances in object detection have benefited significantly from rapid developments in deep neural networks. However, neural networks suffer from the well-known issue of catastrophic forgetting, which makes continual or lifelong learning problematic. In this paper, we leverage the fact that new training classes arrive in a sequential manner and incrementally refine the model so that it additionally detects new object classes in the absence of previous training data. Specifically, we consider the representative object detector, Faster R-CNN, for both accurate and efficient prediction. To prevent abrupt performance degradation due to catastrophic forgetting, we propose to apply knowledge distillation on both the region proposal network and the region classification network, to retain the detection of previously trained classes. A pseudo-positive-aware sampling strategy is also introduced for distillation sample selection. We evaluate the proposed method on PASCAL VOC 2007 and MS COCO benchmarks and show competitive mAP and 6x inference speed improvement, which makes the approach more suitable for real-time applications. Our implementation will be publicly available.
Circuit Satisfiability Problem for circuits of small complexity
Venue: arXiv cs.CC 2020
Year: 2020

The following problem is considered. A Turing machine $M$, that accepts a string of fixed length $t$ as input, runs for a time not exceeding a fixed value $n$ and is guaranteed to produce a binary output, is given. It's required to find a string $X$ such that $M(X) = 1$ effectively in terms of $t$, $n$, the size of the alphabet of $M$ and the number of states of $M$. The problem is close to the well-known Circuit Satisfiability Problem. The difference from Circuit Satisfiability Problem is that when reduced to Circuit Satisfiability Problem, we get circuits with a rich internal structure (in particular, these are circuits of small Kolmogorov complexity). The proof system, operating with potential proofs of the fact that, for a given machine $M$, the string $X$ does not exist, is provided, its completeness is proved and the algorithm guaranteed to find a proof of the absence of the string $X$ in the case of its actual absence is presented (in the worst case, the algorithm is exponential, but in a wide class of interesting cases it works in polynomial time). We present an algorithm searching for the string $X$, for which its efficiency was neither tested, nor proven, and it may require serious improvement in the future, so it can be regarded as an idea. We also discuss first steps towards solving a more complex problem similar to this one: a Turing machine $M$, that accepts two strings $X$ and $Y$ of fixed length and running for a time that does not exceed a fixed value, is given; it is required to build an algorithm $N$ that builds a string $Y = N(X)$ for any string $X$, such that $M(X, Y) = 1$ (details in the introduction).
Excavating "Excavating AI": The Elephant in the Gallery
Venue: arXiv cs.CY 2020
Year: 2020

Two art exhibitions, "Training Humans" and "Making Faces," and the accompanying essay "Excavating AI: The politics of images in machine learning training sets" by Kate Crawford and Trevor Paglen, are making substantial impact on discourse taking place in the social and mass media networks, and some scholarly circles. Critical scrutiny reveals, however, a self-contradictory stance regarding informed consent for the use of facial images, as well as serious flaws in their critique of ML training sets. Our analysis underlines the non-negotiability of informed consent when using human data in artistic and other contexts, and clarifies issues relating to the description of ML training sets.
Lunar Crater Identification in Digital Images
Venue: arXiv cs.CV 2020
Year: 2020

It is often necessary to identify a pattern of observed craters in a single image of the lunar surface and without any prior knowledge of the camera's location. This so-called "lost-in-space" crater identification problem is common in both crater-based terrain relative navigation (TRN) and in automatic registration of scientific imagery. Past work on crater identification has largely been based on heuristic schemes, with poor performance outside of a narrowly defined operating regime (e.g., nadir pointing images, small search areas). This work provides the first mathematically rigorous treatment of the general crater identification problem. It is shown when it is (and when it is not) possible to recognize a pattern of elliptical crater rims in an image formed by perspective projection. For the cases when it is possible to recognize a pattern, descriptors are developed using invariant theory that provably capture all of the viewpoint invariant information. These descriptors may be pre-computed for known crater patterns and placed in a searchable index for fast recognition. New techniques are also developed for computing pose from crater rim observations and for evaluating crater rim correspondences. These techniques are demonstrated on both synthetic and real images.
COVID-19: The Information Warfare Paradigm Shift
Venue: arXiv cs.CY 2020
Year: 2020

In Kuhn's The Structure of Scientific Revolutions, the critical term is paradigm-shift when it suddenly becomes evident that earlier assumptions no longer are correct and the plurality of the scientific community that studies this domain accepts the change. These types of events can be scientific findings or as in social science system shock that creates a punctured equilibrium that sets the stage in the developments. In information warfare, recent years studies and government lines of efforts have been to engage fake news, electoral interference, and fight extremist social media as the primary combat theater in the information space, and the tools to influence a targeted audience. The COVID-19 pandemic generates a rebuttal of these assumptions. Even if fake news and extremist social media content may exploit fault lines in our society and create a civil disturbance, tensions between federal and local government, and massive protests, it is still effects that impact a part of the population. What we have seen with COVID-19, as an indicator, is that what is related to public health is far more powerful to swing public sentiment and create reactions within the citizenry that are trigger impact at a larger magnitude that has rippled through society in multiple directions.
TDMP-Reliable Target Driven and Mobility Prediction based Routing Protocol in Complex VANET
Venue: arXiv cs.NI 2020
Year: 2020

Vehicle-to-everything (V2X) communication in the vehicular ad hoc network (VANET), an infrastructure-free mechanism, has emerged as a crucial component in the advanced Intelligent Transport System (ITS) for special information transmission and inter-vehicular communications. One of the main research challenges in VANET is the design and implementation of network routing protocols which manage to trigger V2X communication with the reliable end-to-end connectivity and efficient packet transmission. The organically changing nature of road transport vehicles poses a significant threat to VANET with respect to the accuracy and reliability of packet delivery. Therefore, a position-based routing protocol tends to be the predominant method in VANET as they overcome rapid changes in vehicle movements effectively. However, existing routing protocols have some limitations such as (i) inaccurate in high dynamic network topology, (ii) defective link-state estimation (iii) poor movement prediction in heterogeneous road layouts. In this paper, a target-driven and mobility prediction (TDMP) based routing protocol is therefore developed for high-speed mobility and dynamic topology of vehicles, fluctuant traffic flow and diverse road layouts in VANET. The primary idea in TDMP is that the destination target of a driver is included in the mobility prediction to assist the implementation of the routing protocol. Compared to existing geographic routing protocols which mainly greedily forward the packet to the next-hop based on its current position and partial road layout, TDMP is developed to enhance the packet transmission with the consideration of the estimation of inter-vehicles link status, and the prediction of vehicle positions dynamically in fluctuant mobility and global road layout.