OpenRaiser/Intern-Atlas huggingface dataset:
YOLACT++: Better Real-time Instance Segmentation
Venue: arXiv cs.CV 2020
Year: 2020

We present a simple, fully-convolutional model for real-time (>30 fps) instance segmentation that achieves competitive results on MS COCO evaluated on a single Titan Xp, which is significantly faster than any previous state-of-the-art approach. Moreover, we obtain this result after training on only one GPU. We accomplish this by breaking instance segmentation into two parallel subtasks: (1) generating a set of prototype masks and (2) predicting per-instance mask coefficients. Then we produce instance masks by linearly combining the prototypes with the mask coefficients. We find that because this process doesn't depend on repooling, this approach produces very high-quality masks and exhibits temporal stability for free. Furthermore, we analyze the emergent behavior of our prototypes and show they learn to localize instances on their own in a translation variant manner, despite being fully-convolutional. We also propose Fast NMS, a drop-in 12 ms faster replacement for standard NMS that only has a marginal performance penalty. Finally, by incorporating deformable convolutions into the backbone network, optimizing the prediction head with better anchor scales and aspect ratios, and adding a novel fast mask re-scoring branch, our YOLACT++ model can achieve 34.1 mAP on MS COCO at 33.5 fps, which is fairly close to the state-of-the-art approaches while still running at real-time.
Theme-Matters: Fashion Compatibility Learning via Theme Attention
Venue: arXiv cs.CV 2020
Year: 2020

Fashion compatibility learning is important to many fashion markets such as outfit composition and online fashion recommendation. Unlike previous work, we argue that fashion compatibility is not only a visual appearance compatible problem but also a theme-matters problem. An outfit, which consists of a set of fashion items (e.g., shirt, suit, shoes, etc.), is considered to be compatible for a "dating" event, yet maybe not for a "business" occasion. In this paper, we aim at solving the fashion compatibility problem given specific themes. To this end, we built the first real-world theme-aware fashion dataset comprising 14K around outfits labeled with 32 themes. In this dataset, there are more than 40K fashion items labeled with 152 fine-grained categories. We also propose an attention model learning fashion compatibility given a specific theme. It starts with a category-specific subspace learning, which projects compatible outfit items in certain categories to be close in the subspace. Thanks to strong connections between fashion themes and categories, we then build a theme-attention model over the category-specific embedding space. This model associates themes with the pairwise compatibility with attention, and thus compute the outfit-wise compatibility. To the best of our knowledge, this is the first attempt to estimate outfit compatibility conditional on a theme. We conduct extensive qualitative and quantitative experiments on our new dataset. Our method outperforms the state-of-the-art approaches.
Grounding-Tracking-Integration
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we study Tracking by Language that localizes the target box sequence in a video based on a language query. We propose a framework called GTI that decomposes the problem into three sub-tasks: Grounding, Tracking, and Integration. The three sub-task modules operate simultaneously and predict the box sequence frame-by-frame. "Grounding" predicts the referred region directly from the language query. "Tracking" localizes the target based on the history of the grounded regions in previous frames. "Integration" generates final predictions by synergistically combining grounding and tracking. With the "integration" task as the key, we explore how to indicate the quality of the grounded regions in each frame and achieve the desired mutually beneficial combination. To this end, we propose an "RT-integration" method that defines and predicts two scores to guide the integration: 1) R-score represents the Region correctness whether the grounding prediction accurately covers the target, and 2) T-score represents the Template quality whether the region provides informative visual cues to improve tracking in future frames. We present our real-time GTI implementation with the proposed RT-integration, and benchmark the framework on LaSOT and Lingual OTB99 with highly promising results. Moreover, we produce a disambiguated version of LaSOT queries to facilitate future tracking by language studies.
AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates
Venue: arXiv cs.LG 2020
Year: 2020

Artificial Intelligence (AI) can now automate the algorithm selection, feature engineering, and hyperparameter tuning steps in a machine learning workflow. Commonly known as AutoML or AutoAI, these technologies aim to relieve data scientists from the tedious manual work. However, today's AutoAI systems often present only limited to no information about the process of how they select and generate model results. Thus, users often do not understand the process, neither do they trust the outputs. In this short paper, we provide a first user evaluation by 10 data scientists of an experimental system, AutoAIViz, that aims to visualize AutoAI's model generation process. We find that the proposed system helps users to complete the data science tasks, and increases their understanding, toward the goal of increasing trust in the AutoAI system.
Joint Reliability-aware and Cost Efficient Path Allocation and VNF Placement using Sharing Scheme
Venue: arXiv cs.NI 2020
Year: 2020

Network Function Virtualization (NFV) is a vital player of modern networks providing different types of services such as traffic optimization, content filtering, and load balancing. More precisely, NFV is a provisioning technology aims at reducing the large Capital Expenditure (CAPEX) of network providers by moving services from dedicated hardware to commodity servers using Virtualized Network Functions (VNF). A sequence of VNFs/services following a logical goal is referred to as a Service Function Chain (SFC). The movement toward SFC introduces new challenges to those network services which require high reliability. To address this challenge, redundancy schemes are introduced. Existing redundancy schemes using dedicated protection enhance the reliability of services, however, they do not consider the cost of redundant VNFs. In this paper, we propose a novel reliability enhancement method using a shared protection scheme to reduce the cost of redundant VNFs. To this end, We mathematically formulate the problem as a Mixed Integer Linear Programming (MILP). The objective is to determine optimal reliability that could be achieved with minimum cost. Although the corresponding optimization problem can be solved using existing MILP solvers, the computational complexity is not rational for realistic scenarios. Thereafter, we propose a Reliability-aware and minimum-Cost based Genetic (RCG) algorithm to solve this problem with low computational complexity. In order to evaluate the proposed solution, We have compared it with four different solutions. Simulation results show that RCG achieves near-optimal performance at a much lower complexity compared with the optimal solution.
Targeted transfer learning to improve performance in small medical physics datasets
Venue: arXiv cs.LG 2020
Year: 2020

The growing use of Machine Learning has produced significant advances in many fields. For image-based tasks, however, the use of deep learning remains challenging in small datasets. In this article, we review, evaluate and compare the current state-of-the-art techniques in training neural networks to elucidate which techniques work best for small datasets. We further propose a path forward for the improvement of model accuracy in medical imaging applications. We observed best results from one cycle training, discriminative learning rates with gradual freezing and parameter modification after transfer learning. We also established that when datasets are small, transfer learning plays an important role beyond parameter initialization by reusing previously learned features. Surprisingly we observed that there is little advantage in using pre-trained networks in images from another part of the body compared to Imagenet. On the contrary, if images from the same part of the body are available then transfer learning can produce a significant improvement in performance with as little as 50 images in the training data.
Deep Context Maps: Agent Trajectory Prediction using Location-specific Latent Maps
Venue: arXiv cs.RO 2020
Year: 2020

In this paper, we propose a novel approach for agent motion prediction in cluttered environments. One of the main challenges in predicting agent motion is accounting for location and context-specific information. Our main contribution is the concept of learning context maps to improve the prediction task. Context maps are a set of location-specific latent maps that are trained alongside the predictor. Thus, the proposed maps are capable of capturing location context beyond visual context cues (e.g. usual average speeds and typical trajectories) or predefined map primitives (such as lanes and stop lines). We pose context map learning as a multi-task training problem and describe our map model and its incorporation into a state-of-the-art trajectory predictor. In extensive experiments, it is shown that use of learned maps can significantly improve predictor accuracy. Furthermore, the performance can be additionally boosted by providing partial knowledge of map semantics.
Integrating Lexical Knowledge in Word Embeddings using Sprinkling and Retrofitting
Venue: arXiv cs.CL 2020
Year: 2020

Neural network based word embeddings, such as Word2Vec and GloVe, are purely data driven in that they capture the distributional information about words from the training corpus. Past works have attempted to improve these embeddings by incorporating semantic knowledge from lexical resources like WordNet. Some techniques like retrofitting modify word embeddings in the post-processing stage while some others use a joint learning approach by modifying the objective function of neural networks. In this paper, we discuss two novel approaches for incorporating semantic knowledge into word embeddings. In the first approach, we take advantage of Levy et al's work which showed that using SVD based methods on co-occurrence matrix provide similar performance to neural network based embeddings. We propose a 'sprinkling' technique to add semantic relations to the co-occurrence matrix directly before factorization. In the second approach, WordNet similarity scores are used to improve the retrofitting method. We evaluate the proposed methods in both intrinsic and extrinsic tasks and observe significant improvements over the baselines in many of the datasets.
Are cookie banners indeed compliant with the law? Deciphering EU legal requirements on consent and technical means to verify compliance of cookie banners
Venue: arXiv cs.CR 2020
Year: 2020

In this work, we analyze the legal requirements on how cookie banners are supposed to be implemented to be fully compliant with the e-Privacy Directive and the General Data Protection Regulation. Our contribution resides in the definition of seventeen operational and fine-grained requirements on cookie banner design that are legally compliant, and moreover, we define whether and when the verification of compliance of each requirement is technically feasible. The definition of requirements emerges from a joint interdisciplinary analysis composed of lawyers and computer scientists in the domain of web tracking technologies. As such, while some requirements are provided by explicitly codified legal sources, others result from the domain-expertise of computer scientists. In our work, we match each requirement against existing cookie banners design of websites. For each requirement, we exemplify with compliant and non-compliant cookie banners. As an outcome of a technical assessment, we verify per requirement if technical (with computer science tools) or manual (with any human operator) verification is needed to assess compliance of consent and we also show which requirements are impossible to verify with certainty in the current architecture of the Web. For example, we explain how the requirement for revocable consent could be implemented in practice: when consent is revoked, the publisher should delete the consent cookie and communicate the withdrawal to all third parties who have previously received consent. With this approach we aim to support practically-minded parties (compliance officers, regulators, researchers, and computer scientists) to assess compliance and detect violations in cookie banner design and implementation, specially under the current revision of the European Union e-Privacy framework.
Comments and Corrections to "Capacity of Multiple-Antenna Systems With Both Receiver and Transmitter Channel State Information"
Venue: arXiv cs.IT 2020
Year: 2020

The correspondence cited in the title [1] derived ergodic capacity of the coherent multiple-input multiple-output (MIMO) channel in independent and identically distributed (IID) Rayleigh fading. While the theoretical results are correct, several plots in the paper are incorrect. In this correspondence, we correct the plots. More importantly, the corrected plots present an interesting and compelling contrast between performances of the coherent MIMO systems with and without channel state information at the transmitter; whereas this view is somewhat limited in [1] because of flaws in the capacity curves.
Bounded haptic teleoperation of a quadruped robot's foot posture for sensing and manipulation
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a control framework to teleoperate a quadruped robot's foot for operator-guided haptic exploration of the environment. Since one leg of a quadruped robot typically only has 3 actuated degrees of freedom (DoFs), the torso is employed to assist foot posture control via a hierarchical whole-body controller. The foot and torso postures are controlled by two analytical Cartesian impedance controllers cascaded by a null space projector. The contact forces acting on supporting feet are optimized by quadratic programming (QP). The foot's Cartesian impedance controller may also estimate contact forces from trajectory tracking errors, and relay the force-feedback to the operator. A 7D haptic joystick, Sigma.7, transmits motion commands to the quadruped robot ANYmal, and renders the force feedback. Furthermore, the joystick's motion is bounded by mapping the foot's feasible force polytope constrained by the friction cones and torque limits in order to prevent the operator from driving the robot to slipping or falling over. Experimental results demonstrate the efficiency of the proposed framework.
Decentralized Runtime Verification for LTL Properties Using Global Clock
Venue: arXiv cs.DC 2020
Year: 2020

Runtime verification is the process of verifying critical behavioral properties in big complex systems, where formal verification is not possible due to state space explosion. There have been several attempts to design efficient algorithms for runtime verification. Most of these algorithms have a formally defined correctness property as a reference and check whether the system consistently meets the demands of the property or it fails to satisfy the property at some point in runtime. LTL is a commonly used language for defining these kinds of properties and is also the language of focus in this paper. One of the main target systems for runtime verification are distributed systems, where the system consists of a number of processes connecting to each other using asynchronous message passing. There are two approaches for runtime verification in distributed systems. The first one consists of centralized algorithms, where all processes send their events to a specific decision-making process, which keeps track of all the events to evaluate the specified property. The second approach consists of distributed algorithms, where processes check the specified property collaboratively. Centralized algorithms are simple, but usually involve sending a large number of messages to the decision-making process. They also suffer from the problem of single point of failure, as well as high traffic loads towards one process. Distributed algorithms, on the other hand, are usually more complicated, but once implemented, offer more efficiency. In this paper, we focus on a class of asynchronous distributed systems, where each process can change its own local state at any arbitrary time and completely independent of others, while all processes share a global clock. We propose a sound and complete algorithm for decentralized runtime verification of LTL properties in these systems.
Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?
Venue: arXiv cs.CV 2020
Year: 2020

Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data driven factors (e.g., image quality, image population statistics, and algorithm architecture), and scenario modeling factors that consider the role of the "user" of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate how these issues apply, we present data from four face recognition algorithms (a previous-generation algorithm and three deep convolutional neural networks, DCNNs) for East Asian and Caucasian faces. First, dataset difficulty affected both overall recognition accuracy and race bias, such that race bias increased with item difficulty. Second, for all four algorithms, the degree of bias varied depending on the identification decision threshold. To achieve equal false accept rates (FARs), East Asian faces required higher identification thresholds than Caucasian faces, for all algorithms. Third, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude that race bias needs to be measured for individual applications and we provide a checklist for measuring this bias in face recognition algorithms.
Deep Learning for Cardiologist-level Myocardial Infarction Detection in Electrocardiograms
Venue: arXiv cs.LG 2020
Year: 2020

Myocardial infarction is the leading cause of death worldwide. In this paper, we design domain-inspired neural network models to detect myocardial infarction. First, we study the contribution of various leads. This systematic analysis, first of its kind in the literature, indicates that out of 15 ECG leads, data from the v6, vz, and ii leads are critical to correctly identify myocardial infarction. Second, we use this finding and adapt the ConvNetQuake neural network model--originally designed to identify earthquakes--to attain state-of-the-art classification results for myocardial infarction, achieving $99.43\%$ classification accuracy on a record-wise split, and $97.83\%$ classification accuracy on a patient-wise split. These two results represent cardiologist-level performance level for myocardial infarction detection after feeding only 10 seconds of raw ECG data into our model. Third, we show that our multi-ECG-channel neural network achieves cardiologist-level performance without the need of any kind of manual feature extraction or data pre-processing.
MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking
Venue: arXiv cs.CV 2020
Year: 2020

In the past few years, Generative Adversarial Networks (GANs) have dramatically advanced our ability to represent and parameterize high-dimensional, non-linear image manifolds. As a result, they have been widely adopted across a variety of applications, ranging from challenging inverse problems like image completion, to problems such as anomaly detection and adversarial defense. A recurring theme in many of these applications is the notion of projecting an image observation onto the manifold that is inferred by the generator. In this context, Projected Gradient Descent (PGD) has been the most popular approach, which essentially optimizes for a latent vector that minimizes the discrepancy between a generated image and the given observation. However, PGD is a brittle optimization technique that fails to identify the right projection (or latent vector) when the observation is corrupted, or perturbed even by a small amount. Such corruptions are common in the real world, for example images in the wild come with unknown crops, rotations, missing pixels, or other kinds of non-linear distributional shifts which break current encoding methods, rendering downstream applications unusable. To address this, we propose corruption mimicking -- a new robust projection technique, that utilizes a surrogate network to approximate the unknown corruption directly at test time, without the need for additional supervision or data augmentation. The proposed method is significantly more robust than PGD and other competing methods under a wide variety of corruptions, thereby enabling a more effective use of GANs in real-world applications. More importantly, we show that our approach produces state-of-the-art performance in several GAN-based applications -- anomaly detection, domain adaptation, and adversarial defense, that benefit from an accurate projection.
Towards Smart Wireless Communications via Intelligent Reflecting Surfaces: A Contemporary Survey
Venue: arXiv cs.IT 2020
Year: 2020

This paper presents a literature review on recent applications and design aspects of the intelligent reflecting surface (IRS) in the future wireless networks. Conventionally, the network optimization has been limited to transmission control at two endpoints, i.e., end users and network controller. The fading wireless channel is uncontrollable and becomes one of the main limiting factors for performance improvement. The IRS is composed of a large array of scattering elements, which can be individually configured to generate additional phase shifts to the signal reflections. Hence, it can actively control the signal propagation properties in favor of signal reception, and thus realize the notion of a smart radio environment. As such, the IRS's phase control, combined with the conventional transmission control, can potentially bring performance gain compared to wireless networks without IRS. In this survey, we first introduce basic concepts of the IRS and the realizations of its reconfigurability. Then, we focus on applications of the IRS in wireless communications. We overview different performance metrics and analytical approaches to characterize the performance improvement of IRS-assisted wireless networks. To exploit the performance gain, we discuss the joint optimization of the IRS's phase control and the transceivers' transmission control in different network design problems, e.g.,~rate maximization and power minimization problems. Furthermore, we extend the discussion of IRS-assisted wireless networks to some emerging use cases. Finally, we highlight important practical challenges and future research directions for realizing IRS-assisted wireless networks in beyond 5G communications.