OpenRaiser/Intern-Atlas huggingface dataset:
Are Fewer Labels Possible for Few-shot Learning?
Venue: arXiv cs.CV 2020
Year: 2020

Few-shot learning is challenging due to its very limited data and labels. Recent studies in big transfer (BiT) show that few-shot learning can greatly benefit from pretraining on large scale labeled dataset in a different domain. This paper asks a more challenging question: "can we use as few as possible labels for few-shot learning in both pretraining (with no labels) and fine-tuning (with fewer labels)?". Our key insight is that the clustering of target samples in the feature space is all we need for few-shot finetuning. It explains why the vanilla unsupervised pretraining (poor clustering) is worse than the supervised one. In this paper, we propose transductive unsupervised pretraining that achieves a better clustering by involving target data even though its amount is very limited. The improved clustering result is of great value for identifying the most representative samples ("eigen-samples") for users to label, and in return, continued finetuning with the labeled eigen-samples further improves the clustering. Thus, we propose eigen-finetuning to enable fewer shot learning by leveraging the co-evolution of clustering and eigen-samples in the finetuning. We conduct experiments on 10 different few-shot target datasets, and our average few-shot performance outperforms both vanilla inductive unsupervised transfer and supervised transfer by a large margin. For instance, when each target category only has 10 labeled samples, the mean accuracy gain over the above two baselines is 9.2% and 3.42 respectively.
A Simplistic Machine Learning Approach to Contact Tracing
Venue: arXiv cs.LG 2020
Year: 2020

This report is based on the modified NIST challenge, Too Close For Too Long, provided by the SFI Centre for Machine Learning (ML-Labs). The modified challenge excludes the time calculation (too long) aspect. By handcrafting features from phone instrumental data we develop two machine learning models, a GBM and an MLP, to estimate distance between two phones. Our method is able to outperform the leading NIST challenge result by the Hong Kong University of Science and Technology (HKUST) by a significant margin.
Autonomous Cooperative Wall Building by a Team of Unmanned Aerial Vehicles in the MBZIRC 2020 Competition
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a system for autonomous cooperative wall building with a team of Unmanned Aerial Vehicles (UAVs). The system was developed for Challenge 2 of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The wall-building scenario of Challenge 2 featured an initial stack of bricks and wall structure where the individual bricks had to be placed by a team of three UAVs. The objective of the task was to maximize collected points for placing the bricks within the restricted construction time while following the prescribed wall pattern. The proposed approach uses initial scanning to find a priori unknown locations of the bricks and the wall structure. Each UAV is then assigned to individual bricks and wall placing locations and further perform grasping and placement using onboard resources only. The developed system consists of methods for scanning a given area, RGB-D detection of bricks and wall placement locations, precise grasping and placing of bricks, and coordination of multiple UAVs. The paper describes the overall system, individual components, experimental verification in demanding outdoor conditions, the achieved results in the competition, and lessons learned. The presented CTU-UPenn-NYU approach achieved the overall best performance among all participants to won the MBZIRC competition by collecting the highest number of points by correct placement of a high number of bricks.
A Generative Approach for Detection-driven Underwater Image Enhancement
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we introduce a generative model for image enhancement specifically for improving diver detection in the underwater domain. In particular, we present a model that integrates generative adversarial network (GAN)-based image enhancement with the diver detection task. Our proposed approach restructures the GAN objective function to include information from a pre-trained diver detector with the goal to generate images which would enhance the accuracy of the detector in adverse visual conditions. By incorporating the detector output into both the generator and discriminator networks, our model is able to focus on enhancing images beyond aesthetic qualities and specifically to improve robotic detection of scuba divers. We train our network on a large dataset of scuba divers, using a state-of-the-art diver detector, and demonstrate its utility on images collected from oceanic explorations of human-robot teams. Experimental evaluations demonstrate that our approach significantly improves diver detection performance over raw, unenhanced images, and even outperforms detection performance on the output of state-of-the-art underwater image enhancement algorithms. Finally, we demonstrate the inference performance of our network on embedded devices to highlight the feasibility of operating on board mobile robotic platforms.
Exploring Deep Neural Networks and Transfer Learning for Analyzing Emotions in Tweets
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we present an experiment on using deep learning and transfer learning techniques for emotion analysis in tweets and suggest a method to interpret our deep learning models. The proposed approach for emotion analysis combines a Long Short Term Memory (LSTM) network with a Convolutional Neural Network (CNN). Then we extend this approach for emotion intensity prediction using transfer learning technique. Furthermore, we propose a technique to visualize the importance of each word in a tweet to get a better understanding of the model. Experimentally, we show in our analysis that the proposed models outperform the state-of-the-art in emotion classification while maintaining competitive results in predicting emotion intensity.
A novel joint points and silhouette-based method to estimate 3D human pose and shape
Venue: arXiv cs.CV 2020
Year: 2020

This paper presents a novel method for 3D human pose and shape estimation from images with sparse views, using joint points and silhouettes, based on a parametric model. Firstly, the parametric model is fitted to the joint points estimated by deep learning-based human pose estimation. Then, we extract the correspondence between the parametric model of pose fitting and silhouettes on 2D and 3D space. A novel energy function based on the correspondence is built and minimized to fit parametric model to the silhouettes. Our approach uses sufficient shape information because the energy function of silhouettes is built from both 2D and 3D space. This also means that our method only needs images from sparse views, which balances data used and the required prior information. Results on synthetic data and real data demonstrate the competitive performance of our approach on pose and shape estimation of the human body.
I-GCN: Robust Graph Convolutional Network via Influence Mechanism
Venue: arXiv cs.LG 2020
Year: 2020

Deep learning models for graphs, especially Graph Convolutional Networks (GCNs), have achieved remarkable performance in the task of semi-supervised node classification. However, recent studies show that GCNs suffer from adversarial perturbations. Such vulnerability to adversarial attacks significantly decreases the stability of GCNs when being applied to security-critical applications. Defense methods such as preprocessing, attention mechanism and adversarial training have been discussed by various studies. While being able to achieve desirable performance when the perturbation rates are low, such methods are still vulnerable to high perturbation rates. Meanwhile, some defending algorithms perform poorly when the node features are not visible. Therefore, in this paper, we propose a novel mechanism called influence mechanism, which is able to enhance the robustness of the GCNs significantly. The influence mechanism divides the effect of each node into two parts: introverted influence which tries to maintain its own features and extroverted influence which exerts influences on other nodes. Utilizing the influence mechanism, we propose the Influence GCN (I-GCN) model. Extensive experiments show that our proposed model is able to achieve higher accuracy rates than state-of-the-art methods when defending against non-targeted attacks.
Learning Edge-Preserved Image Stitching from Large-Baseline Deep Homography
Venue: arXiv cs.CV 2020
Year: 2020

Image stitching is a classical and crucial technique in computer vision, which aims to generate the image with a wide field of view. The traditional methods heavily depend on the feature detection and require that scene features be dense and evenly distributed in the image, leading to varying ghosting effects and poor robustness. Learning methods usually suffer from fixed view and input size limitations, showing a lack of generalization ability on other real datasets. In this paper, we propose an image stitching learning framework, which consists of a large-baseline deep homography module and an edge-preserved deformation module. First, we propose a large-baseline deep homography module to estimate the accurate projective transformation between the reference image and the target image in different scales of features. After that, an edge-preserved deformation module is designed to learn the deformation rules of image stitching from edge to content, eliminating the ghosting effects as much as possible. In particular, the proposed learning framework can stitch images of arbitrary views and input sizes, thus contribute to a supervised deep image stitching method with excellent generalization capability in other real images. Experimental results demonstrate that our homography module significantly outperforms the existing deep homography methods in the large baseline scenes. In image stitching, our method is superior to the existing learning method and shows competitive performance with state-of-the-art traditional methods.
Improving Zero Shot Learning Baselines with Commonsense Knowledge
Venue: arXiv cs.CL 2020
Year: 2020

Zero shot learning -- the problem of training and testing on a completely disjoint set of classes -- relies greatly on its ability to transfer knowledge from train classes to test classes. Traditionally semantic embeddings consisting of human defined attributes (HA) or distributed word embeddings (DWE) are used to facilitate this transfer by improving the association between visual and semantic embeddings. In this paper, we take advantage of explicit relations between nodes defined in ConceptNet, a commonsense knowledge graph, to generate commonsense embeddings of the class labels by using a graph convolution network-based autoencoder. Our experiments performed on three standard benchmark datasets surpass the strong baselines when we fuse our commonsense embeddings with existing semantic embeddings i.e. HA and DWE.
EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts
Venue: arXiv cs.LG 2020
Year: 2020

Climate change is global, yet its concrete impacts can strongly vary between different locations in the same region. Seasonal weather forecasts currently operate at the mesoscale (> 1 km). For more targeted mitigation and adaptation, modelling impacts to < 100 m is needed. Yet, the relationship between driving variables and Earth's surface at such local scales remains unresolved by current physical models. Large Earth observation datasets now enable us to create machine learning models capable of translating coarse weather information into high-resolution Earth surface forecasts. Here, we define high-resolution Earth surface forecasting as video prediction of satellite imagery conditional on mesoscale weather forecasts. Video prediction has been tackled with deep learning models. Developing such models requires analysis-ready datasets. We introduce EarthNet2021, a new, curated dataset containing target spatio-temporal Sentinel 2 satellite imagery at 20 m resolution, matched with high-resolution topography and mesoscale (1.28 km) weather variables. With over 32000 samples it is suitable for training deep neural networks. Comparing multiple Earth surface forecasts is not trivial. Hence, we define the EarthNetScore, a novel ranking criterion for models forecasting Earth surface reflectance. For model intercomparison we frame EarthNet2021 as a challenge with four tracks based on different test sets. These allow evaluation of model validity and robustness as well as model applicability to extreme events and the complete annual vegetation cycle. In addition to forecasting directly observable weather impacts through satellite-derived vegetation indices, capable Earth surface models will enable downstream applications such as crop yield prediction, forest health assessments, coastline management, or biodiversity monitoring. Find data, code, and how to participate at www.earthnet.tech .
Improved Robustness to Disfluencies in RNN-Transducer Based Speech Recognition
Venue: arXiv cs.CL 2020
Year: 2020

Automatic Speech Recognition (ASR) based on Recurrent Neural Network Transducers (RNN-T) is gaining interest in the speech community. We investigate data selection and preparation choices aiming for improved robustness of RNN-T ASR to speech disfluencies with a focus on partial words. For evaluation we use clean data, data with disfluencies and a separate dataset with speech affected by stuttering. We show that after including a small amount of data with disfluencies in the training set the recognition accuracy on the tests with disfluencies and stuttering improves. Increasing the amount of training data with disfluencies gives additional gains without degradation on the clean data. We also show that replacing partial words with a dedicated token helps to get even better accuracy on utterances with disfluencies and stutter. The evaluation of our best model shows 22.5% and 16.4% relative WER reduction on those two evaluation sets.
How question quality drives Web performance in community question answering sites
Venue: arXiv cs.SI 2020
Year: 2020

Users are posting millions of questions on Community question answering sites each day. The quality of those questions significantly affects the satisfactions of the sites' users and, therefore, sites' traffic. We gathered 15 question-quality related features from one of the largest CQA sites and the site's pageview data to estimate the scale of the effect in the corresponding time series. By using a Grey Relational Analysis, we rank those question quality features and estimate the relative strength of these factors on a page's view numbers. Our results show that the features of question quality have a significant influence on web performance. We generate a ranked list of features and find that digital popularity and textual features can drive the page traffic more than questioner related features and question difficulty. The implications of the findings for Web growth and future research are discussed.
SEGSys: A mapping system for segmentation analysis in energy
Venue: arXiv cs.DB 2020
Year: 2020

Customer segmentation analysis can give valuable insights into the energy efficiency of residential buildings. This paper presents a mapping system, SEGSys that enables segmentation analysis at the individual and the neighborhood levels. SEGSys supports the online and offline classification of customers based on their daily consumption patterns and consumption intensity. It also supports the segmentation analysis according to the social characteristics of customers of individual households or neighborhoods, as well as spatial geometries. SEGSys uses a three-layer architecture to model the segmentation system, including the data layer, the service layer, and the presentation layer. The data layer models data into a star schema within a data warehouse, the service layer provides data service through a RESTful interface, and the presentation layer interacts with users through a visual map. This paper showcases the system on the segmentation analysis using an electricity consumption data set and validates the effectiveness of the system.
Decompositional Minimisation of Monolithic Processes
Venue: arXiv cs.LO 2020
Year: 2020

Compositional minimisation can be an effective technique to reduce the state space explosion problem. This technique considers a parallel composition of several processes. In its simplest form, each sequential process is replaced by an abstraction, simpler than the corresponding process while still preserving the property that is checked. However, this technique cannot be applied in a setting where parallel composition is first translated to a non-deterministic sequential monolithic process. The advantage of this monolithic process is that it facilitates static analysis of global behaviour. Therefore, we present a technique that considers a monolithic process with data and decomposes it into two processes where each process defines behaviour for a subset of the parameters of the monolithic process. We prove that these processes preserve the properties of the monolithic process under a suitable synchronisation context. Moreover, we prove that state invariants can be used to improve its effectiveness. Finally, we apply the decomposition technique to several specifications.
Usage Scenarios for Byte-Addressable Persistent Memory inHigh-Performance and Data Intensive Computing
Venue: arXiv cs.DC 2020
Year: 2020

Byte-addressable persistent memory (B-APM) presents a new opportunity to bridge the performance gap between main memory and storage. In this paper, we present the usage scenarios for this new technology, based on the capabilities of Intel's DCPMM. We outline some of the basic performance characteristics of DCPMM, and explain how it can be configured and used to address the needs of memory and I/O intensive applications in the HPC and data intensive domains. Two decision trees are presented to advise on the configuration options for B-APM; their use is illustrated with two examples. We show that the flexibility of the technology has the potential to be truly disruptive, not only because of the performance improvements it can deliver, but also because it allows systems to cater for wider range of applications on homogeneous hardware.
Unsupervised deep learning for individualized brain functional network identification
Venue: arXiv cs.CV 2020
Year: 2020

A novel unsupervised deep learning method is developed to identify individual-specific large scale brain functional networks (FNs) from resting-state fMRI (rsfMRI) in an end-to-end learning fashion. Our method leverages deep Encoder-Decoder networks and conventional brain decomposition models to identify individual-specific FNs in an unsupervised learning framework and facilitate fast inference for new individuals with one forward pass of the deep network. Particularly, convolutional neural networks (CNNs) with an Encoder-Decoder architecture are adopted to identify individual-specific FNs from rsfMRI data by optimizing their data fitting and sparsity regularization terms that are commonly used in brain decomposition models. Moreover, a time-invariant representation learning module is designed to learn features invariant to temporal orders of time points of rsfMRI data. The proposed method has been validated based on a large rsfMRI dataset and experimental results have demonstrated that our method could obtain individual-specific FNs which are consistent with well-established FNs and are informative for predicting brain age, indicating that the individual-specific FNs identified truly captured the underlying variability of individualized functional neuroanatomy.
DeepObjStyle: Deep Object-based Photo Style Transfer
Venue: arXiv cs.CV 2020
Year: 2020

One of the major challenges of style transfer is the appropriate image features supervision between the output image and the input (style and content) images. An efficient strategy would be to define an object map between the objects of the style and the content images. However, such a mapping is not well established when there are semantic objects of different types and numbers in the style and the content images. It also leads to content mismatch in the style transfer output, which could reduce the visual quality of the results. We propose an object-based style transfer approach, called DeepObjStyle, for the style supervision in the training data-independent framework. DeepObjStyle preserves the semantics of the objects and achieves better style transfer in the challenging scenario when the style and the content images have a mismatch of image features. We also perform style transfer of images containing a word cloud to demonstrate that DeepObjStyle enables an appropriate image features supervision. We validate the results using quantitative comparisons and user studies.
Cooperative Location Privacy in Vehicular Networks: Why Simple Mix-zones are not Enough
Venue: arXiv cs.CR 2020
Year: 2020

Vehicular communications disclose rich information about the vehicles and their whereabouts. Pseudonymous authentication secures communication while enhancing user privacy. To enhance location privacy, cryptographic mix-zones were proposed to facilitate vehicles covertly transition to new ephemeral credentials. The resilience to (syntactic and semantic) pseudonym linking (attacks) highly depends on the geometry of the mix-zones, mobility patterns, vehicle density, and arrival rates. We introduce a tracking algorithm for linking pseudonyms before and after a cryptographically protected mix-zone. Our experimental results show that an eavesdropper, leveraging standardized vehicular communication messages and road layout, could successfully link 73% of pseudonyms during non-rush hours and 62% of pseudonyms during rush hours after vehicles change their pseudonyms in a mix-zone. To mitigate such inference attacks, we present a novel cooperative mix-zone scheme that enhances user privacy regardless of the vehicle mobility patterns, vehicle density, and arrival rate to the mix-zone. A subset of vehicles, termed relaying vehicles, are selected to be responsible for emulating non-existing vehicles. Such vehicles cooperatively disseminate decoy traffic without affecting safety-critical operations: with 50% of vehicles as relaying vehicles, the probability of linking pseudonyms (for the entire interval) drops from 68% to 18%. On average, this imposes 28 ms extra computation overhead, per second, on the Roadside Units (RSUs) and 4.67 ms extra computation overhead, per second, on the (relaying) vehicle side; it also introduces 1.46 KB/sec extra communication overhead by (relaying) vehicles and 45 KB/sec by RSUs for the dissemination of decoy traffic. Thus, user privacy is enhanced at the cost of low computation and communication overhead.