Reviewerly/exHarmony huggingface dataset:
Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination

Unsupervised feature learning has made great strides with contrastive learning based on instance discrimination and invariant mapping, as benchmarked on curated class-balanced datasets. However, natural data could be highly correlated and long-tail distributed. Natural between-instance similarity conflicts with the presumed instance distinction, causing unstable training and poor performance.Our idea is to discover and integrate between-instance similarity into contrastive learning, not directly by instance grouping, but by cross-level discrimination (CLD) between instances and local instance groups. While invariant mapping of each instance is imposed by attraction within its augmented views, between-instance similarity could emerge from common repulsion against instance groups.Our batch-wise and cross-view comparisons also greatly improve the positive/negative sample ratio of contrastive learning and achieve better invariant mapping. To effect both grouping and discrimination objectives, we impose them on features separately derived from a shared representation. In addition, we propose normalized projection heads and unsupervised hyper-parameter tuning for the first time.Our extensive experimentation demonstrates that CLD is a lean and powerful add-on to existing methods such as NPID, MoCo, InfoMin, and BYOL on highly correlated, long-tail, or balanced datasets. It not only achieves new state-of-the-art on self-supervision, semi-supervision, and transfer learning benchmarks, but also beats MoCo v2 and SimCLR on every reported performance attained with a much larger compute. CLD effectively brings unsupervised learning closer to natural data and real-world applications. Our code is publicly available at: https://github.com/frank-xwang/CLD-UnsupervisedLearning.
CPM-2: Large-scale cost-effective pre-trained language models

In recent years, the size of pre-trained language models (PLMs) has grown by leaps and bounds. However, efficiency issues of these large-scale PLMs limit their utilization in real-world scenarios. We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference. (1) We introduce knowledge inheritance to accelerate the pre-training process by exploiting existing PLMs instead of training models from scratch. (2) We explore the best practice of prompt tuning with large-scale PLMs. Compared with conventional fine-tuning, prompt tuning significantly reduces the number of task-specific parameters. (3) We implement a new inference toolkit, namely infmoe, for using large-scale PLMs with limited computational resources. Based on our cost-effective pipeline, we pre-train two models: an encoder-decoder bilingual model with 11 billion parameters (CPM-2) and its corresponding MoE version with 198 billion parameters. In our experiments, we compare CPM-2 with mT5 on downstream tasks. Experimental results show that CPM-2 has excellent general language intelligence. Moreover, we validate the efficiency of infmoe when conducting inference of large-scale models having tens of billions of parameters on a single GPU. All source code and model parameters are available at https://github.com/TsinghuaAI/CPM.
AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching

Recently, records on stereo matching benchmarks are constantly broken by end-to-end disparity networks. However, the domain adaptation ability of these deep models is quite poor. Addressing such problem, we present a novel domain-adaptive pipeline called AdaStereo that aims to align multi-level representations for deep stereo matching networks. Compared to previous methods for adaptive stereo matching, our AdaStereo realizes a more standard, complete and effective domain adaptation pipeline. Firstly, we propose a non-adversarial progressive color transfer algorithm for input image-level alignment. Secondly, we design an efficient parameter-free cost normalization layer for internal feature-level alignment. Lastly, a highly related auxiliary task, self-supervised occlusion-aware reconstruction is presented to narrow down the gaps in output space. Our AdaStereo models achieve state-of-the-art cross-domain performance on multiple stereo benchmarks, including KITTI, Middlebury, ETH3D, and DrivingStereo, even outperforming disparity networks finetuned with target-domain ground-truths.
Dual Contradistinctive Generative Autoencoder

We present a new generative autoencoder model with dual contradistinctive losses to improve generative autoencoder that performs simultaneous inference (reconstruction) and synthesis (sampling). Our model, named dual contradistinctive generative autoencoder (DC-VAE), integrates an instance-level discriminative loss (maintaining the instance-level fidelity for the reconstruction/synthesis) with a set-level adversarial loss (encouraging the set-level fidelity for the reconstruction/synthesis), both being contradistinctive. Extensive experimental results by DC-VAE across different resolutions including 32×32, 64×64, 128×128, and 512×512 are reported. The two contradistinctive losses in VAE work harmoniously in DC-VAE leading to a significant qualitative and quantitative performance enhancement over the baseline VAEs without architectural changes. State-of-the-art or competitive results among generative autoencoders for image reconstruction, image synthesis, image interpolation, and representation learning are observed. DC-VAE is a general-purpose VAE model, applicable to a wide variety of downstream tasks in computer vision and machine learning.
Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting

Automatic code summarization frees software developers from the heavy burden of manual commenting and benefits software development and maintenance. Abstract Syntax Tree (AST), which depicts the source code's syntactic structure, has been incorporated to guide the generation of code summaries. However, existing AST based methods suffer from the difficulty of training and generate inadequate code summaries. In this paper, we present the Block-wise Abstract Syntax Tree Splitting method (BASTS for short), which fully utilizes the rich tree-form syntax structure in ASTs, for improving code summarization. BASTS splits the code of a method based on the blocks in the dominator tree of the Control Flow Graph, and generates a split AST for each code split. Each split AST is then modeled by a Tree-LSTM using a pre-training strategy to capture local non-linear syntax encoding. The learned syntax encoding is combined with code encoding, and fed into Transformer to generate high-quality code summaries. Comprehensive experiments on benchmarks have demonstrated that BASTS significantly outperforms state-of-the-art approaches in terms of various evaluation metrics. To facilitate reproducibility, our implementation is available at https://github.com/XMUDM/BASTS.
Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation

Data-driven based approaches, in spite of great success in many tasks, have poor generalization when applied to unseen image domains, and require expensive cost of annotation especially for dense pixel prediction tasks such as semantic segmentation. Recently, both unsupervised domain adaptation (UDA) from large amounts of synthetic data and semi-supervised learning (SSL) with small set of labeled data have been studied to alleviate this issue. However, there is still a large gap on performance compared to their supervised counterparts. We focus on a more practical setting of semi-supervised domain adaptation (SSDA) where both a small set of labeled target data and large amounts of labeled source data are available. To address the task of SSDA, a novel framework based on dual-level domain mixing is proposed. The proposed framework consists of three stages. First, two kinds of data mixing methods are proposed to reduce domain gap in both region-level and sample-level respectively. We can obtain two complementary domain-mixed teachers based on dual-level mixed data from holistic and partial views respectively. Then, a student model is learned by distilling knowledge from these two teachers. Finally, pseudo labels of unlabeled data are generated in a self-training manner for another few rounds of teachers training. Extensive experimental results have demonstrated the effectiveness of our proposed framework on synthetic-to-real semantic segmentation benchmarks.
Multi-obstacle path planning and optimization for mobile robot

In the past few decades, many results have been achieved in the research of mobile robot path planning, and they have been applied in simple scenarios, such as factory AGV, bank guide robot. However, path planning in highly dense and complex scenarios has become an important challenge for applications. Robots face dense map and complex obstacles and hardly find out an optimal path within a reasonable period, such as unmanned vehicles in freight ports and rescue robots in earthquake environment. Therefore, a multi-obstacle path planning and optimization method is proposed. In order to simplify complex environmental obstacles, the obstacles will be divided into basis obstacles and extension obstacles. Firstly, the basis obstacles and their contour point sets are determined according to the starting point and goal point. Furthermore, the basis obstacles are optimized by convex hulls, and then the corresponding basis point set is obtained. Secondly, the extension obstacles are determined by the basis point set, starting point and goal point, and then the corresponding extension point set is generated. After that, a path planner is designed by the multi-objective D* Lite algorithm for distance and smoothness in order to get reasonable and optimized path in a complex environment. Moreover, the path is smoothed by cubic bezier curves to fit the kinematic model of the robot. Finally, The proposed method conduct comparative experiments with other algorithms to verify its accuracy and computational efficiency of planning in complex environments.
Vaccination passports: Challenges for a future of air transportation

COVID-19 has been a major setback for air transportation; many airlines had to request for bailouts and the international flights connectivity is only restarting slowly. Accordingly, many aviation stakeholders put hopes into the ongoing process of vaccination, with the expectation that a high degree of vaccination will push the envelope for a return to normalcy. One prerequisite for reviving international air connectivity is the introduction of verification documents, also called "vaccination passports". These passports, however, come with several challenges which need to be overcome in order to enable recovery. In this study, we propose a framework covering five important aspects and policy challenges concerning the introduction of vaccination passports for a return of aviation, covering the topics: Competition, Epidemiology, Technology, Ethics, and Politics. Neglecting to appropriately address these challenges will likely not only delay the recovery, but possibly miss an important opportunity before new disastrous events appear on the horizon.
FrameExit: Conditional Early Exiting for Efficient Video Recognition

In this paper, we propose a conditional early exiting framework for efficient video recognition. While existing works focus on selecting a subset of salient frames to re-duce the computation costs, we propose to use a simple sampling strategy combined with conditional early exiting to enable efficient recognition. Our model automatically learns to process fewer frames for simpler videos and more frames for complex ones. To achieve this, we employ a cascade of gating modules to automatically determine the earliest point in processing where an inference is sufficiently reliable. We generate on-the-fly supervision signals to the gates to provide a dynamic trade-off between accuracy and computational cost. Our proposed model outperforms competing methods on three large-scale video benchmarks. In particular, on ActivityNet1.3 and mini-kinetics, we outperform the state-of-the-art efficient video recognition methods with 1.3× and 2.1 less GFLOPs, respectively. Addition-ally, our method sets× a new state of the art for efficient video understanding on the HVU benchmark.
Blockchain Technology for Secure Accounting Management: Research Trends Analysis

The scope of blockchain technology, initially associated with the cryptocurrency Bitcoin, is greater due to the multiple applications in various disciplines. Its use in accounting lies mainly in the fact that it reduces risks and the eventuality of fraud, eliminates human error, promotes efficiency, and increases transparency and reliability. This means that different economic sectors assume it as a recording and management instrument. The aim is to examine current and emerging research lines at a global level on blockchain technology for secure accounting management. The evolution of the publication of the number of articles between 2016 and 2020 was analyzed. Statistical and mathematical techniques were applied to a sample of 1130 records from the Scopus database. The data uncovered a polynomial trend in this period. The seven main lines of work were identified: blockchain, network security, information management, digital storage, edge computing, commerce, and the Internet of Things. The ten most outstanding emerging research lines are detected. This study provides the past and future thematic axes on this incipient field of knowledge, which is a tool for decision-making by academics, researchers, and directors of research investment programs.
Onboard Real-Time Aerial Tracking With Efficient Siamese Anchor Proposal Network

Object tracking approaches based on the Siamese network have demonstrated their huge potential in the remote sensing field recently. Nevertheless, due to the limited computing resource of aerial platforms and special challenges in aerial tracking, most existing Siamese-based methods can hardly meet the real-time and state-of-the-art performance simultaneously. Consequently, a novel Siamese-based method is proposed in this work for onboard real-time aerial tracking, i.e., SiamAPN. The proposed method is a no-prior two-stage method, i.e., Stage-1 for proposing adaptive anchors to enhance the ability of object perception and Stage-2 for fine-tuning the proposed anchors to obtain accurate results. Distinct from the traditional predefined anchors, the proposed anchors can adapt automatically to the tracking object. Besides, the internal information of adaptive anchors is utilized to feedback SiamAPN for enhancing the object perception. Attributing to the feature fusion network, different semantic information is integrated, enriching the information flow that is significant for robust aerial tracking. In the end, the regression and multiclassification operation refine the proposed anchors meticulously. Comprehensive evaluations on three well-known aerial tracking benchmarks have proven the superior performance of the presented approach. Moreover, to verify the practicability of the proposed method, SiamAPN is implemented onboard a typical embedded aerial tracking platform to conduct the real-world evaluations on specific aerial tracking scenarios, e.g., fast motion, long-term tracking, and low resolution. The results have demonstrated the efficiency and accuracy of the proposed approach, with a processing speed of over 30 frames/s. In addition, the image sequences in the real-world evaluations are collected and annotated as a new aerial tracking benchmark, i.e., UAVTrack112.
Review on Photovoltaic Power and Solar Resource Forecasting: Current Status and Trends

Abstract Photovoltaic (PV) power intermittence impacts electrical grid security and operation. Precise PV power and solar irradiation forecasts have been investigated as significant reducers of such impacts. Predicting solar irradiation involves uncertainties related to the characteristics of time series and their high volatility due to the dependence on many weather conditions. We propose a systematic review of PV power and solar resource forecasting, considering technical aspects related to each applied methodology. Our review covers the performance analysis of various physical, statistical, and machine learning models. These methodologies should contribute to decision-making, being applicable to different sites and climatic conditions. About 42% of the analyzed articles developed hybrid approaches, 83% performed short-term prediction, and more than 78% had, as forecast goal, PV power, solar irradiance, and solar irradiation. Considering spatial forecast scale, 66% predicted in a single field. As a trend for the coming years, we highlight the use of hybridized methodologies, especially those that optimize input and method parameters without loss of precision and postprocessing methodologies aiming at improvements in individualized applications.
Rethinking Maximum Mean Discrepancy for Visual Domain Adaptation

Existing domain adaptation approaches often try to reduce distribution difference between source and target domains and respect domain-specific discriminative structures by some distribution [e.g., maximum mean discrepancy (MMD)] and discriminative distances (e.g., intra-class and inter-class distances). However, they usually consider these losses together and trade off their relative importance by estimating parameters empirically. It is still under insufficient exploration so far to deeply study their relationships to each other so that we cannot manipulate them correctly and the model's performance degrades. To this end, this article theoretically proves two essential facts: 1) minimizing MMD equals to jointly minimizing their data variance with some implicit weights but, respectively, maximizing the source and target intra-class distances so that feature discriminability degrades and 2) the relationship between intra-class and inter-class distances is as one falls and another rises. Based on this, we propose a novel discriminative MMD with two parallel strategies to correctly restrain the degradation of feature discriminability or the expansion of intra-class distance; specifically: 1) we directly impose a tradeoff parameter on the intra-class distance that is implicit in the MMD according to 1) and 2) we reformulate the inter-class distance with special weights that are analogical to those implicit ones in the MMD and maximizing it can also lead to the intra-class distance falling according to 2). Notably, we do not consider the two strategies in one model due to 2). The experiments on several benchmark datasets not only prove the validity of our revealed theoretical results but also demonstrate that the proposed approach could perform better than some compared state-of-art methods substantially. Our preliminary MATLAB code will be available at https://github.com/WWLoveTransfer/.
Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting

Self-supervised learning has gained prominence due to its efficacy at learning powerful representations from unlabelled data that achieve excellent performance on many challenging downstream tasks. However, supervision-free pre-text tasks are challenging to design and usually modality specific. Although there is a rich literature of self-supervised methods for either spatial (such as images) or temporal data (sound or text) modalities, a common pretext task that benefits both modalities is largely missing. In this paper, we are interested in defining a self-supervised pre-text task for sketches and handwriting data. This data is uniquely characterised by its existence in dual modalities of rasterized images and vector coordinate sequences. We address and exploit this dual representation by proposing two novel cross-modal translation pre-text tasks for self-supervised feature learning: Vectorization and Rasterization. Vectorization learns to map image space to vector coordinates and rasterization maps vector coordinates to image space. We show that our learned encoder modules benefit both raster-based and vector-based downstream approaches to analysing hand-drawn data. Empirical evidence shows that our novel pre-text tasks surpass existing single and multi-modal self-supervision methods.
Learning Affinity-Aware Upsampling for Deep Image Matting

We show that learning affinity in upsampling provides an effective and efficient approach to exploit pairwise interactions in deep networks. Second-order features are commonly used in dense prediction to build adjacent relations with a learnable module after upsampling such as non-local blocks. Since upsampling is essential, learning affinity in upsampling can avoid additional propagation layers, offering the potential for building compact models. By looking at existing upsampling operators from a unified mathematical perspective, we generalize them into a second-order form and introduce Affinity-Aware Upsampling (A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U) where upsampling kernels are generated using a light-weight lowrank bilinear model and are conditioned on second-order features. Our upsampling operator can also be extended to downsampling. We discuss alternative implementations of A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U and verify their effectiveness on two detail-sensitive tasks: image reconstruction on a toy dataset; and a largescale image matting task where affinity-based ideas constitute mainstream matting approaches. In particular, results on the Composition-1k matting dataset show that A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> U achieves a 14% relative improvement in the SAD metric against a strong baseline with negligible increase of parameters (< 0.5%). Compared with the state-of-the-art matting network, we achieve 8% higher performance with only 40% model complexity.
Multi-View correlation distillation for incremental object detection

In real applications, new object classes often emerge after the detection model has been trained on a prepared dataset with fixed classes. Fine-tuning the old model with only new data will lead to a well-known phenomenon of catastrophic forgetting, which severely degrades the performance of modern object detectors. Due to the storage burden, data privacy and time consumption, sometimes it is impractical to train the model from scratch with all data of both old and new classes. In this paper, we propose a novel Multi-View Correlation Distillation (MVCD) based incremental object detection method, which explores the intra-feature correlations in the feature space of the object detector. To better transfer the knowledge learned from the old classes and maintain the ability to learn new classes, we select the sample-specific discriminative features from channel-wise, point-wise and instance-wise views. Meanwhile, the correlation distillation losses on the selective features are designed to regularize the learning of the incremental object detector. A new metric named Stability-Plasticity-mAP (SPmAP) is proposed to evaluate the incremental learning performance as a complementary metric to mAP, which integrates the metrics for the stability on old classes and the plasticity on new classes in incremental object detection. The extensive experiments conducted on VOC2007 and COCO demonstrate that MVCD achieves a better trade-off between stability and plasticity than state-of-the-art first-order distillation-based incremental object detection methods.