Reviewerly/exHarmony huggingface dataset:
A blockchain-based reputation system for trusted VANET nodes

Vehicular network applications, such as Local Danger Warnings (LDW), have become necessary as they provide services to drivers with warnings and instructions to ensure safe driving. However, before any vehicle reacts to an alert, it is essential to verify its veracity because an incorrect warning misleads other vehicles and causes unnecessary delays and detours in traffic. On the other hand, ignoring an alert or taking too long to verify it can cause accidents. A common way to address this problem is via a reputation system, where the veracity and plausibility of an alert message are assessed based on the reputation of the vehicle that discloses it. This paper describes a decentralized reputation system based on a consortium blockchain and smart contracts called BRS4VANETs. This system analyzes the reliability of data generated by a vehicle, detects malicious behavior, and contributes to decision-making. The complete functional architecture of the system, detailing the algorithms, evaluation metrics, communication protocol, and message formats, is described and implemented. Experiments with network and traffic simulators evaluated the risk of a false message attack, the effectiveness of the system and its overheads. The assessment demonstrates the importance of reputation systems and also shows the feasibility of having a decentralized system to store and disseminate reputation information based on blockchain technologies. Finally, the results also demonstrate that BRS4VANETs can successfully detect suspicious and malicious vehicles.
Differences in virtual and physical head orientation predict sickness during active head-mounted display-based virtual reality

During head-mounted display (HMD)-based virtual reality (VR), head movements and motion-to-photon-based display lag generate differences in our virtual and physical head pose (referred to as DVP). We propose that large-amplitude, time-varying patterns of DVP serve as the primary trigger for cybersickness under such conditions. We test this hypothesis by measuring the sickness and estimating the DVP experienced under different levels of experimentally imposed display lag (ranging from 0 to 222 ms on top of the VR system's ~ 4 ms baseline lag). On each trial, seated participants made continuous, oscillatory head rotations in yaw, pitch or roll while viewing a large virtual room with an Oculus Rift CV1 HMD (head movements were timed to a computer-generated metronome set at either 1.0 or 0.5 Hz). After the experiment, their head-tracking data were used to objectively estimate the DVP during each trial. The mean, peak, and standard deviation of these DVP data were then compared to the participant's cybersickness ratings for that trial. Irrespective of the axis, or the speed, of the participant's head movements, the severity of their cybersickness was found to increase with each of these three DVP summary measures. In line with our DVP hypothesis, cybersickness consistently increased with the amplitude and the variability of our participants' DVP. DVP similarly predicted their conscious experiences during HMD VR-such as the strength of their feelings of spatial presence and their perception of the virtual scene's stability.
Detecting and grouping keypoints for multi-person pose estimation using instance-aware attention

Bottom-up human pose estimation models detect keypoints and learn associative information between keypoints, usually requiring human predefined offset fields or embeddings for keypoints grouping (clustering). In this paper, we present a brand new method that can entirely solve these problems based on Transformer, making the grouping process free of the human-defined associative signals. Specifically, the self-attention in vision Transformer measures feature similarity between any pair of locations, which provides a metric space to associate keypoints together into corresponding human instances. However, the naive attention patterns formed in Transformer are still not subjectively controlled, so there is no guarantee that the keypoints only attend to the instances to which they belong. To address it we propose a novel approach of supervising self-attention to be instance-aware, simultaneously accomplishing multi-person keypoint detection and clustering. By doing so, we can group the detected keypoints to their corresponding instances, according to the pairwise attention scores. An additional benefit of our method is that the instance segmentation results of any number of people can be directly obtained from the supervised attention matrix, thereby simplifying the pixel assignment pipeline. The qualitative and quantitative results on the COCO shows that, with a very simple architecture design, our method can achieve comparable performance against the CNN-based bottom-up counterparts with fewer parameters, which also demonstrate a promising way to control self-attention mechanism behavior for specific purposes.
FRCNN-Based Reinforcement Learning for Real-Time Vehicle Detection, Tracking and Geolocation from UAS

In the last few years, uncrewed aerial systems (UASs) have been broadly employed for many applications including urban traffic monitoring. However, in the detection, tracking, and geolocation of moving vehicles using UAVs there are problems to be encountered such as low-accuracy sensors, complex scenes, small object sizes, and motion-induced noises. To address these problems, this study presents an intelligent, self-optimised, real-time framework for automated vehicle detection, tracking, and geolocation in UAV-acquired images which enlist detection, location, and tracking features to improve the final decision. The noise is initially reduced by applying the proposed adaptive filtering, which makes the detection algorithm more versatile. Thereafter, in the detection step, top-hat and bottom-hat transformations are used, assisted by the Overlapped Segmentation-Based Morphological Operation (OSBMO). Following the detection phase, the background regions are obliterated through an analysis of the motion feature points of the obtained object regions using a method that is a conjugation between the Kanade–Lucas–Tomasi (KLT) trackers and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering. The procured object features are clustered into separate objects on the basis of their motion characteristics. Finally, the vehicle labels are designated to their corresponding cluster trajectories by employing an efficient reinforcement connecting algorithm. The policy-making possibilities of the reinforcement connecting algorithm are evaluated. The Fast Regional Convolutional Neural Network (Fast-RCNN) is designed and trained on a small collection of samples, then utilised for removing the wrong targets. The proposed framework was tested on videos acquired through various scenarios. The methodology illustrates its capacity through the automatic supervision of target vehicles in real-world trials, which demonstrates its potential applications in intelligent transport systems and other surveillance applications.
A new evolving operator selector by using fitness landscape in differential evolution algorithm

Due to the problems of low accuracy and increasing control parameters in the existing parameter adaptive methods of differential evolution (DE) algorithm, in this paper a mutation operator selector and a parameter selector are proposed through Fitness Landscape (FL) analysing. At first, the performance differences of the two categories of mutation operators named DE/best/1 and DE/current-to-rand/1 were analyzed on many test problems. Secondly, the relationship between the FL and mutation operator is founded by using ensemble learning and decision tree, and achieved a classifier named mutation operator selector. Thirdly, the relationship between the FL and algorithm parameters is founded by using a neural network, and then a classifier named parameter selector is achieved. Finally, the improved DE algorithm equip with the two selectors is tested on the CEC2017 benchmark set. The results show that the proposed improved DE algorithm is outperforms both the basis DE algorithm and other three state-of-the-arts algorithms.
Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species

Current non-rigid object keypoint detectors perform well on a chosen kind of species and body parts, and require a large amount of labelled keypoints for training. Moreover, their heatmaps, tailored to specific body parts, cannot rec-ognize novel keypoints (keypoints not labelled for training) on unseen species. We raise an interesting yet challenging question: how to detect both base (annotated for training) and novel keypoints for unseen species given a few an-notated samples? Thus, we propose a versatile Few-shot Keypoint Detection (FSKD) pipeline, which can detect a varying number of keypoints of different kinds. Our FSKD provides the uncertainty estimation of predicted keypoints. Specifically, FSKD involves main and auxiliary keypoint representation learning, similarity learning, and keypoint localization with uncertainty modeling to tackle the local-ization noise. Moreover, we model the uncertainty across groups of keypoints by multivariate Gaussian distribution to exploit implicit correlations between neighboring keypoints. We show the effectiveness of our FSKD on (i) novel keypoint detection for unseen species, (ii) few-shot Fine-Grained Vi-sual Recognition (FGVR) and (iii) Semantic Alignment (SA) downstream tasks. For FGVR, detected keypoints improve the classification accuracy. For SA, we showcase a novel thin-plate-spline warping that uses estimated keypoint un-certainty under imperfect keypoint co respondences.
Metaverse Current Status and Prospects: Focusing on Metaverse Field Cases

The purpose of this study is to examine the current status of Metaverse by reflecting Metaverses’ various field cases and national policies, and discuss the opportunities and changes that can be obtained at the Metaverse utilization plan and individual, enterprise, and government levels.
H2FA R-CNN: Holistic and Hierarchical Feature Alignment for Cross-domain Weakly Supervised Object Detection

Cross-domain weakly supervised object detection (CD-WSOD) aims to adapt the detection model to a novel target domain with easily acquired image-level annotations. How to align the source and target domains is critical to the CDWSOD accuracy. Existing methods usually focus on partial detection components for domain alignment. In contrast, this paper considers that all the detection components are important and proposes a Holistic and Hier-archical Feature Alignment (H <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> FA) R-CNN. H2FA R-CNN enforces two image-level alignments for the backbone features, as well as two instance-level alignments for the RPN and detection head. This coarse-to-fine aligning hierarchy is in pace with the detection pipeline, i.e., processing the image-level feature and the instance-level features from bottom to top. Importantly, we devise a novel hybrid supervision method for learning two instance-level align-ments. It enables the RPN and detection head to simultane-ously receive weak/full supervision from the target/source domains. Combining all these feature alignments, H2 FA R-CNN effectively mitigates the gap between the source and target domains. Experimental results show that H2 FA R-CNN significantly improves cross-domain object detection accuracy and sets new state of the art on popular benchmarks. Code and pre-trained models are available at https://github.com/XuYunqiu/H2FA_R-CNN.
Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity

Image deraining is a typical low-level image restoration task, which aims at decomposing the rainy image into two distinguishable layers: clean image layer and rain layer. Most of the existing learning-based deraining methods are supervisedly trained on synthetic rainy-clean pairs. The domain gap between the synthetic and real rains makes them less generalized to different real rainy scenes. Moreover, the existing methods mainly utilize the property of the two layers independently, while few of them have considered the mutually exclusive relationship between the two layers. In this work, we propose a novel non-local contrastive learning (NLCL) method for unsupervised image deraining. Consequently, we not only utilize the intrinsic self-similarity property within samples, but also the mutually exclusive property between the two layers, so as to better differ the rain layer from the clean image. Specifically, the non-local self-similarity image layer patches as the positives are pulled together and similar rain layer patches as the negatives are pushed away. Thus the similar positive/negative samples that are close in the original space benefit us to enrich more discriminative representation. Apart from the self-similarity sampling strategy, we analyze how to choose an appropriate feature encoder in NLCL. Extensive experiments on different real rainy datasets demonstrate that the proposed method obtains state-of-the-art performance in real deraining.
Registration Based Few-Shot Anomaly Detection

This paper considers few-shot anomaly detection (FSAD), a practical yet under-studied setting for anomaly detection (AD), where only a limited number of normal images are provided for each category at training. So far, existing FSAD studies follow the one-model-per-category learning paradigm used for standard AD, and the inter-category commonality has not been explored. Inspired by how humans detect anomalies, i.e., comparing an image in question to normal images, we here leverage registration, an image alignment task that is inherently generalizable across categories, as the proxy task, to train a category-agnostic anomaly detection model. During testing, the anomalies are identified by comparing the registered features of the test image and its corresponding support (normal) images. As far as we know, this is the first FSAD method that trains a single generalizable model and requires no re-training or parameter fine-tuning for new categories. Experimental results have shown that the proposed method outperforms the state-of-the-art FSAD methods by 3%–8% in AUC on the MVTec and MPDD benchmarks. Source code is available at: https://github.com/MediaBrain-SJTU/RegAD .
A Re-Balancing Strategy for Class-Imbalanced Classification Based on Instance Difficulty

Real-world data often exhibits class-imbalanced distributions, where a few classes (a.k.a. majority classes) occupy most instances and lots of classes (a.k.a. minority classes) have few instances. Neural classification models usually perform poorly on minority classes when training on such imbalanced datasets. To improve the performance on minority classes, existing methods typically re-balance the data distribution at the class level, i.e., assigning higher weights to minority classes and lower weights to majority classes during the training process. However, we observe that even the majority classes contain difficult instances to learn. By reducing the weights of the majority classes, such instances would become more difficult to learn and hurt the overall performance consequently. To tackle this problem, we propose a novel instance-level re-balancing strategy, which dynamically adjusts the sampling probabilities of instances according to the instance difficulty. Here the instance difficulty is measured based on the learning speed of instance, which is inspired by the human-leaning process (i.e., easier instances will be learned faster). We theoretically prove the correctness and convergence of our resampling algorithm. Empirical experiments demonstrate that our method significantly outperforms state-of-the-art re-balancing methods on the class-imbalanced datasets.
RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior

In this work, we focus on scene flow learning on point clouds in a self-supervised manner. A real-world scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of rigid motion of each part. Inspired by this observation, we propose to generate pseudo scene flow for self-supervised learning based on piecewise rigid motion estimation, in which the source point cloud is decomposed into a set of local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to represent the flow, which together constitutes the pseudo scene flow labels of the entire scene to enable network training. Compared with most existing approaches relying on point-wise similarities for scene flow approximation, our method explicitly enforces region-wise rigid alignments, yielding locally rigid pseudo scene flow labels. We demonstrate the effectiveness of our self-supervised learning method on FlyingThings3D and KITTI datasets. Comprehensive experiments show that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even outperforming some super-vised counterparts.
Optimal Graph Structure Based Short-Term Solar PV Power Forecasting Method Considering Surrounding Spatio-Temporal Correlations

Accurate short-term photovoltaic (PV) power forecasting is of great significance for the safe and stable operation of power system. Spatial information from neighboring PV sites contributes to improving forecasting performance. However, most of the current methods considering the spatial information of neighboring sites indiscriminately use all sites data for modeling, which will lead to information redundancy, resulting in low forecasting accuracy. Therefore, this paper proposes a short-term solar power forecasting method based on optimal graph structure considering surrounding spatio-temporal correlations. Firstly, the neighboring sites data is analyzed from the perspective of geographical and weather factors to select typical PV sites. Secondly, based on the complex network theory, a new index is proposed to evaluate the connectivity of the graph structure, which improves the predictive ability of the Graph Convolutional Network (GCN) model. Finally, considering numerical weather prediction (NWP) data, a hierarchical directed graph structure is constructed to indicate the unidirectional relationship between input samples, which is used as the input of GCN model to mine the spatio-temporal correlation around the targeted site. Through carrying out the case study, the proposed method shows excellent performance in improving accuracy of power forecasting compared with other benchmark methods.
Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects

Rendering articulated objects while controlling their poses is critical to applications such as virtual reality or animation for movies. Manipulating the pose of an object, however, requires the understanding of its underlying structure, that is, its joints and how they interact with each other. Unfortunately, assuming the structure to be known, as existing methods do, precludes the ability to work on new object categories. We propose to learn both the appearance and the structure of previously unseen articulated objects by ob-serving them move from multiple views, with no joints annotation supervision, or information about the structure. We observe that 3D points that are static relative to one another should belong to the same part, and that adjacent parts that move relative to each other must be connected by a joint. To leverage this insight, we model the object parts in 3D as ellipsoids, which allows us to identify joints. We combine this explicit representation with an implicit one that compensates for the approximation introduced. We show that our method works for different structures, from quadrupeds, to single-arm robots, to humans. The code is available at https://github.com/NVlabs/watch-it-move and a version of this manuscript that uses animations is at https://arxiv.org/abs/2112.11347
Entropy-based Active Learning for Object Detection with Progressive Diversity Constraint

Active learning is a promising alternative to alleviate the issue of high annotation cost in the computer vision tasks by consciously selecting more informative samples to label. Active learning for object detection is more challenging and existing efforts on it are relatively rare. In this paper, we propose a novel hybrid approach to address this problem, where the instance-level uncertainty and diversity are jointly considered in a bottom-up manner. To balance the computational complexity, the proposed approach is designed as a two-stage procedure. At the first stage, an Entropy-based Non-Maximum Suppression (ENMS) is presented to estimate the uncertainty of every image, which performs NMS according to the entropy in the feature space to remove predictions with redundant information gains. At the second stage, a diverse prototype (DivProto) strategy is explored to ensure the diversity across images by progressively converting it into the intra-class and inter-class diversities of the entropy-based class-specific prototypes. Extensive experiments are conducted on MS COCO and Pascal VOC, and the proposed approach achieves state of the art results and significantly outperforms the other counter-parts, highlighting its superiority.
Rethinking IoU-based Optimization for Single-stage 3D Object Detection

Since Intersection-over-Union (IoU) based optimization maintains the consistency of the final IoU prediction metric and losses, it has been widely used in both regression and classification branches of single-stage 2D object detectors. Recently, several 3D object detection methods adopt IoU-based optimization and directly replace the 2D IoU with 3D IoU. However, such a direct computation in 3D is very costly due to the complex implementation and inefficient backward operations. Moreover, 3D IoU-based optimization is sub-optimal as it is sensitive to rotation and thus can cause training instability and detection performance deterioration. In this paper, we propose a novel Rotation-Decoupled IoU (RDIoU) method that can mitigate the rotation-sensitivity issue, and produce more efficient optimization objectives compared with 3D IoU during the training stage. Specifically, our RDIoU simplifies the complex interactions of regression parameters by decoupling the rotation variable as an independent term, yet preserving the geometry of 3D IoU. By incorporating RDIoU into both the regression and classification branches, the network is encouraged to learn more precise bounding boxes and concurrently overcome the misalignment issue between classification and regression. Extensive experiments on the benchmark KITTI and Waymo Open Dataset validate that our RDIoU method can bring substantial improvement for the single-stage 3D object detection. The code is available at https://github.com/hlsheng1/RDIoU .