Reviewerly/exHarmony huggingface dataset:
Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting

Crowd counting is a fundamental yet challenging task, which desires rich information to generate pixel-wise crowd density maps. However, most previous methods only used the limited information of RGB images and cannot well discover potential pedestrians in unconstrained scenarios. In this work, we find that incorporating optical and thermal information can greatly help to recognize pedestrians. To promote future researches in this field, we introduce a large-scale RGBT Crowd Counting (RGBT-CC) benchmark, which contains 2,030 pairs of RGB-thermal images with 138,389 annotated people. Furthermore, to facilitate the multimodal crowd counting, we propose a cross-modal collaborative representation learning framework, which consists of multiple modality-specific branches, a modality-shared branch, and an Information Aggregation-Distribution Module (IADM) to capture the complementary information of different modalities fully. Specifically, our IADM incorporates two collaborative information transfers to dynamically enhance the modality-shared and modality-specific representations with a dual information propagation mechanism. Extensive experiments conducted on the RGBT-CC benchmark demonstrate the effectiveness of our framework for RGBT crowd counting. Moreover, the proposed approach is universal for multimodal crowd counting and is also capable to achieve superior performance on the ShanghaiTechRGBD [22] dataset. Finally, our source code and benchmark have been released at http://lingboliu.com/RGBT_Crowd_Counting.html.
Interpreting Super-Resolution Networks with Local Attribution Maps

Image super-resolution (SR) techniques have been developing rapidly, benefiting from the invention of deep networks and its successive breakthroughs. However, it is acknowledged that deep learning and deep neural networks are difficult to interpret. SR networks inherit this mysterious nature and little works make attempt to understand them. In this paper, we perform attribution analysis of SR networks, which aims at finding the input pixels that strongly influence the SR results. We propose a novel attribution approach called local attribution map (LAM), which inherits the integral gradient method yet with two unique features. One is to use the blurred image as the baseline input, and the other is to adopt the progressive blurring function as the path function. Based on LAM, we show that: (1) SR networks with a wider range of involved input pixels could achieve better performance. (2) Attention networks and non-local networks extract features from a wider range of input pixels. (3) Comparing with the range that actually contributes, the receptive field is large enough for most deep networks. (4) For SR networks, textures with regular stripes or grids are more likely to be noticed, while complex semantics are difficult to utilize. Our work opens new directions for designing SR networks and interpreting low-level vision deep models.
Image Generators with Conditionally-Independent Pixel Synthesis

Existing image generator networks rely heavily on spatial convolutions and, optionally, self-attention blocks in order to gradually synthesize images in a coarse-to-fine manner. Here, we present a new architecture for image generators, where the color value at each pixel is computed independently given the value of a random latent vector and the coordinate of that pixel. No spatial convolutions or similar operations that propagate information across pixels are involved during the synthesis. We analyze the modeling capabilities of such generators when trained in an adversarial fashion, and observe the new generators to achieve similar generation quality to state-of-the-art convolutional generators. We also investigate several interesting properties unique to the new architecture.
Investigating gender fairness of recommendation algorithms in the music domain

Although recommender systems (RSs) play a crucial role in our society, previous studies have revealed that the performance of RSs may considerably differ between groups of individuals with different characteristics or from different demographics. In this case, a RS is considered to be unfair when it does not perform equally well for different groups of users. Considering the importance of RSs in the distribution and consumption of musical content worldwide, a careful evaluation of fairness in the context of music RSs is crucial. To this end, we first introduce LFM-2b, a novel large-scale real-world dataset of music listening records, comprising a subset to investigate bias of RSs regarding users’ demographics. We then define a notion of fairness based on the performance gap of a RS between the users with different demographics, and evaluate a variety of collaborative filtering algorithms in terms of accuracy and beyond-accuracy metrics to explore the fairness in the RS results toward a specific gender group. We observe the existence of significant discrepancies (unfairness) between the performance of algorithms across male and female user groups. Based on these discrepancies, we explore to what extent recommender algorithms lead to intensifying the underlying population bias in the final results. We also study the effect of a resampling strategy, commonly used as debiasing method , which yields slight improvements in the fairness measures of various algorithms while maintaining their accuracy and beyond-accuracy performance.
Data privacy preserving federated transfer learning in machinery fault diagnostics using prior distributions

Federated learning has been receiving increasing attention in the recent years, which improves model performance with data privacy among different clients. The intelligent fault diagnostic problems can be largely benefited from this emerging technology since the private data generally cannot leave local storage in the real industries. While promising federated learning performance has been achieved in the literature, most studies assume data from different clients are independent and identically distributed. In the real industrial scenarios, due to variations in machines and operating conditions, the data distributions are generally different across different clients, that significantly deteriorates the performance of federated learning. To address this issue, a federated transfer learning method is proposed in this article for machinery fault diagnostics. Under the condition that data from different clients cannot be communicated, prior distributions are proposed to indirectly bridge the domain gap. In this way, client-invariant features can be extracted for diagnostics while the data privacy is preserved. Experiments on two rotating machinery datasets are implemented for validation, and the results suggest the proposed method offers an effective and promising approach for federated transfer learning in fault diagnostic problems.
Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization

Training deep networks with limited labeled data while achieving a strong generalization ability is key in the quest to reduce human annotation efforts. This is the goal of semi-supervised learning, which exploits more widely available unlabeled data to complement small labeled data sets. In this paper, we propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels. Concretely, we learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of un-labeled images supplemented with only few labeled ones. We build our architecture on top of StyleGAN2 [45], augmented with a label synthesis branch. Image labeling at test time is achieved by first embedding the target image into the joint latent space via an encoder network and test-time optimization, and then generating the label from the inferred embedding. We evaluate our approach in two important domains: medical image segmentation and part-based face segmentation. We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization, such as transferring from CT to MRI in medical imaging, and photographs of real faces to paintings, sculptures, and even cartoons and animal faces. Project Page: https://nv-tlabs.github.io/semanticGAN/
Psychological determinants of users’ adoption and word-of-mouth recommendations of smart voice assistants

Individuals are increasingly using Smart Voice Assistants (SVA) to derive functional, hedonic, and symbolic benefits. SVA adoption is in the nascent stage and we have little knowledge about what drives SVA usage. Following the ‘attitude shapes behavior’ approach, this study examines the role of hedonic and utilitarian attitudes on SVA usage and word-of-mouth (WOM) recommendations. The study also investigates five antecedents (playfulness, escapism, anthropomorphism, visual appeal, and social presence) to both attitudes. Through an online survey of 360 respondents, the study suggests that playfulness and escapism positively influence hedonic attitude. On the other hand, anthropomorphism, visual appeal, and social presence determine utilitarian attitude. Further, utilitarian attitude has a stronger impact (vs. hedonic attitude) on SVA usage and WOM recommendations. The findings reveal that individuals who perceive SVA as a symbol of high prestige (vs. low prestige) are less likely to relate playfulness to hedonic gratifications. In contrast, people attributing prestige to SVA are more likely to use it as an escape route from everyday life. These findings contribute to the growing literature on SVA adoption and offer insightful recommendations to various stakeholders to increase the likelihood of SVA adoption and generating favorable WOM recommendations.
Neural network approximation

Neural networks (NNs) are the method of choice for building learning algorithms. They are now being investigated for other numerical tasks such as solving high-dimensional partial differential equations. Their popularity stems from their empirical success on several challenging learning problems (computer chess/Go, autonomous navigation, face recognition). However, most scholars agree that a convincing theoretical explanation for this success is still lacking. Since these applications revolve around approximating an unknown function from data observations, part of the answer must involve the ability of NNs to produce accurate approximations. This article surveys the known approximation properties of the outputs of NNs with the aim of uncovering the properties that are not present in the more traditional methods of approximation used in numerical analysis, such as approximations using polynomials, wavelets, rational functions and splines. Comparisons are made with traditional approximation methods from the viewpoint of rate distortion, i.e. error versus the number of parameters used to create the approximant. Another major component in the analysis of numerical approximation is the computational time needed to construct the approximation, and this in turn is intimately connected with the stability of the approximation algorithm. So the stability of numerical approximation using NNs is a large part of the analysis put forward. The survey, for the most part, is concerned with NNs using the popular ReLU activation function. In this case the outputs of the NNs are piecewise linear functions on rather complicated partitions of the domain of f into cells that are convex polytopes. When the architecture of the NN is fixed and the parameters are allowed to vary, the set of output functions of the NN is a parametrized nonlinear manifold. It is shown that this manifold has certain space-filling properties leading to an increased ability to approximate (better rate distortion) but at the expense of numerical stability. The space filling creates the challenge to the numerical method of finding best or good parameter choices when trying to approximate.
Self-supervised learning methods and applications in medical imaging analysis: a survey

The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered an effective solution for the scarcity of annotated medical data. This article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with a concentration on their applications in the field of medical imaging analysis. The article covers a set of the most recent self-supervised learning methods from the computer vision field as they are applicable to the medical imaging analysis and categorize them as predictive, generative, and contrastive approaches. Moreover, the article covers 40 of the most recent research papers in the field of self-supervised learning in medical imaging analysis aiming at shedding the light on the recent innovation in the field. Finally, the article concludes with possible future research directions in the field.
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network

In this paper, we abandon the dominant complex language model and rethink the linguistic learning process in the scene text recognition. Different from previous methods considering the visual and linguistic information in two separate structures, we propose a Visual Language Modeling Network (VisionLAN), which views the visual and linguistic information as a union by directly enduing the vision model with language capability. Specially, we introduce the text recognition of character-wise occluded feature maps in the training stage. Such operation guides the vision model to use not only the visual texture of characters, but also the linguistic information in visual context for recognition when the visual cues are confused (e.g. occlusion, noise, etc.). As the linguistic information is acquired along with visual features without the need of extra language model, Vision-LAN significantly improves the speed by 39% and adaptively considers the linguistic information to enhance the visual features for accurate recognition. Furthermore, an Occlusion Scene Text (OST) dataset is proposed to evaluate the performance on the case of missing character-wise visual cues. The state of-the-art results on several benchmarks prove our effectiveness. Code and dataset are available at https://github.com/wangyuxin87/VisionLAN.
IDM: An Intermediate Domain Module for Domain Adaptive Person Re-ID

Unsupervised domain adaptive person re-identification (UDA re-ID) aims at transferring the labeled source domain’s knowledge to improve the model’s discriminability on the unlabeled target domain. From a novel perspective, we argue that the bridging between the source and target domains can be utilized to tackle the UDA re-ID task, and we focus on explicitly modeling appropriate intermediate domains to characterize this bridging. Specifically, we propose an Intermediate Domain Module (IDM) to generate intermediate domains’ representations on-the-fly by mixing the source and target domains’ hidden representations using two domain factors. Based on the "shortest geodesic path" definition, i.e., the intermediate domains along the shortest geodesic path between the two extreme domains can play a better bridging role, we propose two properties that these intermediate domains should satisfy. To ensure these two properties to better characterize appropriate intermediate domains, we enforce the bridge losses on intermediate domains’ prediction space and feature space, and enforce a diversity loss on the two domain factors. The bridge losses aim at guiding the distribution of appropriate intermediate domains to keep the right distance to the source and target domains. The diversity loss serves as a regularization to prevent the generated intermediate domains from being over-fitting to either of the source and target domains. Our proposed method outperforms the state-of-the-arts by a large margin in all the common UDA re-ID tasks, and the mAP gain is up to 7.7% on the challenging MSMT17 benchmark. Code is available at https://github.com/SikaStar/IDM.
PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving

The accelerating development of autonomous driving technology has placed greater demands on obtaining large amounts of high-quality data. Representative, labeled, real world data serves as the fuel for training deep learning networks, critical for improving self-driving perception algorithms. In this paper, we introduce PandaSet, the first dataset produced by a complete, high-precision autonomous vehicle sensor kit with a no-cost commercial license. The dataset was collected using one 360&#x00B0; mechanical spinning LiDAR, one forward-facing, long-range LiDAR, and 6 cameras. The dataset contains more than 100 scenes, each of which is 8 seconds long, and provides 28 types of labels for object classification and 37 types of labels for semantic segmentation. We provide baselines for LiDAR-only 3D object detection, LiDAR-camera fusion 3D object detection and LiDAR point cloud segmentation. For more details about PandaSet and the development kit, see https://scale.com/open-datasets/pandaset.
Understanding Graph Embedding Methods and Their Applications

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 20 December 2020Accepted: 23 February 2021Published online: 04 November 2021Keywordsdeep neural networks, high-dimensionality, latent space, similarity, uncertainty quantification, intrinsic dimension, graph embedding at scaleAMS Subject Headings68T07, 05C62, 94A15, 68T37, 68R10, 68T30Publication DataISSN (print): 0036-1445ISSN (online): 1095-7200Publisher: Society for Industrial and Applied MathematicsCODEN: siread
Multi-class fuzzy support matrix machine for classification in roller bearing fault diagnosis

As a new classification method with the matrix as the input, support matrix machine (SMM) makes full use of the structured information between rows and columns of the input matrix to establish an accurate prediction model, which has been widely used in the field of fault diagnosis. However, the principle of SMM is to construct two parallel hyperplanes to complete the segmentation between different types of samples. When there are noise and outliers in the sample data, it is difficult for SMM to construct an ideal parallel hyperplane. In view of this, this paper proposes a multi-class fuzzy support matrix machine (MFSMM) by establishing nonparallel hyperplane objective function and integrating fuzzy attributes. In MFSMM, MFSMM establishes two nonparallel fuzzy hyperplanes by objective function, which maximizes the interval between any two fuzzy hyperplanes while considering the sample structure information. Meanwhile, fuzzy plane assigns different membership degrees to different training samples, which greatly reduces the influence of noise on the construction of optimal classification hyperplane. By analyzing two kinds of roller bearing experimental data, the results show that MFSMM has higher classification accuracy and stronger fault tolerance for samples with uncertain information.
Solar photovoltaic power prediction using different machine learning methods

The main aim of the present study is to explore the relationship between numerous input parameters and the solar photovoltaic (PV) power using machine learning (ML) models. Two different ML approaches such as support vector machine (SVM) and Gaussian process regression (GPR) were considered and compared. The basic input parameters including solar PV panel temperature, ambient temperature, solar flux, time of the day and relative humidity were considered for predicting the solar PV power. The results showed that among the proposed ML approaches, Matern 5/2 GPR algorithm provided the optimal performance; whereas cubic SVM had the worst performance. Furthermore, the predicted output results are in good agreement with the experimental values, indicating that the proposed ML approaches are appropriate for use in predicting the power of different solar PV panel. Additionally, to showcase the effectiveness and the accuracy of SVM and GPR models in forecasting solar PV power, the results of these models are compared using root mean squared error (RMSE) and mean absolute error (MAE) criteria.
An Improved Swin Transformer-Based Model for Remote Sensing Object Detection and Instance Segmentation

Remote sensing image object detection and instance segmentation are widely valued research fields. A convolutional neural network (CNN) has shown defects in the object detection of remote sensing images. In recent years, the number of studies on transformer-based models increased, and these studies achieved good results. However, transformers still suffer from poor small object detection and unsatisfactory edge detail segmentation. In order to solve these problems, we improved the Swin transformer based on the advantages of transformers and CNNs, and designed a local perception Swin transformer (LPSW) backbone to enhance the local perception of the network and to improve the detection accuracy of small-scale objects. We also designed a spatial attention interleaved execution cascade (SAIEC) network framework, which helped to strengthen the segmentation accuracy of the network. Due to the lack of remote sensing mask datasets, the MRS-1800 remote sensing mask dataset was created. Finally, we combined the proposed backbone with the new network framework and conducted experiments on this MRS-1800 dataset. Compared with the Swin transformer, the proposed model improved the mask AP by 1.7%, mask APS by 3.6%, AP by 1.1% and APS by 4.6%, demonstrating its effectiveness and feasibility.
New 4D chaotic system with hidden attractors and self-excited attractors and its application in image encryption based on RNG

A new 4D chaotic system with self-excited attractors or hidden attractors is designed and the generation of self-excited attractors and hidden attractors depends on the parameters of the new 4D chaotic system. The detailed dynamical properties of the designed new 4D chaotic system have been vividly demonstrated in terms of phase portraits, bifurcation diagrams, and Lyapunov exponents. When fixing the system parameters and changing the initial value, different kinds of coexisting asymmetric attractors are discovered in the new 4D chaotic system. Besides, the corresponding analog electronic circuit of the 4D chaotic system is designed and implemented to verify its feasibility, concurrently the random number generator and an image encryption algorithm based on the designed new 4D chaotic system are provided. Then the effectiveness of the image encryption system is verified by some basic security analysis.