Reviewerly/exHarmony huggingface dataset:
Thermal images-aware guided early fusion network for cross-illumination RGB-T salient object detection

RGB-T salient object detection (SOD) has been developed rapidly and achieved excellent results in recent years. However, some problems have not yet been solved. The current RGB-T datasets contain only a tiny amount of low-illumination data. The RGB-T SOD method trained based on these RGB-T datasets does not detect the salient objects in extremely low-illumination scenes very well. To improve the detection performance of low-illumination data, we can spend a lot of labor to label low-illumination data, but we tried a new idea to solve the problem by making full use of the properties of Thermal (T) images. Therefore, we propose a T-aware guided early fusion network for cross-illumination salient object detection. Specifically, in the training and testing stage, we use normal illumination data to train our network and then use low and extremely low-illumination data to verify the effectiveness of our method. In the early fusion stage, we propose a T-aware guided module (T-aware) for enhancing salient regions of RGB images at different illumination levels. Secondly, in the decoding stage, we use T images to guide the cross-modal fusion of RGB and T images. In addition, we propose a cross-modal fusion localization-remote correction module (CFL-RCM), which is used to deeply screen and correct redundant information generated by illumination variations. Comparative experiments on the VDT-2048 dataset validate the superior performance of our method on the cross-illumination RGB-T saliency detection. We also obtained favorable results on generalizability experiments with VT5000, VT1000, and VT821 datasets.
A Fast Blockchain-Based Federated Learning Framework With Compressed Communications

Recently, blockchain-based federated learning (BFL) has attracted intensive research attention due to that the training process is auditable and the architecture is serverless avoiding the single point failure of the parameter server in vanilla federated learning (VFL). Nevertheless, BFL tremendously escalates the communication traffic volume because all local model updates (i.e., changes of model parameters) obtained by BFL clients will be transmitted to all miners for verification and to all clients for aggregation. In contrast, the parameter server and clients in VFL only retain aggregated model updates. Consequently, the huge communication traffic in BFL will inevitably impair the training efficiency and hinder the deployment of BFL in reality. To improve the practicality of BFL, we are among the first to propose a fast blockchain-based communication-efficient federated learning framework by compressing communications in BFL, called BCFL. Meanwhile, we derive the convergence rate of BCFL with non-convex loss. To maximize the final model accuracy, we further formulate the problem to minimize the training loss of the convergence rate subject to a limited training time with respect to the compression rate and the block generation rate, which is a bi-convex optimization problem and can be efficiently solved. To the end, to demonstrate the efficiency of BCFL, we carry out extensive experiments with standard CIFAR-10 and FEMNIST datasets. Our experimental results not only verify the correctness of our analysis, but also manifest that BCFL can remarkably reduce the communication traffic by 95–98% or shorten the training time by 90–95% compared with BFL.
CNN‐Transformer for visual‐tactile fusion applied in road recognition of autonomous vehicles

Reliable autonomous driving requires comprehensive environment perception, among which the road recognition is critical for autonomous vehicles to achieve adaptability, reliability, and safety. Existing equipped sensors such as cameras, LiDAR, and accelerometers have been adopted widely for road recognition. However, single sensor based recognition methods present challenges in balancing high accuracy and adaptability. In this study, we propose a visual-tactile fusion road recognition system for autonomous vehicles. The visual modality is derived from the captured road images, and the tactile modality information comes from the designed intelligent tire system, containing a low-cost piezoelectric sensor. For accurate road recognition, we propose a multimodal fusion recognition network based on the CNN-transformer architecture. The visual and tactile modalities are fed into modality-specific SE-CNNs, which emphasize valuable input information to obtain weighted features. These features are subsequently input to "bottleneck" based fusion transformer encoders and output the recognition results. We design a fusion feature extractor to enhance the fusion representation capability and improve accuracy. The vehicle field experiments are conducted to build the dataset consisting of four road surfaces, and the results show that the proposed network achieves an accuracy of 99.48% in road recognition task.
Digitalisation of Enterprises in the Energy Sector: Drivers—Business Models—Prospective Directions of Changes

The energy sector has been a pioneer in the use of information and communication technologies for many years, and has undergone enormous changes in recent years as a result of the transition resulting from the fourth industrial revolution. In the paper, we examine and analyse relevant studies and their findings in order to show the current status of research on three selected aspects of digitalisation in energy sector enterprises. The paper sheds light on the diverse drivers influencing the digitalisation processes of energy sector companies. It also provides an overview of business models that are or will soon be implemented in the energy sector thanks to opportunities offered by digitalisation in response to observed trends in the energy market. Finally, it illustrates open research challenges and future dilemmas related to various aspects of energy sector digitalisation. The paper was prepared using the critical literature review method. It covers a large volume of the most recent and relevant literature referring to the three major research areas mentioned above. The literature review allowed us to identify the drivers influencing the digitalisation of energy companies and distinguish between those specific to this sector and those relating to all businesses as a part of the more general phenomena of Industry 4.0 and Industry 5.0. We also show how the digitalisation-based business model innovation presented in the literature empowers new energy producers and consumers through business models. We have also identified the most frequently indicated challenges and dilemmas in the digitalisation of energy companies related to the risk of the destabilisation of the energy market due to decentralisation, new requirements placed on the competences of energy sector workers, the new culture of interaction between energy suppliers and consumers, and the digital security of data used in the energy system.
A Gateway-based Interoperability Solution for Permissioned Blockchains

During the last years, interoperability has become one of the main challenges within blockchain research as blockchains increasingly require integration between each other. Indeed, blockchains work by design in silos of information as interoperability is not a native feature. The main efforts in the field target permissionless blockchains, such as Bitcoin and Ethereum. However, interoperability in permissioned blockchains remains as an almost untouched area of work as they introduce additional requirements focusing on privacy and identity. Although there are some interoperability solutions for permissioned blockchains, they are either high-level design proposals not providing concrete implementations or focus on interoperability issues between business applications and blockchain platforms. In this paper we propose a gateway-based platform-to-platform interoperability solution for permissioned blockchains, which comprises a detailed solution design and a reference implementation. The proposal was assessed through the development of a social security case scenario, based on the reference implementation and using two blockchain platforms: Hyperledger Fabric and Corda.
City-Scale Multi-Camera Vehicle Tracking of Vehicles based on YOLOv7

Multi-Target Multi-Camera Tracking has a massive Intelligent Traffic Surveillance System application domain. In this paper, we introduce the framework using YOLOv7, DeepSORT, and Re-ID to provide the identification of vehicles via a camera network. At first, we localize a vehicle in every single camera. Then, the Re-ID obtains the unique feature of each bounding box we crop from detection. We use DeepSORT and Trajectory Clustering from the list of features to keep the vehicle's identity from a single camera to a multi-camera. The experiment result shows the promise of the framework in the extensive system.
Heterogeneous question answering community detection based on graph neural network

Topic-based communities have gradually become a considerable medium for netizens to disseminate and acquire knowledge. These communities consist of entities (actual objects, e.g., a real answer or an actual question) with different types (users, questions and answers) and are usually hidden and overlapping. Nowadays, prevalent community question answering (CQA) platforms have formed mature communities by manually marked topics and extensive accumulated user behavior. However, the ever-growing various entities and complex overlapping topic communities make it inefficient to manually label entity tags (e.g., Question labels supplement domain features; Potential user tags indicate the user's specialty.). Therefore, there is an urgent need for a mechanism that automatically finds hidden semantic communities from user social behavior and lays a foundation for community construction and intelligent recommendation of QA platforms. In this paper, we propose a Heterogeneous Community Detection Approach Based on Graph Neural Network, called HCDBG, to detect heterogeneous communities in CQA. Firstly, we define entity relationships based on user interaction behavior and employ a heterogeneous information network to uniformly represent all connections. Afterward, we exploit the heterogeneous graph neural network to fuse content and topological features of nodes for graph embedding. Finally, we convert the community detection issue in CQA into an entity clustering task in the heterogeneous information network and improve the k-means method to achieve heterogeneous community detection. Based on our knowledge of the existing literature, it is an innovative research direction that utilizes the heterogeneous graph neural network to facilitate QA community detection. Extensive experiments on authentic question-answering datasets illustrate that HCDBG outperforms baseline methods in heterogeneous community detection.
Weakly-supervised semantic segmentation via online pseudo-mask correcting

Many existing weakly-supervised semantic segmentation methods focus on obtaining more accurate pseudo-masks with weak labels. So far pseudo-masks have come close to the ground truth. However, the potential of these high-quality pseudo-masks has not been fully explored. This is because pseudo-masks inevitably contain partial noisy labels. Deep segmentation networks tend to overfit noisy labels, which leads to poor generalization performance. In this work, we propose a new method to mitigate the damage caused by noisy labels. First, We use the exponential moving average (EMA) model of the online segmentation model as the teacher. Then, predictions from the teacher model are used to correct pseudo-masks online. Besides, learning with noisy labels has been extensively studied in classification tasks. We also introduce these anti-noise techniques and find them also effective for the segmentation task. Our proposed method can be easily embedded into existing weakly-supervised semantic segmentation algorithms and bring 2.3% IoU improvement without expensive computational cost. It also achieves the state-of-the-art performance on the PASCAL VOC 2012 benchmark dataset.
Federated fuzzy k-means for privacy-preserving behavior analysis in smart grids

Better understanding the behavior of various participants in smart grids, such as electricity consumers and generators, is important and beneficial for flexibility exploration and renewable energy accommodation. Clustering, as an effective data-driven approach to behavior analysis, has been widely applied for extracting the typical electricity consumption behavior of consumers and the bidding behavior of generators in smart grids. Traditionally, the clustering algorithms are implemented centrally with the assumption that data from all consumers or generators can be accessed. However, it may not be the case in the real world because the consumers and generators may not be able to or willing to share their own data due to privacy concerns or commercial competition. To address this issue, in this paper, we propose a federated fuzzy k-means method for privacy-preserving behavior analysis in smart grids. Specifically, two learning strategies, i.e., model averaging and gradient averaging, are designed for the implementation of the federated fuzzy k-means clustering. Both methods are investigated and comprehensively compared on both the electricity consumption behavior dataset and the generator bidding behavior dataset. Experimental results show that our proposed methods achieve similar performance to the traditional centralized fuzzy clustering method on independent and identically distributed (i.i.d.) data, as well as protecting the privacy of different participants in smart grids. As for non-i.i.d., the performance of the model averaging-based method worsen; in contrast, the gradient averaging-based method is more robust to this situation.
Prototyping to elicit user requirements for product development: Using head-mounted augmented reality when designing interactive devices

Designers of interactive devices are challenged by the need to accurately elicit user requirements from low-cost prototypes at the early stages of the design process. Head-mounted augmented reality (AR) can potentially assist in this process by economically representing physical-digital blended features with relatively high-fidelity prototypes. To explore this potential, we present and evaluate a head-mounted AR-enhanced hybrid prototyping system created in the context of a fan product development process. We conducted a mixed-methods study comparing the AR-enhanced prototyping method with a conventional prototyping method. The results reveal that the AR system can elicit similar user requirements as the conventional prototyping method with an improved overall experience.
A hybrid of transformer and CNN for efficient single image super-resolution via multi-level distillation

In recent years, single image super-resolution (SISR) models based on convolutional neural networks (CNN) have made significant progress and have gradually become the mainstream method. However, they still suffer from high computational costs, heavy memory consumption, and a limited receptive field. Although Vision Transformer has a stronger modeling capability and larger receptive field, it also incurs high computing power consumption and memory occupation. To address these issues, we propose a hybrid network of Transformer and CNN with cascaded feature distillation blocks for efficient image super-resolution (TCFDN), which can take advantage of both local information and long-term interactions while being flexible enough. Concretely, TCFDN consists of cascaded Transformer-CNN feature distillation blocks (TCFDB) and an upsampling module. The feature distillation pipeline of TCFDB can help our model gradually learn refined features with better representation ability while remaining lightweight. Besides, we also designed an enhanced Swin Transformer layer (ESTL) by replacing the multi-layer perceptron (MLP) in the standard Transformer with a convolutional feed-forward layer (CFF), which is more suitable for SR tasks. Then, the enhanced spatial attention embedded in TCFDB can boost SR performance further. Moreover, we observe that using a more advanced loss function, i.e., the contrastive loss, can also bring a PSNR gain of 0.01 dB–0.03 dB on public benchmarks. Extensive experiments demonstrate that TCFDN outperforms the state-of-the-art methods in terms of a better trade-off between performance and model size. Under the 4X SR task on the public benchmark Urban100, our TCFDN outperforms the second-best model by 0.37 dB in terms of PSNR. Compared with other state-of-the-art methods, the total number of parameters in TCFDN can be reduced by up to 32 % while maintaining competitive performance.
Spatial and temporal saliency based four-stream network with multi-task learning for action recognition

Action recognition is a challenging video understanding task for the following two reasons: (i) the complex video background impairs the recognition of desirable actions, and (ii) the fusion of spatial information and temporal information. In this paper, we proposed a novel spatial and temporal saliency based four-stream network with multi-task learning. The proposed model comprises four streams: an appearance stream (i.e. a spatial stream), a motion stream (i.e. a temporal stream), a novel spatial saliency stream and a novel temporal saliency stream. The spatial stream captures the global spatial information from videos using the sampled RGB video frames as the input. The temporal stream captures the global motion information of each pixel using the sampled stacked optical flow frames as the input. The novel spatial saliency stream is used to acquire spatial saliency information from spatial saliency frames, and the novel temporal saliency stream is used to acquire temporal saliency information from temporal saliency frames. In addition, based on the four streams, multi-task learning based LSTM is adopted, which can share the complementary knowledge between different CNN features extracted from different stacked frames. The multi-task learning based LSTM can capture long-term dependency relationships between the consecutive frames over temporal evolution, which take full advantage of CNNs and LSTMs. We conduct experiments on three popular video action recognition datasets, including the UCF101 action dataset, the HMDB51 action dataset and the large-scale Kinetics action dataset, to verify the effectiveness of the proposed network, and the results demonstrate that the proposed network achieves better performance than the state-of-the-art methods on these action recognition datasets.
EMO-MVS: Error-Aware Multi-Scale Iterative Variable Optimizer for Efficient Multi-View Stereo

Efficient dense reconstruction of objects or scenes has substantial practical implications, which can be applied to different 3D tasks (for example, robotics and autonomous driving). However, because of the expensive hardware required and the overall complexity of the all-around scenarios, efficient dense reconstruction using lightweight multi-view stereo methods has received much attention from researchers. The technological challenge of efficient dense reconstruction is maintaining low memory usage while rapidly and reliably acquiring depth maps. Most of the current efficient multi-view stereo (MVS) methods perform poorly in efficient dense reconstruction, this poor performance is mainly due to weak generalization performance and unrefined object edges in the depth maps. To this end, we propose EMO-MVS, which aims to accomplish multi-view stereo tasks with high efficiency, which means low-memory consumption, high accuracy, and excellent generalization performance. In detail, we first propose an iterative variable optimizer to accurately estimate depth changes. Then, we design a multi-level absorption unit that expands the receptive field, which efficiently generates an initial depth map. In addition, we propose an error-aware enhancement module, enhancing the initial depth map by optimizing the projection error between multiple views. We have conducted extensive experiments on challenging datasets Tanks and Temples and DTU, and also performed a complete visualization comparison on the BlenedMVS validation set (which contains many aerial scene images), achieving promising performance on all datasets. Among the lightweight MVS methods with low-memory consumption and fast inference speed, our F-score on the online Tanks and Temples intermediate benchmark is the highest, which shows that we have the best competitiveness in terms of balancing the performance and computational cost.
Traffic Sign Recognition Based on the YOLOv3 Algorithm

Traffic sign detection is an essential component of an intelligent transportation system, since it provides critical road traffic data for vehicle decision-making and control. To solve the challenges of small traffic signs, inconspicuous characteristics, and low detection accuracy, a traffic sign recognition method based on improved (You Only Look Once v3) YOLOv3 is proposed. The spatial pyramid pooling structure is fused into the YOLOv3 network structure to achieve the fusion of local features and global features, and the fourth feature prediction scale of 152 × 152 size is introduced to make full use of the shallow features in the network to predict small targets. Furthermore, the bounding box regression is more stable when the distance-IoU (DIoU) loss is used, which takes into account the distance between the target and anchor, the overlap rate, and the scale. The Tsinghua-Tencent 100K (TT100K) traffic sign dataset's 12 anchors are recalculated using the K-means clustering algorithm, while the dataset is balanced and expanded to address the problem of an uneven number of target classes in the TT100K dataset. The algorithm is compared to YOLOv3 and other commonly used target detection algorithms, and the results show that the improved YOLOv3 algorithm achieves a mean average precision (mAP) of 77.3%, which is 8.4% higher than YOLOv3, especially in small target detection, where the mAP is improved by 10.5%, greatly improving the accuracy of the detection network while keeping the real-time performance as high as possible. The detection network's accuracy is substantially enhanced while keeping the network's real-time performance as high as possible.
A federated learning-based approach to recognize subjects at a high risk of hypertension in a non-stationary scenario

Transferring data across nodes could raise concerns about data security and privacy. Federated learning is a tech- nological remedy for these problems. However, in a real federated scenario, there are two main issues: 1) it is difficult to collect locally a large and representative dataset, and 2) the data are non-stationary, where non-stationary means that the data increase in size over time, which may generate catastrophic forgetting events affecting the learning process. The aim of this paper is to investigate and assess the performance and behavior of a federated model during the occurrence of catastrophic forgetting events within the context of a non-stationary data scenario. To achieve this objective, the performance of a proposed federated learning approach has been evaluated in terms of the continuous flow of data in order to train a time series-based model. The results demonstrate the goodness of the solution in terms of performance, with improvements ranging from 2% to 28%. This indicates the benefits that the proposed approach brings to a node in terms of recovering from a catastrophic forgetting event, also taking into account the fact that the probability of catastrophic forgetting is higher at the beginning of the learning process.