Reviewerly/exHarmony huggingface dataset:
Thermal Image Super-Resolution Challenge Results - PBVS 2022

This paper presents results from the third Thermal Image Super-Resolution (TISR) challenge organized in the Perception Beyond the Visible Spectrum (PBVS) 2022 workshop. The challenge uses the same thermal image dataset as the first two challenges, with 951 training images and 50 validation images at each resolution. A set of 20 images was kept aside for testing. The evaluation tasks were to measure the PSNR and SSIM between the SR image and the ground truth (HR thermal noisy image downsampled by four), and also to measure the PSNR and SSIM between the SR image and the semi-registered HR image (acquired with another camera). The results outperformed those from last year’s challenge, improving both evaluation metrics. This year, almost 100 teams participants registered for the challenge, showing the community’s interest in this hot topic.
A Cerebellum-Inspired Prediction and Correction Model for Motion Control of a Musculoskeletal Robot

It is an important issue that how to regulate the existing control models of musculoskeletal robots to improve the ability of motion learning and generalization. In this article, based on the motion modulation function of the cerebellum, a cerebellum-inspired prediction and correction model is proposed to carry out feedforward regulation of the original controller. First, drawing on the reservoir computing mechanism of the cerebellar granular layer, the cerebellum prediction model is established by using the echo state network. Incremental learning for the network is achieved using the replay method, which is able to process control signals with different distributions. The cerebellum prediction network can accurately predict the motion results of the robot under the action of the time-series control signals. Second, referring to the neural pathways of the cerebellum, the cerebellum correction model is constructed. The network learning rules are designed by drawing on the long-term potentiation and depression processes of cerebellar synaptic plasticity. The characteristics of the parameters in the network weight update equations are further analyzed. And the hyperparameter update rules of the correction network weights are proposed. To simulate the function of the cerebellum involved in the limb ballistic movement, the adaptive adjustment method for cerebellum correction duration is proposed. The cerebellum correction network can accurately modulate the original control signals by using the motion prediction results. In experiments, a musculoskeletal robot is used to verify the movement effects under the control of the cerebellum model. The results show that the cerebellum-inspired model can effectively improve the motion accuracy and learning efficiency of the musculoskeletal robot, and enhance the motion generalization ability and system robustness of the robot.
Meta-relationship for course recommendation in MOOCs

Course recommendations are used to help students with different needs to choose courses. However, students’ needs are not always determined by their personal interests, they are also influenced by different curriculum settings, different teacher teams and other factors. Current course recommendation methods lack the consideration of complex relational semantic information that affects students’ needs, resulting in unsatisfied recommendation. To address this issue, we propose Meta-Relationship Course Recommendation (MRCRec) to enrich the expression of relational information. Focusing on complex semantic information of multi-entity relationship and entity association, we construct creatively the multi-entity relational self-symmetric meta-path (MSMP) and associative relational self-symmetric meta-graph (ASMG), which are referred as meta-relationship (MR). We also design an algorithm of meta-relationship correlation measure (MRCor) to obtain semantic correlational information. Then, we adopt the graph embedding to mine and fuse the latent representations of users and that of courses as user preference and course characteristic, respectively. Finally, we optimize matrix factorization to complete recommended task. Comprehensive experiments are conducted on the MOOCCube dataset and XuetangX dataset. The results show that MRCRec can effectively recommend courses for users.
Zero-shot Learning Using Multimodal Descriptions

Zero-shot learning (ZSL) tackles the problem of recognizing unseen classes using only semantic descriptions, e.g., attributes. Current zero-shot learning techniques all assume that a single vector of attributes suffices to describe each category. We show that this assumption is incorrect. Many classes in real world problems have multiple modes of appearance: male and female birds vary in appearance, for instance. Domain experts know this and can provide attribute descriptions of the chief modes of appearance for each class. Motivated by this, we propose the task of multimodal zero-shot learning, where the learner must learn from these multimodal attribute descriptions. We present a technique for addressing this problem of multimodal ZSL that outperforms the unimodal counterpart significantly. We posit that multimodal ZSL is more practical for real-world problems where complex intra-class variation is common.
Improving Robustness to Texture Bias via Shape-focused Augmentation

Despite significant progress of deep neural networks in image classification, it has been reported that CNNs trained on ImageNet have heavily focused on local texture information, rather than capturing complex visual concepts of the objects. To delve into this phenomenon, recent studies proposed to generate images with modified texture information for training the model. However, these methods largely sacrifice the classification accuracy on the in-domain dataset while achieving improved performance on the out-of-distribution dataset. Motivated by the fact that human tends to focus on shape information, we aim to resolve this issue by proposing a shape-focused augmentation where the texture in the object’s foreground and background are separately changed. Key idea is that by applying different modifications to the inside and outside of an object, not only the bias toward texture is reduced but also the model is induced to focus on shape. Experiments show that the proposed method successfully reduces texture bias and also improves the classification performance on the original dataset.
Text Query based Traffic Video Event Retrieval with Global-Local Fusion Embedding

Retrieving event videos based on textual description is a promising research topic in the fast-growing data field. However, traffic data increases every day, so it is essential to need intelligent traffic system management in conjunction with humans to speed up the search. We propose a multi-module system that delivers accurate results that meet objectives, including explainability and scalability at the same time. Our solution considers neighbors entities related to the mentioned object to represent an event by rule-based, which can represent an event by the relationship of multiple objects. In our proposed retrieval method, we add our modified model of Alibaba solution with the post-processing techniques from HCMUS method in AI City Challenge 2021 to boost the explainability of the obtained results. As the traffic data is vehicle-centric, we apply two language and image modules to analyze the input data and obtain the global properties of the context and the internal attributes of the vehicle. We introduce a one-on-one dual training strategy for each representation vector to optimize the interior features for the query. Finally, a refinement module gathers previous results to enhance the final retrieval result. We benchmarked our approach on the data of the AI City Challenge 2022 and obtained the competitive results at an MMR of 0.3611. We were ranked in the top 4 on 50% of the test set and in the top 5 on the full set.
Motion Aware Double Attention Network for Dynamic Scene Deblurring

Motion deblurring in dynamic scenes is a challenging task when the blurring is caused by one or a combination of various reasons such as moving objects, camera movement, etc. Since event cameras can detect changes in intensity with a low latency, necessary motion information is inherently captured in event data, which could be quite useful for deblurring standard camera images. The degradation intensity does not show homogeneity across an image due to factors like object depth, speed, etc. We propose a two-branch network structure, Motion Aware Double Attention Network (MADANet), that pays special attention to areas with high blur. As part of the network, event data is first used by the high blur region segmentation module that creates a probability-like score for areas exhibiting high relative motion to the camera. Then, the event data is also injected to feature maps in the main body, where there is a second attention mechanism available for each branch. The effective usage of event data and two-level attention mechanisms makes the network very compact. During the experiment, it was shown that the proposed network could achieve state-of-the-art performance not only on the bench-mark dataset from GoPro, but also on two newly collected datasets, one of which contains real event data.
Can domain adaptation make object recognition work for everyone?

Despite the rapid progress in deep visual recognition, modern computer vision datasets significantly overrepresent the developed world and models trained on such datasets underperform on images from unseen geographies. We investigate the effectiveness of unsupervised domain adaptation (UDA) of such models across geographies at closing this performance gap. To do so, we first curate two shifts from existing datasets to study the Geographical DA problem, and discover new challenges beyond data distribution shift: context shift, wherein object surroundings may change significantly across geographies, and subpopulation shift, wherein the intra-category distributions may shift. We demonstrate the inefficacy of standard DA methods at Geographical DA, highlighting the need for specialized geographical adaptation solutions to address the challenge of making object recognition work for everyone.
Super-Resolution based Video Coding Scheme

In this paper, we present a super-resolution-based video coding scheme that compresses video data by combining traditional hybrid video coding and Convolutional neural network-based video coding. During video encoding, downsampling reduces the resolution of an original video in both horizontal and vertical directions to reduce original video data, and Convolutional neural networkbased super-resolution is employed after the decoding process to recover the resolution of the reconstructed video during upsampling. For core encoding and decoding processes, the latest video coding standard (i.e., VVC/H.266) is conducted. The experimental results show that the proposed method can provide efficient coding performance while maintaining good visual quality.
Emphasizing Complementary Samples for Non-literal Cross-modal Retrieval

Existing cross-modal retrieval methods assume a straightforward relationship where images and text contain portrayals or mentions of the same objects. In contrast, real-world image-text pairs (e.g. an image and its caption in a news article) often feature more complex relations. Importantly, not all image-text pairs have the same relationship: in some pairs, image and text may be more closely aligned, while others are more loosely aligned hence complementary. In order to ensure the model learns a semantically robust space which captures nuanced relationships, care must be taken that loosely-aligned image-text pairs have a strong enough impact on learning. In this paper, we propose a novel approach to prioritize loosely-aligned samples. Unlike prior sample weighting methods, ours relies on estimating to what extent semantic similarity is preserved in the separate channels (images/text) in the learned multimodal space. In particular, the image-text pair weights in the retrieval loss focus learning towards samples from diverse or discrepant neighborhoods: samples where images or text that were close in a semantic space, are distant in the cross-modal space (diversity), or where neighbor relations are asymmetric (discrepancy). Experiments on three challenging datasets exhibiting abstract image-text relations, as well as COCO, demonstrate significant performance gains compared to recent state-of-the-art models and sample weighting approaches.
Communication-Efficient Federated Data Augmentation on Non-IID Data

Federated learning (FL) is an attractive distributed machine learning framework due to the property of privacy preservation. The implementation of FL encounters the challenge of the Non-Independent and Identically Distributed (Non-IID) data across devices. This work focuses on mitigating the impact of Non-IID datasets in wireless communications. To achieve this goal, we propose a generative models-based federated data augmentation strategy (FedDA) with privacy preservation and communication efficiency. In FedDA, the Conditional AutoEncoder (CVAE) is adopted to generate the missing samples on Non-IID datasets. The Knowledge Distillation Mechanism is introduced to achieve Federated learning, through which knowledge is shared, rather than model parameters or gradients. The knowledge is designed based on the hidden-layer features to reduce the communication overhead and protect raw data privacy. Meanwhile, to generate cross-class samples that are easy to classify, the latent variables in CVAE are constrained and the attention mechanism is introduced. Extensive experiments are conducted on FashionMNIST datasets and CIFAR-10 with different data distributions. The results show that FedDA can improve the model accuracy by up to 8% while reducing the communication overhead by up to 2×, compared to classic baselines with highly Non-IID data.
Watermarking Protocol for Deep Neural Network Ownership Regulation in Federated Learning

With the wide application of deep learning models, it is important to verify an author’s possession over a deep neural network model by watermarks and protect the model. The development of distributed learning paradigms such as federated learning raises new challenges for model protection. Each author should be able to conduct independent verification and trace traitors. To meet those requirements, we propose a watermarking protocol, Merkle-Sign to meet the prerequisites for ownership verification in federated learning. Our work paves the way for generalizing watermark as a practical security mechanism for protecting deep learning models in distributed learning platforms.
Non-Local Spatiotemporal Correlation Attention for Action Recognition

To well perceive human actions, it may be favorable only to consider useful clues of human and scene context during the recognition process. Deep Neural Networks (DNNs) used to build up blocks associate with local neighborhood correlation computations at spatial and temporal domains individually. In this work, we develop a DNN which consists of a 3D convolutional neural network, Non-Local SpatioTemporal Correlation Attention (NSTCA) module, and classifier to retrieve meaningful semantic context for effective action identification. Particularly, the proposed NSTCA module extracts advantageous visual clues of both spatial and temporal features via transposed feature correlation computations rather than individual spatial and temporal attention computations. In the experiments, the dataset of traffic police was fulfilled for analysis and comparison. The experimental outcome exhibits that the proposed DNN obtains an average accuracy of 98.2% which is superior to those from the conventional DNNs. Therefore, the DNN proposed herein can be widely applied to discern various actions of subjects in video scenes.
An improved particle swarm optimization using long short-term memory model for positioning control of a coplanar <i>XXY</i> stage

This paper presents XXY stage alignment with an image feedback system consisting of two charge-coupled devices (CCDs) and a proportional–integral–derivative (PID) image servo system tuned by particle swarm optimization (PSO). The initial stop values for the PSO algorithm often cause problems in calculation. A long short-term memory (LSTM) deep learning model can identify long-term dependences and sequential model data. Using LSTM to improve the PSO algorithm for searching best fitness value. LSTM predicts the fitness value of PSO, eliminating the need to preassess fitness, value, and uses the predicted fitness value to adjust the inertia weights of PSO adaptively. This allows the PSO search to be terminated in an early stage and reduces the time required for the search. Proposed method was applied to a visual servo system consisting of two CCD cameras and a personal computer–based PID controller for XXY stage motion. The experimental results indicate that LSTM can reduce the time required for PSO fitness search for controlling XXY stage motion under different conditions successfully. Through the training of the LSTM model, the stage positioning error and time of finding optimal control parameters for a coplanar XXY stage can reduced significantly for in-line inspection processes.
Generalized Ethereum Blockchain-based recommender system framework

This study presents a new generalized recommender system framework based on Blockchain technology called BC-Rec. Blockchain technology is a decentralized data-sharing ledger across a large network of untrusted entities. The presented idea is motivated by the fact that e-business platforms share common data about users, items, and sellers. In other words, a new user/seller in a particular e-business platform may be a trusted user/seller in other e-business platforms. Also, a new item in a particular marketplace may have a historical record in other marketplaces. The core idea behind BC-Rec is to adopt Blockchain technology to share the data required to generate the recommendation between the e-business platforms. BC-Rec architecture is designed based on a permission-less public Blockchain network, Ethereum Blockchain. The key feature of BC-Rec is the smart contract deployed to determine the agreement between certain entities on the Blockchain. We expect that the proposed framework will help alleviate cold-start, sparsity, and shilling attack problems. • Blockchain technology is a decentralized data-sharing ledger across a large network of untrusted entities. • e-business platforms share common data about users, items, and sellers. • Proposed a generalized recommender system framework using Blockchain technology. • Blockchain technology is adopted to share data between the e-business platforms. • Proposed framework will help alleviate cold-start, sparsity, and shilling attack problems.
Experience of Blended Learning in the Subject - Architecture of Computers

The proposed study discusses about the real-time experience of blended teaching for the students of the Faculty of Mathematics and Programming Technologies in the subject "Computer Architecture" based on the use of distance learning instrumental system DL.GSU.BY and special software complexes developed at the F. Skorina State University under the guidance of author. Special software packages include: HLCCAD - software system for designing, modeling and debugging the functional architectures of digital devices; Winter, a software system for developing and debugging programs in assembler and C for various microcontrollers; С-MPDL – firmware description language for computer architecture components; constructor of training and control flash tasks. The proposed method of conducting lectures and practical classes, as well as the organization of students' independent work and an assessment monitoring system that orients students toward a permanent increase in the assessment and quality of knowledge, skills, and abilities throughout the academic semester, are described.
Research on Road Vehicle Detection based on improved Yolov5s

Vehicle target detection is a very important technology in the field of autonomous driving, This paper presents a road vehicle detection method based on improved Yolov5s. First, SiLU activation function is used to replace the original activation function in Yolov5s backbone network, the number of parameters in the network structure is reduced and the problem of gradient disappearance is solved. CBAM (Convolutional Block Attention Module) attention mechanism module has been added to Backbone, enhanced feature extraction of small and medium target objects in the network. The results show that compared with the original Yolov5s model, after the improvement, the model size is reduced by 0. 4MB, map@0.5 is increased by 4.9% to 92.3%, precision increased 2.6% to 94.5%, the improved Yolov5s model is able to detect vehicles ahead more efficiently and accurately.
Distributed Implementation of Heat Kernel Smoothing for Graph Signal Denoising

In this paper, a graph signal denoising method based on heat kernel smoothing (HKS) and its distributed implementation structures are investigated. First, the graph signal denoising problem and HKS method are briefly described. Because HKS method needs to perform eigen-decomposition of graph Laplacian matrix for computing matrix exponential, it is only suitable for centralized implementation. To obtain a distributed or decentralized implementation of HKS method, three implementation methods are then presented in this study including product expansion method, Taylor series expansion method and numerical differentiation approximation method. Finally, irregular data collected from sensor network and social network are used to demonstrate the effectiveness of the graph signal denoising methods based on heat kernel smoothing.
MorIRNet: A Deep Image Retrieval Network using Morphological Feature and Residual Block

With the advent of deep networks, most computer vision tasks have been revolutionized. Image retrieval is no exception. The use of a stack of linear convolutional operations makes a convolutional neural network (CNN) a powerful tool in computer vision. Morphological operations are powerful nonlinear topological operators that can include morphological features and, therefore, enable a deep convolutional network to capture more informative features. Motivated by these advantages, this paper proposes a deep image retrieval technique using morphological operations based on the residual block. The proposed residual network is tested on various benchmark databases to validate the performance of the proposed model for image retrieval. The results show that our method outperforms other baseline approaches.
Cooperative Hunting of Spherical Multi-robots based on Improved Artificial Potential Field Method

With the development of robot technology, the research of multi-robot cooperative target search in the military field mainly focuses on avoiding obstacles while surrounding the target, but most of the object-oriented research is a single robot and a single obstacle. In view of the object - oriented simplicity, this paper uses robot cluster to avoid multiple obstacles. Artificial potential field algorithm is widely used in the field of robot obstacle avoidance because of its simple structure, small amount of computation and good real-time performance. But in actual use in the artificial potential field algorithm has certain defects, mainly includes the target unreachable problem and local minimum value problems, by improving the traditional artificial potential field algorithm, this study so as to improve the artificial potential field algorithm when use the deficiencies, in order to improve the artificial potential field algorithm when the mobile robot to round up the target of obstacle avoidance. Finally, the simulation results show that the improved artificial potential field method can solve the problems of local minimum and unreachable target, and has good dynamic obstacle avoidance ability. This method ensures the reliability of spherical robot group and improves the efficiency of multi-robot group.