Reviewerly/exHarmony huggingface dataset:
Decoupling features via global and local contexts in multi-object tracking

Multi-object tracking (MOT) system usually consists of two tasks, object detection and re-identification (ReID). Current MOT methods tend to join detection and ReID in a single network to enhance inference speed. Such one-shot models allow joint optimization of detection and Re-ID via a shared backbone, reducing computation cost. However, the different demands of features between the two tasks in one-shot systems lead to competition in the optimization procedure. The detection task needs the features of the instances with the same class to be similar, while the ReID task needs the features of different instances to be distinguishable. Existing methods address the contradiction by disentangling the features into detection-specific and ReID-specific features. But these methods neglect the discussion of semantic interpretation of disentangling modules. In this paper, we propose a feature decoupling module, Global and Local Context-based Decoupling Module (GLCD), to disentangle features extracted by the backbone into two task-specific features. By extracting global and local contexts, the two tasks can choose different contexts by learnable parameters to enforce each self. We conduct our decoupling module into SOTA one-shot MOT method and experiments show performance improvement.
Research of chaotic encryption model in digital watermarking algorithm

Digital watermarking is a new kind of protection technology of anti-counterfeiting and anti-theft that is born because it can not protect its copyright well in the process of multimedia information transmission. The characteristics of chaotic encryption model play an important role in the field of secure communication and image encryption, in this paper, according to its characteristics, the application of chaos model in digital watermarking algorithm is studied, it provides a research method for digital watermarking security rights protection.
Assessment algorithm of scene complexity based on X-CENet

In order to realize the rapid perception of complex scenes, the traditional scene complexity assessment algorithm has strong limitations in feature representation and scope of application, and it is difficult to deal with complex scenes. However, the existing deep network methods are lack of the consideration of the correlation between the underlying features of gray image and the complexity level, and the amount of parameters is too high to meet the needs of rapid response in practical applications. Based on the deep separable convolution module and residual connection structure, this paper designs a lightweight complexity assessment network X-CENet with stronger feature expression ability. A dense connection module which makes full use of multi-level features is introduced to improve the feature expression ability of the network for scene images. The underlying information such as image texture is particularly important for the assessment of complexity, so the feature cascade layer of the head and tail of the main modules is added to strengthen the utilization of the underlying feature information in the network. Experiments show that compared with other deep networks, this method can obtain higher assessment accuracy in the dimensions of image characteristics and detection performance with smaller parameters. Compared with the Inception V3 with similar parameter amount, this method improves the LCC index by 2.849% and the SRCC index by 3.338%.
Research on YOLOv4 model inference acceleration of real time object detection for smart construction site

In smart construction site, multi objects in real time monitoring streams are often needed to be detected at the same time. If YOLOv4 models are not accelerated, higher inference delay will be occurred, so that the purpose of real time detection can’t be achieved. The features of YOLOv4 model are firstly introduced in this paper, and then we discuss how to use YOLOv4-tiny-3l and TensorRT to accelerate the inference process of YOLOv4 model in detail. The experiments show that YOLOv4-tiny-3l models can be used to detection objects in multi real time streams smoothly, but the accuracy is pretty poor, so that the models can’t be used in practices. When adopting TensorRT toolkit to quantize YOLOv4 models with FP16 precision, the accelerated models can be used to detect objects in multi real time streams smoothly with a small loss of accuracy.
Category pattern mining based image retrieval

Image retrieval is to find out the similar semantic images to the query image, which is an important task in the field of image recognition. It is still an open challenging task due to the semantic gap of image understanding. The traditional image retrieval method is a simple retrieval between the query image and the database. However, only a query image contains weaker category information, so that the traditional image-based retrieval results are not satisfactory. In this paper, we propose a category pattern mining (CPM) strategy to extend an image (point) to an image category (plane). It means the semantic extension is performed from the individual query image to the whole image category. The proposed PTP (point to plane) method mined the category pattern of the query image and enriched the semantic information. The main contribution of the PTP framework is to improve the image retrieval from the traditional image-based retrieval into the new category-based retrieval. Experimental results and evaluations on two databases demonstrate that the proposed PTP method achieves an obvious superiority in the image retrieval tasks.
Single image dehazing based on linear transformation

In order to solve the problems of sky distortion, edge artifact and overall dark of the restored image, a single image dehazing algorithm based on linear transformation was proposed. Firstly, adaptive compression normalize the hazy image, the ratio of the minimum channel to the maximum channel of the image is defined as the anti-saturation of the image, and the model of medium transmission and anti-saturation of the image without haze is established. Then, combining the linear transformation model between haze-free image anti-saturation and hazy image anti-saturation, the initial medium transmission is calculated by linear transformation, the medium transmission obtained by different linear transformation rates are weighted fused through the maximum channel map of image with haze, and the final medium transmission is obtained by fast guided filtering. Finally, atmospheric light values are derived from the quad-tree sub-block search method, and then the restored image is obtained. Both subjective and objective results of experimental simulation show that the algorithm has obvious recovery of details, good effect on the hazy images of different scenes, and good restoration effect on sky region.
Fragmented cultural relics restoration based on point cloud data

Three-dimensional scanning technology, graphic image processing, virtual reality and other technologies are an important and inevitable way to protect cultural relics. With the passage of time, most of the cultural relics have been removed. Aiming at the main process of fragmenting cultural relics restoration based on point cloud processing, this paper summarizes the algorithms of 3d data denoising and simplification, model feature extraction, model classification and model assembling in virtual restoration of fragmenting cultural relics.
RDDP: Reliable Detection and Description of Interest Points

Recent works have paid much attention on learning repeatable maps for detection and description of interest points. However, there are always some components which is repeatable but not discriminative in reality. Training on the whole images with these components will lead to poor matching performance. Thus, we propose a self-supervised network with a new branch which can eliminate the negative influence of unreliable image components. We also design a loss function including a part calculated with Average Precision (AP) to make full use of the reliability branch. Evaluations on HPatches dataset show that the proposed method achieves competitive results compared with state-of-the-art methods.
A Slope-Adaptive Navigation Approach for Ground Mobile Robots

The 2-dimensional cost map has been widely used for the navigation of ground mobile robot. Although it is effective when the ground is flat, it becomes clumsy and ineffective when the ground has some slopes, where such slopes are often misjudged as the forbidden area by the cost map. For this reason, we propose a slope-adaptive navigation approach based on multilayer cost map in this paper. Instead of taking the point cloud of slope as obstacles, we actively construct a multi-layer cost map that takes slope information into the map in the stage of building environment map. A slope detection algorithm is developed to switch the cost map during the robot navigation. The slope is then considered as a passable road, only with extra cost. In the case that the slope leads the robot to a new floor, we adopt the Aruco code to switch the map information, such that the navigation can still keep working. Both simulation and real-world experimental results demonstrate the high effectiveness of our proposed approach.
Character animation and retargeting from video streams

Virtual character animation is widely used in 3D games and virtual reality. Traditional character animation can be achieved through key-frame animation or motion capture technology. These methods have limited applications due to expensive equipments or sophisticated operations. Aiming at a lower-cost solution for this issue, in this paper we propose a method of virtual character animations and retargeting from RGB video streams based on human pose reconstruction. We conduct extensive experiments with different videos and virtual characters, and the resulting character animation is well represented in the virtual scene. The proposed method greatly has reduced the production cost of character animation, which has potential applications in virtual reality.
Best Approximate Distribution-based Model for Helpful Vote of Customer Review Prediction

Product reviews are more and more important for potential customers to decide on their purchases in electronic commerce nowadays. The helpful vote is a critical indicator of how much impact the review has on other customers. Therefore, the prediction of helpful votes is an essential task. Linear and Tobit Regression are general methods of the prediction. Those methods share the same objective function and come from the initial assumption that the helpful votes on any dataset follow a normal distribution. However, the assumption is not usually confirmed, and the distribution of the helpful votes often follows other distributions. Consequently, the prediction results might not be fully appropriate. This paper proposes a model that follows the best approximate distribution of helpful votes to predict the number of helpful votes. On top of that, considering the elapsed time since reviews were written, we propose an adaptive window size sampling method to evaluate the model on review datasets sorted chronologically. To validate the proposed model, we conducted extensive experiments on real-world datasets. Experimental results illustrate the validity of the proposed model.
Transformer terminal feature recognition and positioning method based on binocular vision and image processing algorithm

Based on binocular vision and image processing algorithm, the recognition and location of high voltage side and low voltage side of transformer were completed for transformer automatic wiring positioning platform. On the basis of binocular vision calibration and 3D point cloud reconstruction, P3P algorithm was used to complete the positioning of transformer operation space coordinate system. Based on the image threshold segmentation algorithm, the position of transformer high voltage terminal, low voltage terminal, and transformer rotation angle were identified on the basis of 3d point cloud data. The target position information was provided for transformer automatic wiring device.
Evolutionary Approximation in Non-Local Means Image Filters

The non-local means image filter is a non-trivial denoising algorithm for color images utilizing floating-point arithmetic operations in its reference software implementation. In order to simplify this algorithm for an on-chip implementation, we investigate the impact of various number representations and approximate arithmetic operators on the quality of image filtering. We employ Cartesian Genetic Programming (CGP) to evolve approximate implementations of a 20-bit signed multiplier which is then applied in the image filter instead of the conventional 32-bit floating-point multiplier. In addition to using several techniques that reduce the huge design cost, we propose a new mutation operator for CGP to improve the search quality and obtain better approximate multipliers than with CGP utilizing the standard mutation operator. Image filters utilizing evolved approximate multipliers can save 35% in power consumption of multiplication operations for a negligible drop in the image filtering quality.
A Robust Psychologically-Oriented Emotion Recognition Method Using Transfer Learning

Artificial intelligence and machine learning have become increasingly popular in recent years, applying them to different applications are also vital in prediction. As facial emotion is one of the important expressions in our daily life, emotion recognition helps in conversation and understanding more effectively. Although emotion recognition may seem to be an easy task to humans, it is challenging for the machine. Currently, many facial emotion detection technologies are based on the features extracted from the whole face and do further analysis to detect emotions. However, there are some limitations to this method. For example, it may require considerable time in the training and prediction with the high-resolution image input. It is a major issue if we apply the model to a mobile device that gives high-resolution photos. To deal with the problem, reducing the input size while remaining the important information would be a solution. To achieve it, the dimension reduction method may be an option. On the other hand, there is a lot of psychological research on how humans recognize emotion by facial expression. Improvements can be made if we can make good use of the results for emotion recognition tasks. Thus, in this paper, we proposed a psychological-oriented method for reducing the input size on the emotion detection task which is based on the eyes region. It is found that the proposed method gives compatible performance compared to the face based model. With the proposed method, the time used for training and prediction is significantly reduced.
Research on binocular stereo vision based on interactive channel and its application in power intelligent emergency repair

Recently, the whole society has had higher and higher requirements for the stable operation of the power system, and the safety of power production is directly related to people's daily life order and the development of the national economy. How to build a power system security framework to repair problems in the power system in a timely manner, and how to improve the accuracy of remote live work while improving work efficiency have become the main needs of the development of the power industry. In the practical application scenario of power repair, this paper obtains the target coordinates through the line laser scanning binocular stereo vision technology of the interactive channel and completes the preparations for the binocular stereo vision including image preprocessing, camera calibration, and stereo correction. The stereo vision algorithm is used to improve the efficiency of electric power intelligent emergency repair operations, and the advantages of binocular stereo vision algorithm in the application of electric power emergency repair are illustrated by comparing the accuracy of different algorithms through experiments.
Variational Learning of Convolutional Neural Networks with Stochastic Deformable Kernels

Due to fixed convolution kernel structure, it is intractable for Convolutional Neural Networks (CNN) to adequately deal with intact instance geometric transformations, which may paralyze the CNN’s paradigmatic feature extraction. Inspired by Deformable Convolutional Networks, in this work, we introduce stochastic deformatble kernels, where kernel shape is assumed to be random vaiable. Stochastic deformable kernels may extract non-local alienated structural features, which slightly condones geometric transformations. Variational learning of the model based on two approximate posteriors is derived. And experiments over transformed MNIST dataset demonstrate the validity of this approach.
Exploration of Mandibular Inputs for Human-Machine Interfaces

The direct connection of the jaw to the brain allows it to retain its motor and sensory capabilities even after severe spinal cord injuries. As such, it can be an accessible means of providing inputs for people with paralysis to manipulate their environment. This paper explores the potential for using the jaw, specifically the mandible, as an alternative input to human-machine interface systems. Two tests were developed to test the mandible’s ability to respond to visual stimuli. First, a visual response time test to measure the precision and accuracy of user input through a mandible-actuated button. Second, a choice response test to observe coordination between the mandible and a finger.Study results show that the mean response time of mandible inputs is 8.3% slower than the corresponding mean response time of performing the same task with a thumb. The delay in response after making a decision is statistically insignificant between the mandible-and finger-actuated inputs with the mandible being 2.67% slower.Based on these results, the increase in response time while using the mandibular input is minimal for new users. Coordination is feasible in tasks involving both the mandible and thumb. Extensive training with a made-to-fit device has the potential to enable a visual response time equivalent to the fingers in more complex tasks. The mandible is a viable option for accessible HMI for discreet inputs. Further testing into continuous input is needed to explore the mandible’s potential as an input for body augments.
Cost-optimized Task Scheduling with Improved Deep Q-Learning in Green Data Centers

With the rapid development of cloud computing technologies, more and more individual users and enterprises choose to deploy their key applications in green data centers (GDCs), and the scale of GDCs is increasing rapidly. To ensure service quality and maximize the revenue, cloud service providers in GDCs need to reasonably and efficiently allocate computing resources and schedule tasks of users. Traditional heuristic algorithms face challenges of uncertainty and complexity in GDCs for scheduling tasks. To solve them, this work establishes an improved resource allocation and task scheduling method based on deep reinforcement learning. It considers the dependency among different tasks, and builds a workload model based on the real-life data in Google cluster trace. In addition, a deep reinforcement learning-based scheduling model is proposed to reasonably allocate and schedule resources (CPU and memory) in GDCs. Based on two models, an Improved Deep Q-learning Network (IDQN) is proposed to autonomously learn the changing environment of GDCs, and yield the optimal strategy for resource allocation and task scheduling. Real-life data-based experiments demonstrate that IDQN achieves lower task rejection rates and energy cost than several typical task scheduling methods.
MLPT: Multilayer Perceptron based Tracking

The global receptive field plays a critical role in visual object tracking. In most popular tracking paradigms, we find that the local receptive field introduced by the convolutional neural network prevents the tracker from focusing on the long-range dependency. Although the Vision transformer brings the global receptive field in downstream tasks, its computational burden remains unaffordable. In this paper, we present a simple yet effective Multilayer Perceptron-based Tracking (MLPT), including the global receptive field. The MLPT contains three Components: Feature Correlation (FC) module, Global Information Encoder (GIE) module and Corner Head(CH). Firstly, the FC module is proposed to effectively converge the template and search region features for generating delicate features. Secondly, the GIE is designed to integrate the channel and spatial-information dealed with channel-encoding the token-encoding, separately. Specially, the same kernel is utilized for token-encoding in all channels so that our model has the global receptive field. Then, the CH is applied to establish a simple flexible way via computing the box corners coordinates for tracking. Finally, the MLPT, to our knowledge, is the first baseline of MLP-based architecture for object tracking. Extensive experiments are conducted on four challenging datasets, including GOT-10K, LaSOT, UAV123, and TrackingNet. The results show that the proposed method achieves state-of-the-art performance.
Deep Knowledge Reasoning guided Disease Prediction

Disease prediction, which aims to predict possible future diseases for patients, is a fundamental research problem in medical informatics. Many studies have proposed the introduction of external knowledge to enhance deep learning models which have achieved good results, but as most of these studies only consider using single-hop relationship information from knowledge graphs or simply introduce partial knowledge graphs, fail to effectively mine knowledge paths to understand the relationships between diseases. To this end, we propose a new approach, which uses existing medical knowledge graphs for multi-hop reasoning to guide the self-attention based transformer model for disease prediction. Specifically, we design a reinforcement learning algorithm to perform path reasoning in the knowledge graph to obtain explicit disease progression paths and fusion them with the original Electronic Health Records (EHR) data. After embedding we capture the implicit relationships between the deep knowledge and the original EHR information through several transformer encoders based on the self-attention mechanism to better extract features. At the same time, multi-hop knowledge is more interpretable than single-hop knowledge in terms of disease prediction. Experimental results on the real-world medical dataset MIMIC-III show the superiority of the proposed approach compared to a series of state-of-the-art baselines. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> The code has been uploaded to https://github.com/15536385781/DKRDP
PL-TD3: A Dynamic Path Planning Algorithm of Mobile Robot

In this paper, Prioritized Experience Replay (PER) strategy and Long Short Term Memory (LSTM) neural network are introduced to the path planning process of mobile robots, which solves the problems of slow convergence and inaccurate perception of dynamic obstacles with the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. We dubbed this new method as PL-TD3. Firstly, we improve the convergence speed of the algorithm by introducing PER strategy. Secondly, we use LSTM neural network to achieve the improvement of the algorithm for dynamic obstacle perception. In order to verify the method of this paper, we design static environment, dynamic environment and adaptability to dynamic experiments to compare and analyze the methods before and after improvement. The experimental results show that PL-TD3 outperforms TD3 in terms of execution time and execution path length in all environments.