Reviewerly/exHarmony huggingface dataset:
Sort Planning of Multi-Fingered Dexterous Hands Using Deep Learning

In real application scenarios, objects are usually in complex unstructured environments, so object sorting operations based on machine vision are essential. To solve it, we propose a sort planning method for multi-fingered dexterous hand to handle multiple target objects by using the two-stage object detection network and the sort order decision algorithm. Firstly, the two-stage object detection network based on RGB images is designed, in which the first-stage network is used to search candidate regions where objects may appear while the second-stage network evaluates each region to detect the positions of all target objects in the RGB image. Secondly, the global depth information of the objects and the relative positions between them are obtained from the depth image. Thirdly, the sorting order of the target objects is determined according to the human sorting behaviors, and the grasp planning method is applied to the segmented depth image which contains the chosen target object. Finally, the above steps are repeated until all sorting operations for all visible target objects are finished, and the performance of the sort planning method is verified on the actual robot platform with multi-fingered dexterous hand and the success rate of sorting is up to 82%.
Fourier ViT: A Multi-scale Vision Transformer with Fourier Transform for Histopathological Image Classification

Histopathology examination is regarded as the gold standard for cancer diagnosis, and accurate classification of medical images is essential. It is worth noting that histopathological images are highly unstructured in relation to natural images. Thus, some key variations will be ignored by directly applying deep learning approaches of classifying natural images to categorize medical images, which leads to inaccuracies. While Fourier transform can extract the unremarkable features in the image. In addition, Vision Transformer (ViT) has proven its strong attention to the relation between the entire and partial images. The synergy between Fourier transform and ViT may greatly benefit the classification of unstructured histopathological images, which is neglected by existing efforts. To realize this vision, this paper proposed Fourier ViT, a universal attention-to-details architecture for fine-grained representation and classification. Furthermore, the paper implemented a mixed attention algorithm to enhance the model’s attention. Extensive experiments indicate the superior performance of our method that surpasses the classic CNN-based models and Transformer-based models.
A correlation filter tracking method with the addition of new temporal regular term suppression

In this paper, we proposed an algorithm named ANRCF which add a new temporal regular term suppression based on the ARCF. The algorithm adjusts the suppression of the regular term from the perspective of continuity and frame by introducing global variables in the objective function to cope with the problems of background complexity and deformation during tracking. In this paper, the proposed algorithm has been exhaustively experimented and tested under OTB100, DTB70, TColor-128 datasets, and relevant filtering classical algorithms such as ECO, and high precision algorithms under UAV proposed in recent years are selected for comparison. The experimental results show that: ours success rate reaches 76.8% and the accuracy reaches 82.1% on the OTB100 dataset, and the overall performance is significantly better than that of ARCF algorithm, not only in terms of success rate and accuracy, but also in terms of success rate, which is higher than that of ECO algorithm;Under the UAV sequence DTB70, the precision of our algorithm is higher than the mainstream algorithms LADCF, ARCF, ECO, etc.; on the TColor-128 dataset, our algorithm is higher than the excellent ARCF, AutoTrack, etc. It is 2.5%, 3.1% and 8.0% respectively higher than ARCF, AutoTrack and BACF, and 2.7%, 3.8% and 12.8% respectively higher than AutoTrack, ARCF and BACF, in terms of precision. In addition, experiment shows that the performance improvement of the our algorithm is more significant when dealing with background complexity and deformation.
A Novel System for Robot Trajectory Programming based on VR Tracking System

This paper proposes an efficiently robot trajectory programming system that enables operators to program a robot conveniently. Based on VR tracking devices, a pen-shaped teaching tool is designed to track its position and orientation for demonstration, so the system could reach the advantages of low cost and high precision. A valid experiment is set by adapting UR5 system to assess the efficiency of the proposed system, the maximum deviation between real and demonstration trajectory is 1.13 mm. As proved by the experiments, the proposed robot trajectory programming approach is more efficient than UR kinesthetic teaching system.
Inductive Representation Learning of Multiple ICD Codes for Healthcare

The International Classification of Disease (ICD) coding scheme authorized by the World Health Organization, is a standard used to classify medical diagnosis in health insurance claims. One of the key challenges in adopting machine learning techniques for health insurance claims processing lies in the sparse and complex nature of ICD codes. There are over 69,000 unique diagnosis codes in the 10th version of the ICD coding scheme where multiple codes are used to represent complex medical diagnosis. In this work, we address the challenge of embedding both single and multiple ICD codes into a vector space. This representation addresses the sparse representation of ICD codes by providing a more compact representation that captures their diagnosis relationship. Our proposed inductive multi-code representation introduces a novel application of inductive node embedding by representing multiple ICD codes as a multi-code node. This enables both single and multiple ICD codes to be represented in the same vector space. For multi-code relevance, our proposed approached surpassed the benchmark node2vec implementation by 34%, highlighting its effectiveness.
An Observation Based Method for Human Robot Writing Skill Transfer

This paper proposes a novel method of Chinese character stroke extraction and a framework for human robot skill transfer through vision-based observation. By analyzing the structure of Chinese characters, a direction vector update rule and a pixel finding rule were proposed to find the basic strokes. Then we designed a basic stroke connection algorithm to achieve stroke extraction. Afterward, to adapt to human interference in real-time, we adopt dynamical movement primitives (DMPs) to model writing skills. Finally, the adaptive capability of the method was verified by experiments in which the robot writes Chinese characters on a randomly moved writing board.
Safe Motion Planning for Goal Reaching of Multiple Autonomous Surface Vehicles Based on High-order Control Barrier Functions

Real-time motion planning is critical to real-world applications of autonomous surface vehicles (ASVs) moving in a complex sea environment. This paper presents an efficient motion planning method to generate collision-free trajectories for a fleet of ASVs subject to static obstacles. The safe motion planning algorithm is proposed based on high-order control barrier functions (HOCBFs), which are able to address the obstacle and inter-vehicle collision avoidance. The optimal collision-free trajectories are obtained by solving a quadratic program (QP) problem. The proposed safe motion planning algorithm is able to achieve the following two objectives: generating feasible yaw rates to guide each ASV toward its goal, and handling static obstacles and model constraints, simultaneously. Simulation results are given to substantiate the efficacy of the proposed safety-critical motion planning method for multiple ASVs subject to kinematic model constraints and collision avoidance constraints.
Multi-level Variational Auto Encoder for Unsupervised Abnormal Data Recognition

High-dimensional and high-complexity unsupervised abnormal data recognition with neural networks remains an open research question. We propose a multi-level Variational Auto Encoder (VAE) structure in this work. First, based on traditional VAE, we deconstruct the original data using convolution kernels of different specifications to establish multiple feature latent spaces, thereby reducing the excessive data aggregation in the latent space. Then we use an equal-window convolution kernel to fuse feature blocks of different sizes, thereby transforming local features into globally identifiable features. Moreover, the model structure shortens the distance between the sub-feature block and the output end-point, reducing the training inefficiency caused by the vanishing gradient.
Sign language image dataset with a hand pose type attribute

We present hand image datasets for training convolutional neural networks (CNNs) for sign language recognition and translation. Technologies to translate sign language into another language will facilitate the communication between deaf and hearing people. Recently, deep learning has been considered to be promising for analyzing sign language videos and estimating a signer's message. If a CNN is trained to discriminate hand poses, it would work to estimate sign language. For the training of CNN, it is required to have hand pose dataset. We constructed hand pose datasets by automatically classifying images based on two types of features: 1) the shape of five fingers as bent/stretched and aligned/separated, and 2) palm orientation and rotation. We devised a method to calculate feature parameters for the finger shape and palm orientation and rotation. Hand images were clustered using these feature parameters. We classified more than 200,000 images from sign language videos showing nine different signers expressing more than 1000 different signs. A CNN trained on our dataset successfully discriminated hand poses. The trained CNN further worked to estimate sign language messages. The results show that, compared with a baseline network, the estimation performance was improved.
Wireless Channel Recognition by CNN

This paper proposes a wireless channel recognition algorithm based on convolutional neural network (CNN) combined with multi-domain feature extraction. Channel State Information (CSI) of signals usually involves the information of the environment in which the receiver is located. In this paper, the Digital Terrestrial Multimedia Broadcast (DTMB) signal is used to extract the CSI features. The cross-correlation between PN420 sequences in frame header and the local PN sequences in the receiver is employed as the time-domain features. Moreover, the time-domain features are transferred into time-frequency images by Wigner-Ville Distribution (WVD). Then, CNN is adopted to recognize the wireless channels. The performances of the combined WVD time-frequency domain and time-domain feature extraction are compared with the WVD time-frequency domain feature extraction. Simulation results show that the proposed channel recognition method based on time-frequency domain has good performance; and its accuracy can reach more than 96% when SNR is −2dB; the performances of the jointed multi-domain feature extraction outperform those of the time-frequency domain feature extraction method. The accuracy can reach more than 99% when SNR is −10dB.
Trustworthiness Measurement for Multimedia Domain Knowledge Graph

With the upsurge of knowledge graph research, the multimedia field supports the upper multimedia services by constructing the domain knowledge graph to improve the user experience quality of multimedia services. However, the quality of knowledge graph will have a great impact on the performance of the upper multimedia service supported by it. The existing quantitative methods of knowledge graph quality have some problems, such as incomplete information utilization and so on. Therefore, this paper proposes a trustworthiness measurement method of joint entity type embedding and transformer encoder, which is quantified respectively from the local level and the global level, makes full use of the rich semantic information in the knowledge graph, and comprehensively evaluates the quality of the knowledge graph. The experimental results show that the method proposed in this paper can solve the problem that the traditional methods can not detect the matching error of entity and entity type under the condition that the error detection effect of traditional triples is basically unchanged.
Flexible Tiles in Adaptive Viewing Window: Enabling Bandwidth-Efficient and Quality-Oriented 360°VR Video Streaming

360° VR video offering a unique interactive experience has become a dominant streaming form for immersive multimedia applications. At the same time, remote transmission of these applications is not feasible under conventional network and rendering infrastructure. The existing viewport-based solutions employ fixed tiling patterns to stream quality variable views intending to reduce the amount of data transmission. However, they still could end up with unsatisfactory visual boundaries and feeble bandwidth utilization. For instance, employing larger resolution tiles can include redundant pixels that contribute to the waste of bandwidth resources, and smaller resolution tiles can introduce several visual distortions, e.g., flickering, floating, and blurring at the edges of the tiles. This paper proposes FlexTile, a bandwidth-efficient and quality-oriented solution that utilizes flexible tiles in an adaptive viewing window during 360° VR video streaming. FlexTile overcomes the limitations of existing works by implementing three levels of adaptation: (i) adaptive tiles selection, (ii) adaptive bitrate selection, and (iii) adaptive tiling pattern selection. FlexTile interactively selects best-fit viewing tiles for each adaptive viewing window considering a region-based visual quality metric to facilitate visual comforts and bandwidth utilization. The extensive performance evaluations show that our flexible tiling-based approach significantly improves the viewing comfort levels measured in terms of visual quality and visual smoothness while improving bandwidth utilization under real-world 4G bandwidth conditions.
Image Enhancement for Rotation Sushi Plates Detection in Smart Multi-Platform Billing System

Conveyor belt sushi restaurants have become worldwide popular. The customers are drawn to the low prices and accessibility of the rotation sushi to save their dining time. These small dishes are typically served on different kinds of plates depending on price. Counting the number of each type of plates can then be computed for bill payment. An AI-based approach for sushi plate detection to overcome the time-consuming plate count is introduced in this work. A smart billing system is developed on multi-platform mobile application. The user can take a photo of stacks of consumed dishes as the input image to the YOLOv4 object detector for classifying the different colored plates. This paper focuses on parts of image enhancement as the system can work on iOS and Android systems. The techniques of color transfer with LAB color space and CLAHE are applied for image quality improvement. The experimental results reported the achievement of higher confidence and the successful detection of the objects from out of color space images.
Multi-scale Context Enhancement Network for Object Detection

Convolutional neural networks-based object detection has made significant progress in the last decade. However, most detectors adopt square kernels in convolution and pooling operations that ignores long range correlation between pixels. In this work, we propose a new object detection method which uses rectangular pooling kernels to capture image level long-range relationship. Besides image level context, dilated convolutions with different dilation rates are also adopted to collect multi-scale contextual information. The proposed rectangular pooling and dilated convolutions are combined into a context enhancement module which can be integrated into existing models to improve detection accuracy. Experimental results on the MS COCO benchmark show that applying the proposed method on anchor-free detector (e.g., FCOS) and anchor-based detector (e.g., RetinaNet) obtained 1.9% AP and 1.2% AP performance improvement respectively.
Multi-frame Video Super-resolution Based on Efficient and Parallel Network

Multi-frame video super-resolution(VSR) aims to restore a high-resolution video from both its corresponding low-resolution frame and multiple neighboring frames, in order to make full use of the inter-frame information. However, vast computation complexity hinders the inference speed of video super-resolution. In order to increase the inference speed while ensuring the accuracy of the model, we proposed an efficient and parallel multi-frame VSR network, termed EPVSR. The proposed EPVSR is based on spatio-temporal adversarial learning to achieve temporal consistency and uses TecoGAN as the baseline model. By adding an improved non-deep network, which is composed of parallel subnetworks with multi-resolution streams, these streams are fused together at regular intervals to exchange information. we reduced the number of parameters and make the model lighter. Besides, we implement structural re-parameterization network acceleration technique to optimize the inference process of EPVSR network. Finally, our EPVSR achieves the real-time processing capacity of 4K@36.45FPS. compared with TecoGAN, we achieve 9.75 × performance speedups, but the effect is not reduced. the PSNR of EGVSR are increased by 3.36%. The experimental results show that the non-deep network can effectively speed up the model inference, and the proposed EPVSR has a good super-resolution effect.
Evolutionary Multitasking with Scale-Adaptive Subspace for Large-Scale Optimization

Large-scale optimization is an important research topic in the field of evolutionary computation which has been widely applied in many scientific and engineering fields. In large-scale optimization, many research studies on simplifying the search space for large-scale optimization problems (LSOPs) have been proposed, such as cooperative co-evolution, and dimension reduction. However, the validity of simplified solution spaces with different scales in the evolutionary process is not fully explored to guide the search direction. Therefore, this paper puts forward an evolutionary multitasking method with scale-adaptive subspace (EMT-SAS) for LSOPs, which uses a set of subspaces with different accuracy scales to cooperatively assist original problem search. The basic idea is to adopt the low-accuracy scale subspace to fast locate promising regions and utilize the high-accuracy scale subspace to refine the solution accuracy. Moreover, an archive management strategy is also proposed to select training samples, which can further enhance the diversity of the algorithm. The performance of the proposed EMT-SAS is evaluated on the CEC2013 large-scale benchmark problems. The experimental results have demonstrated the superiority of EMT-SAS when compared with other advanced algorithms.
A Method to Evaluate the Health of OSS Projects Based on the PSR Model and Combination Weighting

With the rapid development of open-source software (OSS), an increasing number of enterprises and developers have gradually joined OSS projects. OSS development has become a growing trend. For users, developers, and managers of OSS projects, understanding the health status of a project is conducive to making correct decisions and preventing the risks caused by a decline in the project's health. Therefore, an effective method is needed to evaluate the health of OSS projects. Based on the pressure state response (PSR) model, this paper propose a health evaluation model for OSS projects, defines t he healthof OSS projects from the three levels of pressure, state and response, and selects scientific, reasonable a nd representative indicators. The weight of each index is determined by the combination weighting method, which combines AHP and the entropy method. The health of the project is quantitatively analyzed, and the overall health score of a project is calculated. In addition, to verify the effectiveness of the proposed evaluation model, we conduct an empirical experiment based on real project data on the GitHub open-source platform. The experiment results and analysis show that our evaluation model can effectively reflect the health status of OSS projects.
Semi-supervised Video Deraining Based on Enhanced Spatio- Temporal Interaction Network

Video rain removal is an important task in computer vision. Although the video rain removal method based on deep learning has achieved great success in recent years, there are still two main challenges: how to use the large amount of information between consecutive frames to extract cross-space and the powerful spatio-temporal characteristics of the time domain, and how to restore the use of high-speed methods to produce high-quality rain-removing videos. In this article, we propose a new end-to-end video rain removal framework called Enhanced Spatio-temporal Interaction Network (ESTINet), which greatly improves the current state-of-the-art video rain removal quality and speed. The enhanced spatio-temporal interaction network uses the advantages of deep residual networks and convolutional long and short-term memory to capture the spatial characteristics and temporal correlation between consecutive frames at the cost of few computing resources. In addition, Different prior formats are designed for labeled synthetic data and unlabeled real data, and a semi-supervised learning mechanism is designed through different prior formats.
Locally Linear Embedding Based on Faure Sequence Initialized Killer Whale Algorithm

In the field of artificial intelligence (AI), and to be specific, in the field of manifold learning, local linear embedding (LLE) algorithm, which is a nonlinear data dimensionality reduction method, has become an indispensable and important tool. The strength of LLE algorithm is that after dimensionality reduction, it can make the points in high-dimensional space isomorphic with the points in low-dimensional space after projection, and the algorithm has low complexity and fast operation speed. However, LLE also has its own drawback, it depends on gradient descent (GD) method or stochastic gradient descent (SGD) method in solving the optimization of its objective function, resulting in falling into the trap of local minimum. In the past, there was little discussion about the global optimum solution issue for LLE, therefore, in this paper, a Faure sequence initialized killer whale algorithm (FSKWA) has been proposed to effectively overcome the drawback in finding the global optimum solution of the objective function for LLE. And via numerical experiment, the superiority of the method proposed has been verified.