Reviewerly/exHarmony huggingface dataset:
Bayesian Personalized Ranking based on Knowledge Graph

Collaborative filtering algorithms have serious data sparsity and cold start problems as the amount of data increases and the movie dataset keeps growing.To solve the above problems, this paper proposes to combine the knowledge graph with Matrix factorization algorithm.Through the user's historical interests, mining the user's similar interests on the knowledge graph, to form the candidate items, useing eventually to predict users' interests, and finally using Bayesian personalized recommendation to predict the user's rating of the candidate items to achieve top-K recommendation.Through experiments, it is demonstrated that the algorithm proposed in this paper significantly improves the recommendation effect of matrix decomposition model. With its AUC=0.9348 and ACC=0.8474 on the movie dataset, the experimental data show that the algorithm can improve the recommendation effect more effectively.
Activity classification of the elderly based on lightweight convolutional neural networks

Accurate implementation of action classification for the elderly on lightweight convolutional neural networks benefits resource-limited embedded and mobile devices in the healthcare industry. The study proposes a lightweight convolutional neural network model called mD-MobileNet. The micro-Doppler feature spectrograms of 106 elderly people were studied as a dataset. Transfer learning methods were used to train the proposed model, and three lightweight convolutional neural networks (MobileNetV3-Small, ShuffleNetV2, and EfficientNet-B0) were compared using the same training method. All of these models were able to correctly classify various actions. By comparison, mD-MobileNet gave the best classification results. mD-MobileNet’s Top-1 Accuracy reached 96.1% while Marco F1 was 96.30. By comparing the results with Grad-CAM’s visualization and analyzing them in conjunction with its network structure features, it was determined that mD-MobileNet has the best local perception with the least number of model parameters and the highest accuracy rate compared to other models.
Drivable Area Segmentation in Unstructured Roads for Autonomous Vehicles based on Multi-sensor Fusion

Drivable area segmentation is vital for autonomous vehicle driving safety, especially on unstructured roads. Mainstream drivable area algorithms are suited for structured environments, such as urban roads. However, these algorithms perform poorly in unstructured environments. This paper proposes a drivable area segmentation algorithm based on multi-sensor late-fusion for unstructured environments. The algorithm uses the visual segmentation results to correct the light detection and ranging (LiDAR) segmentation results, which can effectively solve those environments with unapparent boundary height differences. Desert experiments show that our algorithm achieves 96.02 on Intersection over Union (IoU), which is 36.75 and 38.31 higher than the LiDAR-based and the Vision-based algorithm, respectively.
Multi-Scale Channel Attention for Chinese Scene Text Recognition

Scene text recognition have proven to be highly effective in solving various computer vision tasks. Recently, numerous recognition algorithms based on the encoder-decoder framework have been proposed for handling scene texts with perspective distortion and curve shape. Nevertheless, most of these methods only consider single-scale features while not taking multi-scale features into account. Meanwhile, the existing text recognition methods are mainly used for English texts, whereas ignoring Chinese texts' pivotal role. In this paper, we proposed an end-to-end method to integrate multi-scale features for Chinese scene text recognition (CSTR). Specifically, we adopted and customized the Dense Atrous Spatial Pyramid Pooling (DenseASPP) to our backbone network to capture multi-scale features of the input image while simultaneously extending the receptive fields. Moreover, we added Squeeze-and-Excitation Networks (SE) to capture attentional features with global information to improve the performance of CSTR further. The experimental results of the Chinese scene text datasets demonstrate that the proposed method can efficiently mitigate the impacts of the loss of contextual information caused by the text scale varying and outperforms the state-of-the-art approaches.
Semantic Maximum Relevance and Modal Alignment for Cross-Modal Retrieval

With the increasing abundance of multimedia data resources, researches on mining the relationship between different modalities to achieve refined cross-modal retrieval are gradually emerging. In this paper, we propose a novel Semantic Maximum Relevance and Modal Alignment (SMR-MA) for Cross-Modal Retrieval, which utilizes the pre-trained model with abundant image text information to extract the features of each image text, and further promotes the modal information interaction between the same semantic categories through the modal alignment module and the multi-layer perceptron with shared weights. In addition, multi-modal embedding is distributed to the normalized hypersphere, and angular edge penalty is applied between feature embedding and weight in angular space to maximize the classification boundary, thus increasing both intra-class similarity and inter-class difference. Comprehensive analysis experiments on three benchmark datasets demonstrate that the proposed method has superior performance in cross-modal retrieval tasks and is significantly superior to the state-of-the-art cross-modal retrieval methods.
From StarCraft II to Military Combat

This paper focuses on the cutting edge technology of the Artificial Intelligence, introducing the Decision-Centric Warfare in detail and making it as basis to design the technology framework of auxiliary decision system in marine warfare. On this basis, an application technology framework of intelligent auxiliary decision in marine combat is constructed, which is supported by knowledge driven, data-driven computing model, network training method technology, situation cognition and intelligent decision technology and combat information distributed computing technology. The framework is based on the current successful application examples of artificial intelligence in Deep Mind, Open AI, DARPA and Tencent teams, which are proved practical and operable.
Multi-Channel Text Classification Model Based on ERNIE

Aiming at the large amount of news and review text data, sparse features, and the inability of traditional text feature representation to dynamically obtain grammatical structure, semantic information, and multi-dimensional rich feature representation of entity phrases. This paper proposes to obtain more generalized knowledge semantic feature information such as rich context phrases, entity words and so on by integrating knowledge enhanced semantic representation (Enhanced Representation Through Knowledge Integration, ERNIE). The pre-trained language model ERNIE hides words and entities by random Semantic unit prediction context realizes word vector language representation, and the output vector representation of ERNIE is input to BiLSTM, Attention mechanism and DPCNN network model to generate high-order text feature vectors, and each channel vector is processed by BatchNormalization and ReLU activation functions respectively.Thus, the semantic description information of the multi-channel word vector is fused. The model proposed in this paper can not only improve the training speed and prevent overfitting, but also enhance the feature information such as semantics and grammatical structure, thereby improving the text classification effect. By comparing the two datasets with other improved ERNIE models in terms of accuracy, precision, recall, and F1 value, the experimental results show that the model proposed in this paper can obtain multi-dimensional rich semantic grammatical structure features for text classification, and then improve the Text classification effect.
Few-shot Object Detection via Refining Eigenspace

Few-shot object detection (FSOD) aims to retain the performance of detector when only given scarce annotated instances. We reckon that its difficulty lies in the fact that the scare positive samples restrict the accurate construction of the eigenspace of involved categories. In this paper, we proposed a novel FSOD detector based on refining the eigenspace, which is implemented through a pure positive augmentation, a full feature mining module and a modified loss function. The pure positive augmentation expands the quantity and enriches the scale distribution of positive samples, inhibiting the expansion of negative samples. The full feature mining module enables the model to mining more information about objects. The modified loss function drives prediction results closer to ground truths. We apply these two improvements to YOLOv4, the representative of one-stage detector, which is termed YOLOv4-FS. On PASCAL VOC and MS COCO datasets, our YOLOv4-FS achieves competitive performance compared with existing progressive detectors.
Speech Recognition Method based on CTC Multilayer Loss

Due to the conditional independent assumption of a CTC model, a language model is usually added to improve its speech recognition performance. However, adding a language model will increase the complexity and computation cost. Therefore, we proposed a simple and effective speech recognition method based on CTC multilayer loss. Unlike the traditional CTC model which only optimizes the CTC loss of the last layer, in this method, the CTC multilayer loss, which guides the training of the model, is obtained by weighted summation of the CTC losses of different layers. Through optimizing the losses of different layers, the information of different layers of the CTC model can be taken into account, and the information obtained is more comprehensive, so that the model obtained has better recognition performance. With a small amount of code modification, this CTC multilayer loss method can well regulate the training of CTC and improve the performance of speech recognition. Since this method only changes the loss function of the CTC model and does not change the structure of the CTC model and its testing process, the training stage is simple and the testing stage has no extra memory cost and computation cost. We evaluated the method on Aishell-1 dataset using WeNet as the baseline, and it was able to reduce the character error rate (CER) by 7.5% and improve speech recognition performance without adding a language model.
Recognition of Human Walking Motion Using a Wearable Camera

In recent years, the computer vision technology has been attracting more attention than ever and being applied in a wide range of fields. Among them, the technology on automatic recognition of human motion is particularly important, since it leads to automatic detection of suspicious persons and automatic monitoring of elderly people. Therefore, the research on human motion recognition using computer vision techniques has been actively conducted in Japan and overseas. However, most of the conventional researches on human motion recognition employs a video of a human motion taken using an external fixed camera. There is no research on human motion recognition using a video of a surrounding scenery provided from a wearable camera. This paper proposes a method of recognizing a human motion by estimating the posture change of a wearable camera attached to a walking human from the motion of a scenery in the video provided from the wearable camera and by analyzing a human trunk change obtained from the posture change of the camera. In the method, AKAZE is applied to the images to detect feature points and to find their correspondence. The 5-point algorithm is used to estimate the Epipolar geometry constraint and an essential matrix which provides a camera relative motion. The change of the camera relative motion is then used to analyze the shape of a human trunk. The analyzed results, i.e., walking motion features, are finally fed into a SVM to identify the motion. In the experiment, five types of walking motions are captured by a wearable camera from five subjects. The accuracy on human motion recognition was 80%. More precise feature points extraction, more exact estimation of motions, and considering variety of human walking motions are needed to improve the proposed technique.
ResAsapp: An Effective Convolution to Distinguish Adjacent Pixels For Scene Text Detection

The segmentation-based approach is an essential direction of scene text detection, and it can detect arbitrary or curved text, which has attracted the increasing attention of many researchers. However, extensive research has shown that the segmentation-based method will be disturbed by adjoining pixels and cannot effectively identify the text boundaries. To tackle this problem, we proposed a ResAsapp Conv based on the PSE algorithm. This convolution structure can provide different scale visual fields about the object and make it effectively recognize the boundary of texts. The method's effectiveness is validated on three benchmark datasets, CTW1500, Total-Text, and ICDAR2015 datasets. In particular, on the CTW1500 dataset, a dataset full of long curve text in all kinds of scenes, which is hard to distinguish, our network achieves an F-measure of 81.2%.
RGFGM-LXMERT-An Improve Architecture Based On LXMERT

LXMERT (Learning Cross-Modality Encoder Representations from Transformers) is a two-stream cross-modality pre-trained model that performs well in different downstream tasks which contain two visual question answering datasets and a challenging visual-reasoning task (i.e., VQA, GQA, and NLVR). But the large-scale model still has a lot of room for progress. That is, the model accuracy is very low, the generalization ability is weak, and it is easy to be attacked by adversarial attacks. Furthermore, training the LXMERT model takes a lot of time and money, so there is an urgent need to improve. Thus, I try to improve the training speed, generalization ability, and accuracy of the model by enhancing both the training method and the model structure. In the training method, FGM (Fast Gradient Method) adversarial training is introduced in the finetune phase of the model by adding the disturbances in both the language embedding layer's and visual feature linear layer's weights, which effectively improves the model accuracy and generalization ability. In the model structure, a residual block with weight is used to improve the training speed by 1.6% in the pre-training phase of this model without losing the model performance. Next, t the most important structure, the Encoder, is redesigned to make the model more convergent. The Encoder's FFN (Feed-Forward Neural Network) is replaced by GLU (Gated Linear Unit), which also improves the ability of model fitting and model performance. The improved model performs better on the VQA task than the benchmark (i.e., LXMERT). In the end, detailed ablation studies prove that my enhancement strategies are effective for LXMERT and observe the effectiveness of different measures on the model.
Vehicle and pedestrian detection method based on improved YOLOX in foggy environment

Most of the current vision sensor-based target detection is suitable for good weather conditions. Adverse weather conditions, especially foggy environments, significantly reduce visibility, which seriously affects the target detection performance. To improve driving safety in foggy environments, this paper proposes an improved YOLOX-based vehicle and pedestrian detection method in foggy environments. The method is based on the advanced YOLOX network model and introduces an attention mechanism in the feature extraction network to enhance the network's extraction of target features in foggy images. Some images in the training dataset are fogged to supplement the target-specific features in foggy environments and improve the robustness of the target detection network in foggy environments. The idea of migration learning is used in the training process to save training time and optimize the training effect. The experimental results show that the target detection method proposed in this paper has significantly improved the detection performance of vehicles and pedestrians in the foggy environment, with an 11.35% improvement in mAP, and the detection effect is better than the GCANet image defogging method. The effectiveness of the method improvement is proved.
VRGNet: A Robust Visible Region-Guided Network for Occluded Pedestrian Detection

Pedestrian detection has made significant progress in both academic and industrial fields. However, there are still some challenging questions with regard to occlusion scene. In this paper, we propose a novel and robust visible region-guided network (VRGNet) to specially improve the occluded pedestrian detection performance. Specifically, we leverage the adapted FPN-based framework to extract multi-scale features, and fuse them together to encode more precision localization and semantic information. In addition, we construct a pedestrian part pool that covers almost all the scale of different occluded body regions. Meanwhile, we propose a new occlusion handling strategy by elaborately integrating the prior knowledge of different visible body regions with visibility prediction into the detection framework to deal with pedestrians with different degree of occlusion. The extensive experiments demonstrate that our VRGNet achieves a leading performance under different evaluation settings on Caltech-USA dataset, especially for occluded pedestrians. In addition, it also achieves a competitive of 48.4%, 9.3%, 6.7% under the Heavy, Partial and Bare settings respectively on CityPersons dataset compared with other state-of-the-art pedestrian detection algorithms, while keeping a better speed-accuracy trade-off.
Chinese Electronic Medical Record Named Entity Recognition Based on Bi-RNN-LSTM-RNN-CRF

Based on the mainstream deep learning model BiLSTM-CRF, the electronic medical record named entity recognition model Bi-RNN-LSTM-RNN-CRF is established. First collect the electronic medical record data set, then convert the characters into vectors through the word vector tool, enter them into the bidirectional RNN-LSTM-RNN layer for training, and then enter the training results into the CRF layer, calculate the loss function to obtain the prediction results, and record the time that the process took.Finally, repeat the above steps with the traditional BiLSTM-CRF model to compare the results of the two models. Experimental results show that the F1 value of the Bi-RNN-LSTM-RNN-CRF model can reach 97.80%, and the recognition effect is slightly inferior to that of BiLSTM-CRF.
HRPM Net: An Efficient Feature Learning Network from The Biological Modelling Of Human Retinal Perception Mechanism

The biological model of the mammal visual mechanisms is very beneficial to feature learning in motionless images. It is proved that the visual mechanisms can improve the performance of the hand-crafted methods and CNNS method. Recently CNNs learn discriminate and robust features by changing the backbone, processing multi-scale feature maps, and adding attention mechanisms, etc. While they are relatively short of changing the network structure with human retina mechanisms, which have been proven to have a strong feature extract capability of images by traditional feature descriptors. To address this problem, we present two CNN blocks, multi-scale receptive field convolutional block (MSRF) and Sensitivity block (SENSI), both of which are constructed by modeling the human retina ganglion cell's mechanisms. MSRF is designed to enhance the feature discriminability and robustness by imitating the exponentially increased way of the receptive fields of the P ganglion cells in the human retina. We constructed experiments to get the specific value of the size of the receptive fields, and it can capture both local and global features with various convolution kernels. SENSI is presented to make sure each receptive field has a suitable weight to choose which receptive field can better learn features. Both of them help to learn features and can be easily integrated into the existing CNN models. The framework is evaluated on two benchmark datasets. We further assemble MSRF and SENSI to the top of SSD, constructing the HRPM Net. The model outperforms the state-of-the-art approaches by a considerable margin on MS COCO, VOC 2012, and VOC 2007 datasets. The results also show that MSRF block and SENSI block are helpful in feature learning and can improve the performance by a margin.
MoAFormer: Aggregating Adjacent Window Features into Local Vision Transformer Using Overlapped Attention Mechanism for Volumetric Medical Segmentation

The window-based attention is used to alleviate the problem of abrupt increase in computation as the input image resolution grows and shows excellent performance. However, the problem that aggregating global features from different windows is waiting to be resolved. Swin-Transformer is proposed to construct hierarchical encoding by a shifted-window mechanism to interactively learn the information between different windows. In this work, we investigate the outcome of applying an overlapped attention block (MoA) after the local attention layer and apply plenty to medical image segmentation tasks. The overlapped attention module employs slightly larger and overlapped patches in the key and value to enable neighbouring pixel information transmission, which leads to significant performance gain. The experimental results on the ACDC and Synapse datasets demonstrate that the used method performs better than previous Transformer models.
A Brief Comparison of Deep Learning Methods for Semantic Segmentation

Semantic segmentation is widely used in many fields, which is the key and difficult point in computer vision field.In recent years, with the rapid development of deep learning, deep learning has greatly improved the performance of semantic segmentation. Many methods have been proposed.This paper mainly reviews the research progress of semantic segmentation model based on convolutional neural network (CNN), compares the methods of the same class, and analyzes the connection and difference of each method. In this paper, we mainly discuss the recent semantic segmentation models to improve segmentation accuracy from different strategies, and compare and analyze the relationships and differences between these methods. we also prospected the future development direction of semantic segmentation methods.This paper hopes to give readers an understanding of the progress and challenges of CNN based semantic segmentation research.
A Mask Detection Algorithm Based on Improved Yolov5s

Aiming at the severe form of new coronavirus epidemic prevention and control, a target detection algorithm is proposed to detect whether masks are worn in public places. The Ghostnet and SElayer modules with fewer design parameters replace the BottleneckCSP part in the original Yolov5s network, which reduces the computational complexity of the model and improves the detection accuracy. The bounding box regression loss function DIOU is optimized, the DGIOU loss function is used for bounding box regression, and the center coordinate distance between the two bounding boxes is considered to achieve a better convergence effect. In the feature pyramid, the depthwise separable convolution DW is used to replace the ordinary convolution, which further reduces the amount of parameters and reduces the loss of feature information caused by multiple convolutions. The experimental results show that compared with the yolov5s algorithm, the proposed method improves the mAP by 4.6% and the detection rate by 10.7 frame/s in the mask wearing detection. Compared with other mainstream algorithms, the improved yolov5s algorithm has better generalization ability and practicability.