Reviewerly/exHarmony huggingface dataset:
Cross-Modality Encoder Representations Based On External Attention Mechanism

With the prevalence of deep learning, people use multi-modality information for interpretation and reasoning. In this paper, a cross-modality encoder CMEEA (cross-modality encoder representation based on external attention mechanism) is adopted to improve the accuracy of the model. The two external storage units of CMEEA can be regarded as a dictionary of the whole dataset to improve the performance of the network, while learning more representative features of the input and reducing the computational cost. This paper uses five pretraining tasks that help the model learn internal modal and cross-modal relationships. It also verifies the generalizability of the model by applying the pretraining cross-modal model to the visual inference task NLVR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , and improves the previous results by 0.1%. By fine-tuning the pretrain parameters, the model improves by 1.3% on visual question answering (VQA).
Evaluation Model of Learner's Cognitive Level Based on RoBERTa Fused with CNN

The cognitive level is an essential indicator of the learning process evaluation. However, most of the existing cognitive level evaluation methods use manual coding or traditional machine learning methods, which cannot fully mine the implicit cognitive semantic information in unstructured text data. In this study, a fusion model named RoBERTa-CNN is proposed, which uses RoBERTa pre-training model to capture the cognitive semantic features implied between words in the sentence. Furthermore, it combines a convolution neural network to extract the global semantic features of sentences, to extract the deep-level features of the text. Using Bloom's cognitive goal classification theory as the framework of cognitive level evaluation, 9657 text data from MOOC were evaluated automatically. The results show that the mixed RoBERTa-CNN model has the best evaluation effect, and the overall accuracy rate of the six cognitive levels is 85.26%, demonstrating the model's validity.
An artificial bee colony algorithm improved by a search strategy based on adaptive selection

In order to solve the problems of slow convergence and low accuracy of the basic Artificial Bee Colony algorithm, an adaptive selection strategy using population information is proposed in this paper. The search strategy is controlled by using the change rate of focusing distance, which not only ensures the utilization of the population information, but also improves the utilization of the best individual information by the algorithm. By comparing the improved algorithm with the basic algorithm through experiments, the results show that the improved algorithm has improved the search performance and convergence performance.
Research on power prediction of clean energy and operation optimization of clean energy supply system for typical regions

With the world energy shortage and the trend of global warming, the focus of energy is gradually shifting to clean energy. The promotion of clean energy represented by solar energy and wind energy and the development of efficient energy supply system and operation have gradually become the focus of attention. However, when large-scale photovoltaic power generation is integrated into the grid, it is easy to cause the paralysis of the power system. At the same time, due to the lack of effective energy system operation strategy, it is easy to cause the problem of resource waste and low energy utilization in typical regional clean energy supply system. Based on this, this paper first proposed a short-term photovoltaic power prediction method based on LSTM according to the characteristics of strong timing of photovoltaic power, predicted the irradiance data, and evaluated and compared the predicted effect. Then, a high-efficiency CCHP microgrid structure for typical areas is proposed, and the operation strategy of this system is optimized based on particle swarm algorithm. The experimental results show that the proposed short-term photovoltaic power prediction method based on LSTM has higher prediction accuracy than other prediction methods. At the same time, the micro-grid structure and the operation strategy of the design proposed in this paper have high economy, thus verifying the advanced nature of the work in this paper.
ViT-ReID: A Vehicle Re-identification Method Using Visual Transformer

In the current vehicle re-identification task based on CNN method, due to the loss of detail information of image caused by convolution and downsampling operation, the ability to distinguish similar vehicles is affected. In this paper, a new vehicle Re-ID method ViT-ReID using visual Transformer was proposed. In order to extract robust discriminative features to enhance the ability of the model to distinguish similar vehicles, a SIE module based on ViT backbone network was designed to effectively reduce the deviation of learning features caused by camera perspective changes. At the same time, in order to improve the performance of the model, the latest VPT method for fine-tuning of large-scale Transformers was introduced. Finally, the ViT-ReID model proposed in this paper was compared with other methods in the VeRi-776 and VehicleID datasets. The experimental results showed that the proposed method can effectively extract robust discriminative features in vehicle re-identification and distinguish similar vehicles with high accuracy.
Fast rare itemset mining in uncertain database

Rare itemset mining in uncertain database finds itemsets which have their support no less than a given probability minimum threshold. Discovering probabilistic rare itemset from a uncertain database can help user in better decision making by focusing on information that it is important but rare to something. Usually, frequent itemset and its relevant researches have gotten a lot of attention in the previous works, and much more efficient and effective algorithms was proposed to obtain valued knowledge under various constrained scenarios. Compared with frequent itemset mining, especially probabilistic rare itemset mining are rarely studied. Therefore, we present the concept of mining rare itemsets in uncertain data based on rare probabilities, and propose two algorithms, i.e. the bottom-up algorithm and the top-down algorithm. On this basis, two pruning strategies are designed to reduce the computational cost by traversing from long itemsets to short itemsets. Finally, we validate and evaluate the correctness and effectiveness of the algorithms in several experiments.
STANet: a Spatial-Temporal Aggregation Network for Video Deraining

Rain can significantly reduce the visibility of a scene, which hinders a lot of computer vision systems, such as outdoor surveillance and autonomous driving. Comparing with single images, rainy videos can record abundant spatial-temporal information of target scenes, which have tremendous advantages in a wide of computer vision tasks. However, how to effectively extract and aggregate spatial representations and temporal correlations of a video sequence for rain removal is a nontrivial task. In this paper, we propose a novel spatial-temporal aggregation network, called STANet, for video deraining. It can learn the temporal correlations and spatial representations of consecutive rainy frames, and then adaptively aggregate them through an iterative attention mechanism. Finally, a refinement sub-network is proposed to further remove the remaining rain streaks and generate clean rain-free video. Extensive experiments demonstrate that our proposed network outperforms state-of-the-art methods by delivering excellent results in both quality and speed.
Layer-wise Top-$k$ Gradient Sparsification for Distributed Deep Learning

Distributed training is widely used in training large-scale deep learning models, and data parallelism is one of the dominant approaches. Data-parallel training has additional communication overhead, which might be the bottleneck of training system. Top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{k}$</tex> sparsification is a successful technique to reduce the communication volume to break the bottleneck. However, top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{k}$</tex> sparsification cannot be executed until backpropagation is completed, which disables the overlap of backpropagation computations and gradient communications, leading to limiting the system scaling efficiency. In this paper, we propose a new distributed optimization approach named LKGS-SGD, which combines synchronous SGD (S-SGD) with a novel layer-wise top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{k}$</tex> sparsification algorithm (LKGS). The LKGS-SGD enables the overlap of computations and communications, and adapts to gradient exchange at layer-wise. Evaluations are conducted by real-world applications. Experimental results show that LKGS-SGD achieves similar convergence to dense S-SGD, while outperforming the original S-SGD and S-SGD with top- <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{k}$</tex> sparsification.
Spatiotemporal Action Detection Based on Improved Action Tube Connection

Spatiotemporal action detection is a hotspot in the field of computer vision, and has a wide range of application prospects. By analyzing the algorithms of action tube connection, we found that the large disparity between linking scores of regions in consecutive frames reduces the detection performance. To address this problem, a framework is proposed, which consists of the 2D ConvNet branch, 3D ConvNet branch, channel attention fusion block, bounding box regression and smooth action tube connection block. In particular, the smooth action tube connection block reduces the negative impact on the detection results due to the large difference in detection scores. Furthermore, extensive experiments are conducted on the UCF101-24 dataset. The proposed method achieves better performance than the baseline methods, and obtains the best experimental results, when video-mAP with an IoU threshold of 0.5 is used as the evaluation metric.
PLA: Fast and Accurate Face Alignment Network based on Prior Landmarks

In this paper, we proposed a new method called Prior Landmark Algorithm (PLA) to address the limitations of traditional face landmark prediction techniques. PLA predicts the offset between real and prior landmarks to overcome the issue of solvable domain coverage and to handle different face poses and shapes. A new loss function with weights (PL-Loss) and the strategy of Online Hard Example Mining were also introduced to improve the accuracy of the model. The adoption of prior landmarks and an auxiliary classifier eliminate the need for face normalization during training. The results show that PLA achieved comparable accuracy to state-of-the-art models and had a small model size of only 10 MB with fast processing speed of 162 FPS on average.
BERTGACN: Text Classification by Combining BERT and GCN and GAT

With the explosive growth of electronic data, text has become one of the most important information carriers, and it has become the most common form of data display. Text classification involves many fields such as artificial intelligence and pattern recognition, so it has important academic research significance and commercial value. How to make the computer accurately locate the effective information in the text and realize automatic classification has become one of the research hotspots. In this study, we propose the BERTGACN model, which combines the large-scale pre-training model and the double-tower structure Graph Neural Networks (GNNs) model for text classification tasks. The BERTGACN model creates a heterogeneous graph for representing documents and words as nodes, respectively. By jointly training the Bidirectional Encoder Representations from Transformers (BERT) module, Graph Convolutional Network (GCN) module and Graph Attention Network (GAT) module, the BERTGACN model effectively learns the structural information of the graph and the association information between nodes, which enhances its text classification ability. Experiments show that the BERTGACN model achieves better results than previous models in a wide range of text classification tasks.
A Multi-scale Fusion Network with Transformer for Medical Image Segmentation

To further exploit the advantages of convolutional neural network (CNN) and Transformer, we introduce a new Multi-scale Fusion Network to this paper. With the U-shaped attention model, we introduce multi-scale blocks in the encoder phase to sufficiently exploit the multi-scale semantic information. We further invoke cross-fusion of the multi-scale channels with Transformer to reconstruct skip connections, which provides the decoder with different levels of long-range information. Moreover, we utilize a scale-aware pyramid fusion module built into the bottom of our framework for the dynamic fusion of multiscale contextual information from higher-level features. The results on two datasets indicate that the proposed approach obtains competitive performance and exceeds the comparison networks, which to some extent relieves the burden of physicians.
Towards Graph Contrastive Learning for Recommendation with Sampling Embedding Perturbation

Nowadays, graph collaborative filtering is the most practical and classical method for recommender systems, which learns user preferences through historical interactions between users and items. The recent combination of contrastive learning and recommender systems has led to a continuous improvement in the performance of graph collaborative filtering. A variety of data augmentation methods have also emerged. However, some graph augmentation methods are proven to lose graph information, such as the way of dropout on the graph structure. Inspired by different adversarial samples and different data augmentation methods, we adopt the method of adding interference patterns to the embedding space, as opposed to other models, we take a partial sampling of the embedding and then add the partially sampled data into the embedding, thus forming different sub-views. The extensive experimental results show that our experiments can improve the performance of the model more compared to other contrastive learning methods with our data enhancement method.
Cycling image encryption algorithm based on the new composite chaotic mapping

In this paper, a new two-dimensional chaotic system (BTCS) that is based on composite of Bezier surface and trigonometric function is proposed. This system has large key space and demonstrates good chaotic behavior. In the meantime, using BTCS, a new encryption scheme (CIEA), named the Cycling image encryption algorithm, is proposed for encrypting numerous same-size images. The kernel of the scheme is that using the newly encrypted image encrypts a new image, circularly, which can improve security and is very efficient. Generation of pseudo-random matrices, scrambling based on matrix and XOR diffusion operation are included in each loop of CIEA. Security analyses and experiment results show that the CIEA performs well in key space, key sensitivity, histogram, information entropy, correlation analysis and anti-differential attack test.
A New Approach in Discrete Event System by Deep Monte Carlo Tree Search

The decision-making problem is one of core issues in discrete event system, which is often used to describe adversarial complex environment. Aiming at the problem that there exists huge space in this kind of environment, we use Monte Carlo Tree Search (MCTS) as our main search algorithm. One of the key challenges is that, during simulation, it is difficult to accurately predict opponent agents' actions. To deal with this challenge, we propose a deep neural network based on Long Short-Term Memory Network (LSTM), which is trained to predict most possible actions from opponent agents, the prediction is used to guide simulation. We test the performance of our method in the field of autonomous driving (AD), experiments are carried out on the simulation platform CARLA. We design a route containing several dangerous events. Results show that, if the prediction is accurate enough, algorithm using MCTS and deep learning outperforms the one using MCTS only, the former has a higher success rate in avoiding collisions, meanwhile it has a higher average speed.
A Flexible and Low-Resource CNN Accelerator on FPGA for Edge Computing

With the development of deep learning, convolutional neural networks are widely used in the areas of edge computing, such as gait recognition, video surveillance, human-computer interaction, entertainment, and gaming. However, as the number of network layers deepens, CNNs are inappropriate to be deployed in the time-sensitive edge system. Most existing CNN accelerators require significant data movement and frequent off-chip memory (DRAM) access, which will decreases the inference speed and power efficiency. Field Programmable Gate Arrays (FPGA) is considered as a promising platforms for accelerating CNN s, due to its reconfigurable property. To address this issue, This paper designs an FPGA based CNN accelerator for edge computing, which can significantly reduce the dependence on logical resources and storage resources. An improved 4-bit PoT quantization (CPoT) was proposed to compress the model. Our design is assessed by deploying models on Intel Arria 10 GXl150 FPGA platform. As revealed from the experimental results, compared with other accelerator designs, our design can significantly reduce the utilization of Logic Elements (LE). The performance(throughput) of our accelerator achieves 8.92ms per image with a size of, 1.58 times faster than the GPU accelerator on an high-end NVIDIA GPU.
Unsupervised Abnormality Detection with Normalizing Flow and Neural Network

In anomaly detection, the limited number of abnormal samples results in a significant long-tail distribution in the data. Additionally, labeling is heavily dependent on the experience and can be time-consuming. This problem is particularly prominent in the field of abnormality detection from medical images. To this end, we propose a novel end-to-end unsupervised method for detecting abnormalities using a flow model-based neural network. Specifically, This approach utilizes deep neural networks to obtain the features of normal images and model the distribution with the statistical method. It maps image features to tractable distributions and effectively integrates both local and global relationships to identify distinctive features. Comprehensive experiments on the MURA dataset demonstrate that our method outperforms recent methods by a considerable margin.
Partial VGG16: reformed network for classification under partial occlusion

Standard deep Convolutional Neural Networks (CNNs), even those with encoder-decoder transformers, often struggle to generalize well when faced with partial occlusion in images. To address this issue, we were inspired by the success of inpainting models that are particularly sensitive to the missed areas in an image. Therefore, we propose to replace the conventional convolutional layers in CNNs with partial convolution blocks, which are commonly used in image inpainting and other related tasks. This new structure allows CNNs to focus only on the valid parts of an image by ignoring the masked parts. To evaluate this approach, we conducted experiments on a model enhanced with partial convolution blocks, by artificially removing rectangular areas from the original images. The results indicate that this model is more sensitive to the non-occluded parts of the object and achieves higher precision than the standard VGG16 network, with a 5% (±2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup> ) increase in top-1 accuracy, has reached to 94.11% in top-2 accuracy.
YOLO-Owl: An Occlusion Aware Detector for Low Illuminance Environment

Object detection algorithms can be roughly divided into two categories, two-stage detectors like Fast RCNN and one-stage detectors like YOLO. We propose an object detection model that works well in low illuminance environment. We design the MRFM and FEID module for low illuminance targets, as well as a new training model method for low illumination target that allows the model to converge stably and quickly. In addition, we propose an occlusion-aware attention module MPCM to deal with target occlusion in low-light environment. We expand the ExDark dataset and run experiments on the expanded dataset. The results show that the performance of our proposed model is better than YOLOv5 and YOLOX.
A new appearance-based gaze estimation via multi-modal fusion

Appearance-based gaze estimation has gained more and more attention because of its generality, robustness, and subject independence. Deep learning, which has made a great deal of success in computer vision, has also greatly improved the accuracy of appearance-based gaze estimation. To further reduce the error in gaze estimation, we focus on extracting better feature information from eye and face images. In this paper, we propose a novel multimodal fusion gaze estimation model based on ConvNext and dilated convolution. In this model, the eye image and face image are used as input, and the ConvNext network is used to extract the features of the face image and the eye features are extracted by a dilated convolution-based network, and the feature map of the two images are fused using the fully connected layer to perform gaze estimation. In the experimental part, the designed model is verified on the public dataset MPIIGaze, and compared the proposed model with other gaze estimation models. The experimental results show that our proposed method has greatly improved the accuracy of gaze estimation on the MPIIGaze dataset compared to other related works. Our proposed multimodal fusion gaze estimation model achieves state-of-the-art result on the MPIIGaze dataset.
Neural Matrix Factorization Model Based On Latent Factor Learning

Aiming at the typical problem of predicting users' evaluation of items in recommender systems, we propose a neural matrix factorization model based on latent factor learning. Based on the classical latent factor model, the model utilizes the representation ability of deep learning, adds auxiliary features and cross features in the process of vectorization to improve the abstract expression ability of the model to the latent factor vector, and realizes the improvement of the model by auxiliary feature information. Finally, compared with the existing deep learning models, the prediction results are significantly improved in RMSE and MAE metrics.
An improved multi-label classification algorithm based on YOLOV5s

At present, the optimization of multi-label classification algorithm based on deep learning is not perfect. Based on this, this paper studies the multi-label classification algorithm based on detection. Based on YOLOv5s, an improved algorithm for image multi- label classification is proposed, which integrates SimAM attention- free module and GAM attention-mechanism effectively. Retain channel and spatial information to enhance cross-dimensional interaction, and improve the flexibility and effectiveness of the module. The experimental results show that the mean average accuracy (mAP) of the improved algorithm on the COCO dataset reaches 54.8%, which is 2.2% higher than that of the original YOLOv5s algorithm, and achieves good results in multi-label classification tasks.