Reviewerly/exHarmony huggingface dataset:
Temporal interaction and excitation for action recognition

Two-stream networks have been widely used in action recognition by integrating the appearance information from RGB frames with the motion-rich optical flow data, resulting in impressive recognition accuracy. However, the drawbacks of two-stream networks using 2D convolutional neural network (CNN) are apparent. The computation of optical flow is resource intensive and time consuming, and 2D CNN cannot model temporal information. To alleviate these problems, we propose a temporal interaction and excitation (TIE) module that can be embedded into existing 2D CNN in a plug-and-play manner. It comprises two components: the temporal interaction fusion (TIF) module and the motion excitation (ME) module. The TIF module employs channel-wise temporal convolution to adaptively fuse information from adjacent video frames, facilitating information exchange between the adjacent frames while maintaining its spatial feature learning capability. The ME module is a lightweight motion extraction module that leverages the differences between adjacent frames to model feature-level motion information, replacing the need for traditional optical flow. This enhancement helps the model better understand human actions in videos. The TIE module is designed to improve the network performance while adding minimal computational cost. Experimental results demonstrate that our proposed method adds only a few parameters and computational costs while achieving competitive results on HMDB-51 and UCF-101 datasets.
Balanced Cloud Shadow Compensation Method in High-Resolution Image Combined with Multi-Level Information

As clouds of different thicknesses block sunlight, large areas of cloud shadows with varying brightness can appear on the ground. Cloud shadows in high-resolution remote sensing images lead to uneven loss of image feature information. However, cloud shadows still retain feature information, and how to compensate for and restore unbalanced cloud shadow occlusion is of great significance in improving image quality. Though traditional shadow compensation methods can enhance the shaded brightness, the results are inconsistent in a single shadow region with over-compensated or insufficient compensation problems. Thus, this paper proposes a shadow-balanced compensation method combined with multi-level information. Multi-level information comprising the information of a shadow pixel, a local super-pixel centered with the pixel, the global cloud shadow region, and the global non-shadow region information, to comply with the cloud shadow’s internal difference. First, the original image is detected via the cloud shadow detection method and post-processing. The initial shadow is detected combined with designed complex shadow features and morphological shadow index features with threshold methods. Then, post-processing considering shadow area and morphological operation is applied to remove the small, non-cloud-shadow objects. Meanwhile, the initial image is also divided into super-pixel homogeneity regions using the super-pixel segmentation principle. A super-pixel region is between the pixel and the shadow area. Different from pixel and other window regions, it can provide a different measurement levels considering object homogeneity. Thus, a balanced compensation model is designed by combining the feature value of a shadow pixel and the mean and variance of a super-pixel, shadow region, and non-shadow region on the basis of the linear correlation correction principle. The super-pixel around the shadow pixel provides a local reliable homogenous region. It can reflect the internal difference inside the shadow region. Therefore, introducing a super-pixel in the proposed model can effectively compensate for the shaded information in a balanced way. Compared to those of only using pixel and shadow region information, the compensated results introduce super-pixel information, can deal with the homogenous region as a global one, and can be adaptive to the illustration differences in a cloud shadow. The experimental results show that compared to that of other reference methods, the quality of the proposed compensation result is better. The proposed method can enhance brightness and recover detailed information in shadow regions in a more balanced way. The issue of over-compensation and insufficient compensation inside a single shadow region can be resolved. Thus, the total result is similar to that of a non-shadow region. The proposed method can be used to recover the cloud shadow information more self-adaptively to improve image quality and usage in other applications.
Breast Cancer Classification Using Concatenated Triple Convolutional Neural Networks Model

Improved disease prediction accuracy and reliability are the main concerns in the development of models for the medical field. This study examined methods for increasing classification accuracy and proposed a precise and reliable framework for categorizing breast cancers using mammography scans. Concatenated Convolutional Neural Networks (CNN) were developed based on three models: Two by transfer learning and one entirely from scratch. Misclassification of lesions from mammography images can also be reduced using this approach. Bayesian optimization performs hyperparameter tuning of the layers, and data augmentation will refine the model by using more training samples. Analysis of the model’s accuracy revealed that it can accurately predict disease with 97.26% accuracy in binary cases and 99.13% accuracy in multi-classification cases. These findings are in contrast with recent studies on the same issue using the same dataset and demonstrated a 16% increase in multi-classification accuracy. In addition, an accuracy improvement of 6.4% was achieved after hyperparameter modification and augmentation. Thus, the model tested in this study was deemed superior to those presented in the extant literature. Hence, the concatenation of three different CNNs from scratch and transfer learning allows the extraction of distinct and significant features without leaving them out, enabling the model to make exact diagnoses.
A multi-scenario approach to continuously learn and understand norm violations

Abstract Using norms to guide and coordinate interactions has gained tremendous attention in the multiagent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to changing human needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning to detect norm violations from the limited interaction data and to explain the reasons for such violations. To do that, we propose a framework that combines Machine Learning (ML) models and incremental learning techniques. Our proposal is equipped to solve tasks in both tabular and text classification scenarios. Incremental learning is used to continuously update the base ML models as interactions unfold, ensemble learning is used to handle the imbalance class distribution of the interaction stream, Pre-trained Language Model (PLM) is used to learn from text sentences, and Integrated Gradients (IG) is the interpretability algorithm. We evaluate the proposed approach in the use case of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Results show that the proposed framework can learn to detect norm violation in a setting with data imbalance and concept drift.
Towards hypergraph cognitive networks as feature-rich models of knowledge

Abstract Conceptual associations influence how human memory is structured: Cognitive research indicates that similar concepts tend to be recalled one after another. Semantic network accounts provide a useful tool to understand how related concepts are retrieved from memory. However, most current network approaches use pairwise links to represent memory recall patterns (e.g. reading “airplane” makes one think of “air” and “pollution”, and this is represented by links “airplane”-“air” and “airplane”-“pollution”). Pairwise connections neglect higher-order associations, i.e. relationships between more than two concepts at a time. These higher-order interactions might covariate with (and thus contain information about) how similar concepts are along psycholinguistic dimensions like arousal, valence, familiarity, gender and others. We overcome these limits by introducing feature-rich cognitive hypergraphs as quantitative models of human memory where: (i) concepts recalled together can all engage in hyperlinks involving also more than two concepts at once (cognitive hypergraph aspect), and (ii) each concept is endowed with a vector of psycholinguistic features (feature-rich aspect). We build hypergraphs from word association data and use evaluation methods from machine learning features to predict concept concreteness. Since concepts with similar concreteness tend to cluster together in human memory, we expect to be able to leverage this structure. Using word association data from the Small World of Words dataset, we compared a pairwise network and a hypergraph with $N= 3586$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>N</mml:mi> <mml:mo>=</mml:mo> <mml:mn>3586</mml:mn> </mml:math> concepts/nodes. Interpretable artificial intelligence models trained on (1) psycholinguistic features only, (2) pairwise-based feature aggregations, and on (3) hypergraph-based aggregations show significant differences between pairwise and hypergraph links. Specifically, our results show that higher-order and feature-rich hypergraph models contain richer information than pairwise networks leading to improved prediction of word concreteness. The relation with previous studies about conceptual clustering and compartmentalisation in associative knowledge and human memory are discussed.
Assessment of various Hamiltonian partitionings for the electronic structure problem on a quantum computer using the Trotter approximation

Solving the electronic structure problem via unitary evolution of the electronic Hamiltonian is one of the promising applications of digital quantum computers. One of the practical strategies to implement the unitary evolution is via Trotterization, where a sequence of short-time evolutions of fast-forwardable (i.e. efficiently diagonalizable) Hamiltonian fragments is used. Given multiple choices of possible Hamiltonian decompositions to fast-forwardable fragments, the accuracy of the Hamiltonian evolution depends on the choice of the fragments. We assess efficiency of multiple Hamiltonian partitioning techniques using fermionic and qubit algebras for the Trotterization. Use of symmetries of the electronic Hamiltonian and its fragments significantly reduces the Trotter error. This reduction makes fermionic-based partitioning Trotter errors lower compared to those in qubit-based techniques. However, from the simulation-cost standpoint, fermionic methods tend to introduce quantum circuits with a greater number of T-gates at each Trotter step and thus are more computationally expensive compared to their qubit counterparts.
Modular quantum circuits for secure communication

Abstract Quasi‐chaotic generators are used for producing a pseudorandom behaviour that can be used for encryption/decryption and secure communications, introducing an implementation of them based on quantum technology. Namely, the authors propose a quasi‐chaotic generator based on quantum modular addition and quantum modular multiplication and they prove that quantum computing allows the parallel processing of data, paving the way for a fast and robust multi‐channel encryption/decryption scheme. The resulting structure is validated by means of several experiments, which assessed the performance with respect to the original VLSI solution and ascertained the desired noise‐like behaviour.
A WebExtension framework for experimentation and evaluation of webpage segmentation methods

Current webpages contain areas with different functions and contents. Many studies and applications have used webpage segmentation methods to separate these areas or extract only specific areas for their purposes. Examining these methods requires laborious tasks, such as collecting many webpages, inspecting them with human participants, and applying various performance metrics to their results. Therefore, we developed a WebExtension (browser extension) framework to support the examination and analysis of webpage segmentation methods. This framework can build a WebExtension to collect webpages, curate data for labeling web documents, evaluate methods, and measure the results with various performance metrics in a web browser environment. Furthermore, researchers can use preloaded well-known methods and metrics in the framework and add more methods and metrics for their research purposes.
Printing roller image salient object detection based on multi-scale feature fusion

Abstract In this study, aiming at the problem that the surface defect image of the printing roller has different sizes and a large number of defects, a salient object detection method based on stepwise multi-scale feature fusion is proposed, which consists of a stepwise multi-scale feature guidance module, a multi-resolution information fusion module, and a deeply supervised side output. Specifically, the stepwise multi-scale feature guidance module contains two paths, top-down and bottom-up, which strengthen the transmission of multi-scale features in the network, and fuse multi-level features through the multi-resolution information fusion module to suppress the interference of background noise and redundant features. In addition, deeply supervised side outputs are used to eliminate differences caused by background and foreground imbalance. The experimental results on the roller surface defect dataset show that the method can achieve Fmax , MAE of 0.7959, 0.0144, and the detection rate can reach 24.98fps, which has better detection performance than other algorithms.
Multi-feature Fusion Based on RV Correlation Coefficient for Facial Expression Recognition

Abstract Extracting effective features of expressions becomes a hot research topic, and a single feature pattern cannot reflect the diversity of expressions. Therefore, to obtain rich information feature data and raise the expression recognition performance, we propose a feature fusion model of multiple feature selection by the measure of the RV correlation coefficient. In the proposed feature fusion model, the feature patterns are firstly selected by RV correlation coefficient from various expression texture features. And then according to rank the values of the RV correlation coefficient, we build a CCA subspace and PCA subspace respectively to fuse selected features. Finally, a new facial expression feature presentation is constructed through weighting and combining the two fusion features from the subspaces. The new features are fed to SVM classifier for expression recognition. Experimental verification shows that our proposed model has a superior performance than the existing algorithms.
Improvement of knowledge tracing by integrating exercise features and relationships of concepts

Abstract Knowledge Tracing (KT) is a challenging task in personalized learning where the objective is to track the progression of students’ understanding of the concepts over time based on their learning history. Typically, there are many more exercises than knowledge points. Tracing the state of the concepts seems a reasonable approach but ignores the additional information contained in the exercise itself. Moreover, there are numerous relationships between knowledge points, and the modification of one point’s state can affect related points. In this paper, we introduce a method that splits exercises into constituent knowledge points, and incorporates the exercise features to improve the differential representation of the exercises containing the same concepts. We also describe two ways of propagating knowledge point states: one-way and two-way, which update the state of the current knowledge point and its related knowledge points. Our model is tested on two real datasets, and the experimental results show that our model outperforms the existing methods.
Research and Development of Machine Vision Algorithm Performance Evaluation System in Complex Scenes

Abstract In recent years, deep learning technology has gained widespread attention and has been successfully applied in various applications, including machine vision for automatic driving. However, one of the major challenges in this field is the ability of AI models to perform well in complex scenes with varying weather conditions and lighting conditions. The proposed scheme uses VR technology to simulate different weather conditions and lighting conditions and to evaluate the performance of AI products in such complex scenes. To further enhance the generalization capabilities of AI models, the Software Development Kit has been updated with complex scene data with configurable parameters, and training data with typical characteristics according to machine vision tasks. This approach has the potential to significantly improve the accuracy and reliability of machine vision systems for automatic driving, ultimately enhancing their safety and effectiveness.
Multi-branch Context Awareness Network for Prostate MRI Segmentation

Abstract Prostate image segmentation is a precondition for the diagnosis of prostate diseases and subsequent treatment. However, the blurry organ edges and low image contrast make accurate segmentation difficult. In order to overcome the difficulties, this paper proposes an innovative Multi-branch Context Awareness Network (MBCA-Net) for prostate MRI segmentation. MBCA-Net uses the 3D UNet as the backbone network with an encoder-decoder framework. To improve the feature extraction capacity of the network, a multi-branch residual module with different convolution kernels is proposed. To better make use of extracted features and reduce semantic ambiguity, a context awareness module is used to fuse low-level features in the encoder and high-level features in the decoder in a full process manner. To improve the stability of the model during training, a mixed loss function is adopted. Experiments on the public dataset PROMISE12 show that our model’s Dice coefficient is 90.19%. Compared with other advanced techniques, the proposed model has better segmentation results.
Research on Eye Movement Classification Based on the Deep Learning Model

Abstract To improve the recognition accuracy of eye movement direction based on EOGM, the Emotiv Epoc, and EEG acquisition headset was used to collect the EEG signals of the subjects in real time. After the 14-channel signals were filtered by wavelet, the short-time energy calculation method was used to select the two channels most sensitive to eye movement. The long and short-term memory network (LSTM) was then used to classify the eye movement signals. On the same data set, the experimental results show that the classification rates in the upper, lower, left, and right directions are up to 91%, 90%, 83%, and 75%, respectively, and the average classification rate is up to 84%. Compared with the existing classification methods, the classification accuracy of the proposed method is higher and the implementation process of the classification algorithm is simpler, which further verifies the feasibility and effectiveness of using EEG to identify the direction of eye movement.
Spectral-domain Spatial-temporal Convolution Graph Neural Network for Industrial Fault Diagnosis

Abstract In modern industrial production, once a fault occurs in the production process, it will affect the whole production and bring huge economic losses. There are a large number of device sensors in modern production, and there are complex interactions between them. But the existing methods can not efficiently and accurately mine the depth of information from mass production data. We proposed the Spectral-domain Spatial-Temporal Convolution Graph Neural Network model, which comprehensively considers the information of spatial and temporal dimensions in production data. The nodes and edges of the heterogeneous graph represent the sensor signals and their interactions. Each sensor data is converted to the spectral domain to extract the time domain features, and graph convolution is used to carry out the spatial relationship between input sequences. Finally, the proposed model is verified using the Three-Phase Flow Facility (TFF) dataset, and the overall accuracy rate reached 96.28%, showing better performance than other baseline models.
The underwater obstacle avoidance method based on ROS

Abstract To achieve real-time obstacle avoidance for underwater robots, the artificial potential field method can be selected as the obstacle avoidance path planning algorithm. However, this method applies a large attractive force to the underwater robot when it is far from the target point, causing it to collide with obstacles. Additionally, when the underwater robot just enters the repulsion influence zone, the repulsion force that it experiences undergoes an abrupt change, resulting in a non-smooth obstacle avoidance path. Furthermore, developing the obstacle avoidance function based on software platforms such as MATLAB has a long algorithm verification cycle, and it is difficult to transfer the algorithm model to practical scenarios. Therefore, this paper proposes an underwater robot obstacle avoidance method based on ROS. Firstly, a dynamic adjustment factor is introduced into the potential field function of the artificial potential field method to redefine the obstacle influence zone, and an improved artificial potential field method is designed as the obstacle avoidance path planning algorithm for underwater robots. The experiment is conducted on the ROS simulation platform, and the results show that the improved algorithm can solve the two problems existing in the artificial potential field method. Secondly, nodes required for underwater robot obstacle avoidance are established based on the ROS operating system, and the improved artificial potential field method is integrated into the obstacle avoidance path planning node. Finally, the obstacle avoidance experiment of the Bluerov2 underwater robot is conducted in a pool, and the experimental results demonstrate that the proposed method can achieve real-time obstacle avoidance for underwater robots.
Multi-person Action Recognition Based on the HARNet Network in the Park Environment

Abstract Action recognition is crucial to public security, playing an important role in pedestrian protection. Recently, numerous approaches based on deep learning models have been reported for action recognition, which mainly has problems such as poor versatility, not good robustness, low recognition effects for low resolution, and occlusion in outdoor environments. This paper addresses this issue and aims to develop an approach for multi-person action recognition in the park environment. Frame extraction operation is first used to extract images from surveillance video in real-time, then YOLOv5 is adopted to detect and segment human targets from images. Thirdly, the AlphaPose algorithm is used to estimate the keypoint heatmap of the human body according to human targets. Finally, the heatmaps are preprocessed and then fused and classified by deep learning networks to complete the multi-person action recognition. In addition, an optimization method, i.e., Heatmap-based Action Recognition Network (HARNet), for ResNet is proposed in order to overcome the difficulty of information extraction from heatmaps, where the attention mechanism is considered. The approach presented here was tested on park surveillance video with multiple different scenarios, and the corresponding results verified its performance.