Reviewerly/exHarmony huggingface dataset:
Future video frame prediction based on generative motion-assistant discriminative network

With the continuous development of deep learning, video frame prediction has become a hotspot in the field of computer vision due to its wide range of applications in anomaly detection, robot decision-making, weather forecasting, and autonomous driving. Although current video frame prediction methods have made remarkable progress, the majority of them directly generate prediction frames by extracting potential spatial distribution patterns from the video data. They lack spatiotemporal information modeling, which leads to high latency, ambiguity, and unrealistic results. In this work, we propose an end-to-end video prediction network model (Generative Differential-Assisted Discriminative Network, abbreviated as GDDNet). It combines the advantages of the difference generation method to extract short-term variations from the image and attention mechanisms to recall global contextual motion information. Furthermore, the differential attention mechanism (DAM) module can guide the model to allocate attention resources more efficiently. These strategies considerably improve the model’s ability to represent motion features in video frames. To further optimize the prediction effect, we introduce adversarial training to enhance the clarity and authenticity of the video frames. In order to ensure the consistency of spatiotemporal distribution between predicted and real frames, we introduce a sequential frame discriminator. Experimental results on the KITTI, UCF-101, and Caltech pedestrian datasets demonstrate the effectiveness of the GDDNet and compare it to the state-of-the-art model. Multi-frame prediction and ablation experiments show that our proposed model not only improves the quality of predictions, but also provides a more flexible prediction framework.
Eye-Blink Detection under Low-Light Conditions Based on Zero-DCE

Eye-blink is an effective tool for human-computer interaction, and it could be a physiological index to judge human activities. Nonetheless, eye-blink reactions not only happen during the daytime, but also blink a lot during nighttime, as blink can moisten the eye when people feel fatigued. In this paper, eye-blink detection under a low-light environment is proposed, improving the success rate of detecting blinks in an insufficient light environment. After comparing two face meshes, which are generated by Dlib and MediaPipe, MediaPipe can yield an abundant and precise face landmark. Even without applying some methods of low-light image enhancement (LLIE), the method of MediaPipe can locate an approximate area of eyes in a nighttime environment. For the problem of detecting blink under a low-light environment, Zero-Reference Deep Curve Estimation (Zero-DCE), a deep learning-based method, is applied. Zero-DCE is used to improve the details of dark blurry images, the advantage of which is zero-reference, i.e., no paired or unpaired data are needed in the training process. Also, Zero-DCE can yield a pleasing result in the aspects of brightness, color, contrast, and naturalness, the details of which will be shown in the following images. When under sufficient light environment, the average success rate of detecting right eye blink is 95.9%, and for left eye blink is 91.2%; when under insufficient light environment without enhancing the image, the average success rate of detecting right eye blink is only 39.7%, and for left eye blink is only 48.8%; when under an insufficient light environment with Zero-DCE, enhancing the quality of image, the average success rate of detecting right eye blink raise to 84%, and for left eye blink raises to 92.7%.
VisdaNet: Visual Distillation and Attention Network for Multimodal Sentiment Classification

Sentiment classification is a key task in exploring people's opinions; improved sentiment classification can help individuals make better decisions. Social media users are increasingly using both images and text to express their opinions and share their experiences, instead of only using text in conventional social media. As a result, understanding how to fully utilize them is critical in a variety of activities, including sentiment classification. In this work, we provide a fresh multimodal sentiment classification approach: visual distillation and attention network or VisdaNet. First, this method proposes a knowledge augmentation module, which overcomes the lack of information in short text by integrating the information of image captions and short text; secondly, aimed at the information control problem in the multi-modal fusion process in the product review scene, this paper proposes a knowledge distillation based on the CLIP module to reduce the noise information of the original modalities and improve the quality of the original modal information. Finally, regarding the single-text multi-image fusion problem in the product review scene, this paper proposes visual aspect attention based on the CLIP module, which correctly models the text-image interaction relationship in special scenes and realizes feature-level fusion across modalities. The results of the experiment on the Yelp multimodal dataset reveal that our model outperforms the previous SOTA model. Furthermore, the ablation experiment results demonstrate the efficacy of various tactics in the suggested model.
VNGEP: Filter pruning based on von Neumann graph entropy

To facilitate the deployment of convolutional neural networks on resource-limited devices, neural network pruning, especially filter pruning, has been shown to be a promising approach. Precise channel importance evaluation requires the extraction of global feature information. In this paper, we propose a novel filter pruning method, VNGEP, which combines intra- and inter-channel information to evaluate channel importance comprehensively. We describe the feature map correlation information from a graph perspective. The feature map tensor is converted into an undirected weighted graph, where the edge weights are the cosine distance between feature maps. To characterize the individual information of the feature maps, we present the feature map energy matrix and embed it into the above undirected weighted graph. Ultimately, the filter importance is measured by the von Neumann graph entropy, thus identifying filters that are less informative and more replaceable by other filters. Moreover, we systematically investigate the computational feasibility and data sensitivity of the proposed scheme. Pruning experimental results for several models on different datasets show the superiority of our approach. For example, with ResNet110, our method achieves a 48.3% and 52.1% reduction in model size and FLOPs, respectively, along with a 1.07% improvement in accuracy over the baseline model on CIFAR10. With ResNet50, we achieve an accuracy improvement of 0.25% over the baseline model on ImageNet, while the model size and FLOPs are reduced by 40.8% and 44.8%, respectively.
Multi-dimensional hypercomplex continuous orthogonal moments for light-field images

Image moments have always been topical in the domain of image processing, and the proposed hypercomplex moments and multi-channel moments have greatly broadened the application of image moments in recent years. However, the hypercomplex moments are limited by the specific number of imaginary parts of the hypercomplex number, and the redundancy of imaginary parts or the overflow of image channels will occur when the number of imaginary parts does not fit the image structure. The multi-channel moments allow flexible adjustment of the number of channels, but their loose structure will result in the loss of correlation between multiple channels of images. Moreover, the hypercomplex moments and multi-channel moments cannot handle images with multi-dimensional structures such as light-field (LF) images. For the purpose of addressing the above problems, this paper proposes generalized hypercomplex continuous orthogonal moments (HCOMs) and extends them to multi-dimensional hypercomplex continuous orthogonal moments (MHCOMs) in multi-dimensional space. MHCOMs can be used to process images with arbitrary multi-dimensional structures, and when they are used for LF image processing, there is no need to map the LF images to two-dimensional space to change the multi-dimensional structural properties of LF images. By calculating MHCOMs of LF images, a set of multi-dimensional hypercomplex tensors containing robust feature information of LF images can be obtained, and the original LF images can be reconstructed using this information. Furthermore, the MHCOMs-based zero-watermarking scheme is designed for LF images, which has excellent robustness and can achieve lossless copyright protection for LF images.
Endoscopic Image Classification using Block-based Color Feature Descriptors

Compared to biopsy, endoscopy is a less invasive way to screen abnormalities in gastrointestinal (GI) track. In chromoendoscopy, one of the advancements to endoscopy, suspicious regions are made more visible visually by spraying dyes over the mucosal surface. However, identifying and categorizing abnormal regions are challenging tasks. Nowadays, automated systems or computer-aided diagnosis (CAD) systems has been developing. Such systems are helpful for gastroenterologists by pointing out abnormal frame. Extraction of features in endoscopic images plays an important role in every CAD system. In this paper, new color feature descriptors for endoscopic images have been proposed. These descriptors are constituted from Block Difference of Inverse Probabilities (BDIP) and Block Variation of Local Correlation coefficients (BVLC) features extracted on color space. Experimental results show that the method is prospective to investigate more.
AICARE: Health Abnormality Detection of Elderly Automatically Using Deep Learning

Elderly people who are far from relatives or live alone need regular attention to timely detect abnormalities in their health. To solve this problem, technology has been applied to make it easier and more convenient to monitor the health of the elderly remotely. In this work, we proposed a method and system for automatically tracking and detecting behavioral abnormalities of the elderly via surveillance cameras using deep learning. In our methodology, the elderly with various postures such as standing, sitting, lying, chest holding will be detected and tracked when appearing on surveillance cameras in their homes. From postures identified in continuous frames during the tracking process, behavioral abnormalities can be detected and can be seen as signs of health problems such as fall, faint, stroke, heart attack. We experimented on our dataset collected from private surveillance cameras with normal quality. The result showed that our methodology can be used with cheap cameras to provide service for massive homes on elderly health abnormalities tracking and early warning.
Mining Top-K High Utility Itemsets Using Bio-Inspired Algorithms with a Diversity within Population Framework

High-utility itemset mining (HUIM), as a necessary data mining task, has paid the attention of many researchers. It includes numerous applications in various arears. Recently, a method, which improved the memory usage and runtime for HUIs mining, was proposed, is called TKO-BPSO. It helps to automatically increase the border thresholds and might considerably reduce the combinational problem for pruning the search space effectively. However, the idea only works to maintain the current optimal values in the next populations, leading to the variety within populations is limited. To handle this problem, we propose a new bio-inspired algorithm-based HUIM framework to explore HUIs, namely TKO-HUIMF-PSO (Top-K high utility itemset mining in One phase based on a HUIM Framework of Particle Swarm Optimization). The main idea of TKO-HUIMF-PSO adapts the standard roadmap of bio-inspired algorithms by applying roulette wheel selection to all the discovered HUIs to determine the target values of the next population. Consequently, it improves the diversity within populations. Significant experiments conducted on publicly available several real and synthetic datasets delineate that the proposed algorithm is efficient and effective in terms of runtime and memory usage.
Handwritten Digit Recognition using Deep Learning Networks

Handwritten digit recognition has been widely researched. Numerous classification methods have been proposed. This article improves the recognition of handwritten numbers method based on CNN neural network. Recognition of handwritten digits has made great progress, but due to insufficient recognition accuracy, it will have a certain impact on work efficiency. In this paper, a two-layer CNN network with two fully connected layers can achieve a good prediction performance. In our framework, the activation function used is the ReLU function, to some extent, this avoids the difficulties of gradient disappearance and gradient saturation. At the same time, dropout is added to the full connection layer to discard the neurons in the network with a certain probability, which can avoid certain accidents, and the final trained network will be more robust and the generalization performance will be stronger. This method effectively improves the accuracy of recognition in the final experiment, which achieves 98.75% accuracy on theMNIST dataset and 99.25% on the CIFAR-100 dataset.
Deep learning approach to security enforcement in cloud workflow orchestration

Supporting security and data privacy in cloud workflows has attracted significant research attention. For example, private patients' data managed by a workflow deployed on the cloud need to be protected, and communication of such data across multiple stakeholders should also be secured. In general, security threats in cloud environments have been studied extensively. Such threats include data breaches, data loss, denial of service, service rejection, and malicious insiders generated from issues such as multi-tenancy, loss of control over data and trust. Supporting the security of a cloud workflow deployed and executed over a dynamic environment, across different platforms, involving different stakeholders, and dynamic data is a difficult task and is the sole responsibility of cloud providers. Therefore, in this paper, we propose an architecture and a formal model for security enforcement in cloud workflow orchestration. The proposed architecture emphasizes monitoring cloud resources, workflow tasks, and the data to detect and predict anomalies in cloud workflow orchestration using a multi-modal approach that combines deep learning, one class classification, and clustering. It also features an adaptation scheme to cope with anomalies and mitigate their effect on the workflow cloud performance. Our prediction model captures unsupervised static and dynamic features as well as reduces the data dimensionality, which leads to better characterization of various cloud workflow tasks, and thus provides better prediction of potential attacks. We conduct a set of experiments to evaluate the proposed anomaly detection, prediction, and adaptation schemes using a real COVID-19 dataset of patient health records. The results of the training and prediction experiments show high anomaly prediction accuracy in terms of precision, recall, and F1 scores. Other experimental results maintained a high execution performance of the cloud workflow after applying adaptation strategy to respond to some detected anomalies. The experiments demonstrate how the proposed architecture prevents unnecessary wastage of resources due to anomaly detection and prediction.
A Novel Hybrid High-Dimensional PSO Clustering Algorithm Based on the Cloud Model and Entropy

With the increase in the number of high-dimensional data, the characteristic phenomenon of unbalanced distribution is increasingly presented in various big data applications. At the same time, most of the existing clustering and feature selection algorithms are based on maximizing the clustering accuracy. In addition, the hybrid approach can effectively solve the clustering problem of unbalanced data. Aiming at the shortcomings of the unbalanced data clustering algorithm, a hybrid high-dimensional multi-objective PSO clustering algorithm is proposed based on the cloud model and entropy (HHCE-MOPSO). Furthermore, the feasibility of the hybrid PSO is verified by the simulation of the multi-objective test function. The results not only broaden the new theory and method of clustering algorithm for unbalanced data, but also verify the accuracy and feasibility of the hybrid PSO. Furthermore, the clustering analysis method based on information entropy is a new method. As a result, the research results have both important scientific value and good practical significance.
Krill herd optimization algorithm with deep convolutional neural network fostered breast cancer classification using mammogram images

Summary In this paper proposes a Krill Herd Optimization algorithm with Deep Convolutional neural network fostered Breast Cancer Classification using Mammogram Images (BC‐APPDRC‐DCNN‐KHO). Here, the input images are taken from Real time and MAMMOSET datasets. These images are pre‐processed using Altered Phase Preserving Dynamic Range Compression (APPDRC) technique. This APPDRC is applied for preserving local features, compressing dynamic range of images, and enhancing the speckle noise filtering, these are all necessary for better boundary detection. Then, the Pre‐processed images are classified using Deep Convolutional neural network (DCNN). The DCNN weight parameters are optimized based on Krill Herd Optimization algorithm. The Proposed BCC‐DCNN‐KHO‐MI method classifies the input breast cancer imageries into 3 categories: benign, malignant, and normal. The proposed BCC‐DCNN‐KHO‐MI method in Real time dataset attains 18.505%, 19.45%, 16.19%, 17.56% and 16.19% higher accuracy; 15.38%, 12.06%, 12.71%, 26.62% and 18.902% higher Precision; 3.12%, 10.52%, 13.57%, 22.75% and 14.93% higher F‐score, 59.56%, 41.25%, 56.47%, 42.36% and 37.27% lower computation time; 23.87%, 21.87%, 32.87%, 42.76% and 21.05% higher AUC compared with the existing methods, like BCC‐Google Net‐MI, BCC‐Visual Geometry Group Network‐MI, BCC‐Residual Networks‐MI, BC‐RERNN‐LOA‐MI and BC‐CNN‐MI respectively.
On the diffusion of test smells and their relationship with test code quality of Java projects

Abstract Test smells are considered bad practices that can reduce the test code quality, thus harming software testing goals and maintenance activities. Prior studies have investigated the diffusion of test smells and their impact on test code maintainability. However, we cannot directly compare the outcomes of the studies as most of them use customized datasets. In response, we introduced the TSSM (Test Smells and Structural Metrics) dataset, containing test smells detected using the JNose Test tool and structural metrics (test code and production code) calculated with the CK metrics tool of 13,703 open‐source Java systems from GitHub. In addition, we perform an empirical study to investigate the relationship between test smells and structural metrics of test code and the relationship between test smells on a large‐scale dataset. We split the projects into three clusters to analyze the distribution of test smells, the co‐occurrences among test smells, and the correlation of test smells and structural metrics of test code. The ratio of smelly test classes with a specific test smell is similar among the clusters, but we could observe a significant difference in the number of test smells among them. The test smells Sleepy Test , Mystery Guest , and Resource Optimism rarely occur in the three clusters, and the last two are strongly correlated, indicating that those test smells are more severe than others. Our results point out that most test smells have a moderate correlation with high complexity, large size, and coupling of the test code, indicating that they can also negatively affect its quality. To support further studies, we made our dataset publicly available.
The Effect of Vocabulary Mastery and Learning Motivation towards Description Writing Ability

The aim is to determine the effect of vocabulary mastery and motivation to improve descriptive writing skills. Writing must be applied in class according to the right formula and obtained through practice. The accuracy of this learning formulation requires motivation because it will be able to improve the quality of student learning more diligently when they have high motivation. Motivation can always determine the intensity of learning efforts and learning achievement for students. This research was conducted in class XI SMA Negeri in Purwakarta District, Purwakarta Administrative City, West Java Province, for the 2017/2018 academic year. In this area, there are 2 public high schools that will be used as sources of research data, namely SMA Negeri 2 Purwakarta and SMA Negeri 1 Purwakarta. The method used is a survey with a questionnaire on 60 randomly selected students. The results of this activity have a significant effect on the use of learning motivation on students' descriptive text writing skills as evidenced by the value of sig. 0.000 &lt; 0.05. There is a significant influence between vocabulary mastery on students' descriptive text writing skills which is indicated by the value of sig. -0.010 &lt; 0.05. And, there is a significant effect of using learning motivation and vocabulary mastery on students' descriptive text writing skills with a sig value. 0.000 &lt; 0.05.
Implementation of Deep Learning-based Hierarchical Object Detection System for High-Resolution Images

We propose a new deep learning-based hierarchical object detection algorithm for high-definition vision sensors for the goal of controlling autonomous vehicles and drones. The proposed algorithm is implemented based on the YOLOv5 network. Wide area detection using a reduced image and local area detection using the original image are performed hierarchically for a Full-HD image. When combining wide and local area detection, by applying the confidence score extracted by wide area detection to the original image selection, both the detection accuracy and detection numbers are improved compared with the conventional model. In evaluations with the 2K video data set VisDrone, our algorithm achieves two times the number of detections and an improved mean average precision (mAP) of about 3% compared to the conventional method. We also implemented this system on a small, low-cost Jetson Nano board and confirmed that real-time operation is possible at approximately 3 frames per second. Since this hierarchical object detection algorithm supports resolution scaling, it can also be applied to super-high-definition videos such as 4K/8K.
Dialog Breakdown Detection using Multimodal Features for Non-task-oriented Dialog Systems

In this paper, we propose a dialog breakdown detection method for non-task-oriented dialog systems, using multimodal information such as language information, acoustic information, facial expressions, and gaze behavior. Existing automated dialog systems do not always respond appropriately to the utterances of their users, and these systems lack the ability to restore the flow of the dialog by noticing when they have made a serious error. To address this problem, in this study we propose a method of detecting dialog breakdowns using multimodal information collected from users. Data from interactions between human users and the dialog system, such as the content and acoustic features of the user’s speech, as well as their facial expressions and gaze direction, are collected using the‘ Wizard of Oz’ (WOZ) method. We then focus on exchanges where breakdowns in the dialog occurred. Features are extracted from the data for each modality (language, acoustic and visual). After dimension compression using principal component analysis (PCA), dialog breakdown detection is conducted using a deep neural network (DNN). In our evaluation experiment, the proposed method achieved an 85.7% accuracy rate for dialog breakdown detection.