Reviewerly/exHarmony huggingface dataset:
Transformer-Based Visual Grounding with Cross-Modality Interaction

This article tackles the challenging yet important task of Visual Grounding (VG), which aims to localize a visual region in the given image referred by a natural language query. Existing efforts on the VG task are twofold: (1) two-stage methods first extract region proposals and then rank them according to their similarities with the referring expression, which usually leads to suboptimal results due to the quality of region proposals; (2) one-stage methods usually predict all the possible coordinates of the target region online by leveraging modern object detection architectures, which pay little attention to cross-modality correlations and have limited generalization ability. To better address the task, we present an effective transformer-based end-to-end visual grounding approach, which focuses on capturing the cross-modality correlations between the referring expression and visual regions for accurately reasoning the location of the target region. Specifically, our model consists of a feature encoder, a cross-modality interactor, and a modality-agnostic decoder. The feature encoder is employed to capture the intra-modality correlation, which models the linguistic context in query and the spatial dependency in image respectively. The cross-modality interactor endows the model with the capability of highlighting the localization-relevant visual and textual cues by mutual verification of vision and language, which plays a key role in our model. The decoder learns a consolidated token representation enriched by multi-modal contexts and further directly predicts the box coordinates. Extensive experiments on five public benchmark datasets with quantitative and qualitative analysis clearly demonstrate the effectiveness and rationale of our proposed method.
An Accelerated Double-Integral ZNN with Resisting Linear Noise for Dynamic Sylvester Equation Solving and Its Application to the Control of the SFM Chaotic System

The dynamic Sylvester equation (DSE) is frequently encountered in engineering and mathematics fields. The original zeroing neural network (OZNN) can work well to handle DSE under a noise-free environment, but may not work in noise. Though an integral-enhanced zeroing neural network (IEZNN) can be employed to solve the DSE under multiple-noise, it may fall flat under linear noise, and its convergence speed is unsatisfactory. Therefore, an accelerated double-integral zeroing neural network (ADIZNN) is proposed based on an innovative design formula to resist linear noise and accelerate convergence. Besides, theoretical proofs verify the convergence and robustness of the ADIZNN model. Moreover, simulation experiments indicate that the convergence rate and anti-noise ability of the ADIZNN are far superior to the OZNN and IEZNN under linear noise. Finally, chaos control of the sine function memristor (SFM) chaotic system is provided to suggest that the controller based on the ADIZNN has a smaller amount of error and higher accuracy than other ZNNs.
Automated Arabic Long-Tweet Classification Using Transfer Learning with BERT

Social media platforms like Twitter are commonly used by people interested in various activities, interests, and subjects that may cover their everyday activities and plans, as well as their thoughts on religion, technology, or the products they use. In this paper, we present bidirectional encoder representations from transformers (BERT)-based text classification model, ARABERT4TWC, for classifying the Arabic tweets of users into different categories. This work aims to provide an enhanced deep-learning model that can automatically classify the robust Arabic tweets of different users. In our proposed work, a transformer-based model for text classification is constructed from a pre-trained BERT model provided by the hugging face transformer library with custom dense layers. The multi-class classification layer is built on top of the BERT encoder to categorize the tweets. First, data sanitation and preprocessing were performed on the raw Arabic corpus to improve the model’s accuracy. Second, an Arabic-specific BERT model was built and input embedding vectors were fed into it. Using five publicly accessible datasets, substantial experiments were executed, and the fine-tuning technique was assessed in terms of tokenized vector and learning rate. In addition, we assessed the accuracy of various deep-learning models for classifying Arabic text.
UAV Abnormal State Detection Model Based on Timestamp Slice and Multi-Separable CNN

With the rapid development of UAVs (Unmanned Aerial Vehicles), abnormal state detection has become a critical technology to ensure the flight safety of UAVs. The position and orientation system (POS) data, etc., used to evaluate UAV flight status are from different sensors. The traditional abnormal state detection model ignores the difference of POS data in the frequency domain during feature learning, which leads to the loss of key feature information and limits the further improvement of detection performance. To deal with this and improve UAV flight safety, this paper presents a method for detecting the abnormal state of a UAV based on a timestamp slice and multi-separable convolutional neural network (TS-MSCNN). Firstly, TS-MSCNN divides the POS data reasonably in the time domain by setting a set of specific timestamps and then extracts and fuses the key features to avoid the loss of feature information. Secondly, TS-MSCNN converts these feature data into grayscale images by data reconstruction. Lastly, TS-MSCNN utilizes a multi-separable convolution neural network (MSCNN) to learn key features more effectively. The binary and multi-classification experiments conducted on the real flight data, Air Lab Fault and Anomaly (ALFA), demonstrate that the TS-MSCNN outperforms traditional machine learning (ML) and the latest deep learning methods in terms of accuracy.
Multi-view clustering via matrix factorization assisted k-means

Matrix factorization based multi-view clustering algorithms has attracted much attention in recent years due to the strong interpretability and efficient implementation. In general, these approaches firstly compute the coefficient matrices of each data views, and learn a consensus matrix simultaneously. By applying the classical clustering techniques, such as k-means, on the consensus matrix, the final partition can be easily obtained. However, most of previous models work in a “step-by-step” manner, which cannot perform multi-view matrix factorization and clustering label generation simultaneously, leading to degenerated performance. In this paper, we propose a novel “one-pass” method, which integrates matrix factorization and k-means into a unified framework, named multi-view clustering via matrix factorization assisted k-means (MFK). In MFK, the generation of cluster indicator matrix and coefficient matrix learning can boost each other, leading to final improved clustering performance. Furthermore, we adopt a graph Laplacian regularization on the indicator matrix in order to capture the intrinsic geometric structure of original data. An alternating optimization strategy is designed to solve the resultant optimization problem and extensive experiments on six publicly datasets are conducted to demonstrate the superiority and effectiveness of the proposed MFK model.
Deep convolutional neural networks using an active learning strategy for cervical cancer screening and diagnosis

Cervical cancer (CC) is the fourth most common malignant tumor among women worldwide. Constructing a high-accuracy deep convolutional neural network (DCNN) for cervical cancer screening and diagnosis is important for the successful prevention of cervical cancer. In this work, we proposed a robust DCNN for cervical cancer screening using whole-slide images (WSI) of ThinPrep cytologic test (TCT) slides from 211 cervical cancer and 189 normal patients. We used an active learning strategy to improve the efficiency and accuracy of image labeling. The sensitivity, specificity, and accuracy of the best model were 96.21%, 98.95%, and 97.5% for CC patient identification respectively. Our results also demonstrated that the active learning strategy was superior to the traditional supervised learning strategy in cost reduction and improvement of image labeling quality. The related data and source code are freely available at https://github.com/hqyone/cancer_rcnn.
Strategic academic leadership and high-tech economic growth

Considering the context of the Fourth Industrial Revolution, higher education must play a significant role in the social-investment model of economic growth. To what extent higher education supports strategic academic leadership and high-tech economic development is still being determined. The article proposes direct actions for improving university management through higher educational and technological functions. We establish the specific directions for university management in neo-industrialization 4.0 to ensure extensive accessibility of higher education and enhance university management efficiency.
Spatiotemporal adaptive attention 3D multiobject tracking for autonomous driving

Three-dimensional (3D) multiobject tracking (MOT) is an essential perception task for autonomous vehicles (AVs). Studies have indicated that multimodal data fusion can provide more stable and efficient perception information to AVs than a single sensor. Therefore, this paper proposes a new spatiotemporal adaptive attention 3D (3DSTAA) tracker, which attempts to improve the tracking performance of the end-to-end 3D MOT by adaptively correlating spatiotemporal data. The novelty of this paper includes the following. (1) Different from nonintelligent fusion methods, this paper uses an efficiently adaptive spatial-guided fusion (SGFus) module for multimodal feature fusion. As a result, the 3D structural information obtained from point cloud data can provide additional spatial information as complementary information to the 2D texture information extracted from the image data, collaboratively facilitating and refining the perception information representation in the margin area. (2) This paper develops a spatiotemporal object-unique attention (STOUA) module that calculates the relational degree of each perceived object between two adjacent frames through attentional encoding. At the same time, an adaptive weighting strategy is used to further study the spatiotemporal correlation of unique objects, reducing the similarity among various objects and the differences across the same object. Experiments tested using the KITTI tracking benchmark show that the 3DSTAA tracker is highly competitive in both inference time and tracking performance compared with state-of-the-art (SOTA) methods. Our corresponding code will be released on the https://github.com/xf-zh.
Metanalysis: A connected-wall video installation based on the movement of traditional Chinese ink painting

The authors present the visual display installation as the basis for creating the artwork series “Metanalysis”. The work explores the visual algorithmic representation of traditional Chinese ink painting and provides a video installation designed to showcase this visualization project. Exploring these visual representations is an attempt to understand and modernize expressions in the developing digital age visually. These visual displays are rendered in real-time and produced one after another in a continuous overlap and accumulation of black ink lines to reflect the specific random and malleable characteristics of the ink. They were eventually using mathematical algorithms to generate synthetic images that mimic the process of staining ink. The value of this paper lies in the fact that it proposes a new path for the expression of traditional Chinese ink painting and aims to bring the nonvitality classic art back to the stage through digital forms and to provide contemporary artists with new ideas to promote traditional culture.
Stumbling Blocks and Alternative Paths: Reconsidering the Walkthrough Method for Analyzing Apps

The walkthrough method was developed as a way to trace an app or platform’s technological mechanisms and cultural references to understand how it guides users. This article explores the method’s enduring strengths and emergent weaknesses regarding technological advances and developments in app studies. It engages with adjacent methods for understanding apps’ intensifying structural and economic complexity, datafication, algorithmic logic, and personalization as well as approaches fostering a feminist ethics of care toward users. Considering these perspectives, the article discusses challenges encountered in teaching the method and applying it to algorithmically driven apps. With TikTok as a central example, examining the walkthrough process demonstrates the method’s incongruence for investigating several aspects of the app, especially its automated personalization. These challenges highlight the need to combine, supplement, or exchange the method with other approaches as part of an expanding and flexible toolkit of methods in app studies.
A Theoretical Framework for End-to-End Learning of Deep Neural Networks With Applications to Robotics

Deep Learning (DL) systems are difficult to analyze and proving convergence of DL algorithms like backpropagation is an extremely challenging task as it is a highly non-convex and high-dimensional problem. When using DL algorithms in robotic systems, theoretical analysis of stability, convergence, and robustness is a vital procedure as robots need to operate in a predictable manner to ensure safety. This paper presents the first unified End-to-End (E2E) learning framework that can be applied to both classification problems and real-time kinematic robot control tasks. In the proposed forward simultaneous learning method, the weights of all layers of the fully connected neural networks are updated concurrently. The proposed E2E learning framework uses an adjustable ReLU activation function so that convergence of the output error to a bound, which is dependent on the neural network approximation error, can be ensured. In particular, it is shown that the error between ideal output and estimated output of the network can converge to and stay in a certain bound, and this bound reduces to zero when the approximation error is zero. Therefore, the robustness of the learning system can be ensured even in the presence of the approximation error. Two case studies are done on classification tasks using MNIST and CIFAR10 datasets by using the proposed learning method, and the results show that the E2E learning method achieves comparable test accuracy as the gradient descent method and the main advantage is that convergence can be ensured during training. The framework is also implemented on a UR5e robot with unknown kinematic model, which is the first result on real-time E2E learning of deep neural networks for kinematic control of robots with guaranteed convergence.
PCSF: Privacy-Preserving Content-Based Spam Filter

The purpose of privacy-preserving spam filtering is to inspect email while preserving the privacy of its detection rules and the email content. Although many solutions have emerged, they suffer from the following: 1) The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">privacy</i> provided is insufficient as the email content or detection rules may be exposed to third parties; 2) Due to improper use of encryption, exhaustive word search attacks are possible, potentially breaking the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">confidentiality</i> of encrypted emails; 3) When spam filtering is outsourced, email is given to the outsource, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">user privacy may be compromised</i> if privacy protection measures are not properly put in place; 4) Confirmation of whether the encrypted email is spam is only determined <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">after</i> the receiver receives the email, which can lead to a situation in which spam is loaded to the memory of the receiver’s terminal for spam filtering. This can be harmful, for example, when an attacker inserts a web browser vulnerability into the body of an email to lure users to a phishing site simply by reading the email; 5) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Computationally expensive operations</i> are unavoidable to provide required features of privacy-preserving spam filtering. We present Privacy-preserving Content-based Spam Filter (PCSF), which is a spam filter system that does not suffer from the aforementioned issues. Additionally, our system provides <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pre-validation</i> before the receiver reads the email. We provide an implementation of our system based on the Naive Bayes spam filter and prove its security.
Enhanced Vehicle Re-Identification for Smart City Applications Using Zone Specific Surveillance

Vehicle re-identification is an important feature of an intelligent transportation system as part of a smart city application. Vehicle re-identification aims at matching vehicles from images acquired by surveillance cameras at different locations. During rush hours, vehicles are densely occupied across regions such as entry/exit of gated campuses, railways, airports, educational institutions, etc. Due to this uneven flow of traffic, there is a possibility of violation of traffic rules by the vehicles that lead to a security breach. In such scenarios to speed up the re-identification process, it is justified to look into a specific group of surveillance cameras to detect and re-identify vehicles on day to day basis in near real-time. However, the existing vehicle re-identification datasets do not contain zone specific information and therefore can not be used to evaluate the performance of re-identification algorithms in different zones. In the proposed work for re-identification, a framework is developed that performs vehicle re-identification across a group of cameras that monitors varying traffic movements over an area. These areas defined as “strategic zones” comprise a subset of non-overlapping cameras that are installed to monitor non-uniform vehicle movements. The re-identification framework is evaluated on a novel dataset developed to understand the performance of vehicle re-identification across strategic zones. The dataset consists of videos of vehicles captured through 20 CCTV surveillance cameras that are grouped into four different zones. Various experiments are conducted to study the performance of re-identification across four zones using a deep neural network with triplet loss, L2 regularization, and re-ranking. The experiments conducted with an image dimension of 224 × 224 have demonstrated an overall mAP of 77.22%. Also, for each of the four zones a mAP of 82.16%, 69.1%, 66.5%, and 75.76% is achieved. The experimental results demonstrate huge variations in the accuracy of vehicle re-identification method across different zones. Therefore, the study assess the possible measures that can be taken to improve the performance in individual zones for an accurate vehicle re-identification in intelligent transport system.
Toward Realistic 3D Human Motion Prediction With a Spatio-Temporal Cross- Transformer Approach

Human motion prediction intends to predict how humans move given a historical sequence of 3D human motions. Recent transformer-based methods have attracted increasing attentions and demonstrated their promising performance in 3D human motion prediction. However, existing methods generally decompose the input of human motion information into spatial and temporal branches in a separate way and seldom consider their inherent coherence between the two branches, hence often failing to register the dynamic spatio-temporal information during the training process. Motivated by these issues, we propose a spatio-temporal cross-transformer network (STCT) for 3D human motion predictions. Specifically, we investigate various types of interaction methods (i.e., Concatenation Interaction, Msg token interaction, and Cross-transformer) to capture the coherence of the spatial and temporal branches. According to the obtained results, the proposed cross-transformer interaction method shows its superiority over other methods. Meanwhile, considering that most existing works treat the human body as a set of 3D human joint positions, the predicted human joints are proportionally less appropriate to the realistic human body due to unreasonable bone length and non-plausible poses as time progresses. We further resort to the bone constraints of human mesh to produce more realistic human motions. By fitting a parametric body model (i.e., SMPL-X model) to the predicted human joints, a reconstruction loss function is proposed to remedy the unreasonable bone length and pose errors. Comprehensive experiments on AMASS and Human3.6M datasets have demonstrated that our method achieves superior performance over compared methods.
Video captioning: A comparative review of where we are and which could be the route

Video captioning is the process of describing the content of a sequence of images capturing its semantic relationships and meanings. Dealing with this task with a single image is arduous, not to mention how difficult it is for a video (or image sequence). The amount and relevance of the applications of video captioning are vast, mainly to deal with a significant amount of video recordings in video surveillance, or assisting people visually impaired, to mention a few. To analyze where the efforts of our community to solve the video captioning task are, as well as what route could be better to follow, this manuscript presents an extensive review of more than 142 papers for the period of 2016 to 2022. As a result, the most-used datasets and metrics are identified and described. Also, the main approaches used and the best ones are analyzed and discussed. Furthermore, we compute a set of rankings based on several performance metrics to obtain, according to its reported performance, the best method with the best result on the video captioning task across of several datasets and metrics. Finally, some insights are concluded about which could be the next steps or opportunity areas to improve dealing with this complex task.
Multi-UAV Path Planning in GPS and Communication Denial Environment

This paper proposes a feature fusion algorithm for solving the path planning problem of multiple unmanned aerial vehicles (UAVs) using GPS and communication denial conditions. Due to the blockage of GPS and communication, UAVs cannot obtain the precise position of a target, which leads to the failure of path planning algorithms. This paper proposes a feature fusion proximal policy optimization (FF-PPO) algorithm based on deep reinforcement learning (DRL); the algorithm can fuse image recognition information with the original image, realizing the multi-UAV path planning algorithm without an accurate target location. In addition, the FF-PPO algorithm adopts an independent policy for multi-UAV communication denial environments, which enables the distributed control of UAVs such that multi-UAVs can realize the cooperative path planning task without communication. The success rate of our proposed algorithm can reach more than 90% in the multi-UAV cooperative path planning task. Finally, the feasibility of the algorithm is verified by simulations and hardware.