Reviewerly/exHarmony huggingface dataset:
Efficient Visual Tracking with Exemplar Transformers

The design of more complex and powerful neural network models has significantly advanced the state-of-the-art in visual object tracking.These advances can be attributed to deeper networks, or the introduction of new building blocks, such as transformers.However, in the pursuit of increased tracking performance, runtime is often hindered.Furthermore, efficient tracking architectures have received surprisingly little attention.In this paper, we introduce the Exemplar Transformer, a transformer module utilizing a single instance level attention layer for realtime visual object tracking.E.T.Track, our visual tracker that incorporates Exemplar Transformer modules, runs at 47 FPS on a CPU.This is up to 8× faster than other transformer-based models.When compared to lightweight trackers that can operate in realtime on standard CPUs, E.T.Track consistently outperforms all other methods on the LaSOT [16], OTB-100 [52], NFS [27], TrackingNet [36], and VOT-ST2020 [29] datasets.Code and models are available at https://github.com/pblatter/ettrack.
Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution

Super-resolving medical images can help physicians in providing more accurate diagnostics. In many situations, computed tomography (CT) or magnetic resonance imaging (MRI) techniques capture several scans (modes) during a single investigation, which can jointly be used (in a multimodal fashion) to further boost the quality of super-resolution results. To this end, we propose a novel multi-modal multi-head convolutional attention module to super-resolve CT and MRI scans. Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple concatenated input tensors, where the kernel (receptive field) size controls the reduction rate of the spatial attention, and the number of convolutional filters controls the reduction rate of the channel attention, respectively. We introduce multiple attention heads, each head having a distinct receptive field size corresponding to a particular reduction rate for the spatial attention. We integrate our multimodal multi-head convolutional attention (MMHCA) into two deep neural architectures for super-resolution and conduct experiments on three data sets. Our empirical results show the superiority of our attention module over the state-of-the-art attention mechanisms used in super-resolution. Moreover, we conduct an ablation study to assess the impact of the components involved in our attention module, e.g. the number of inputs or the number of heads. Our code is freely available at https://github.com/lilygeorgescu/MHCA.
Deep Learning Enabled Semantic Communications With Speech Recognition and Synthesis

In this paper, we develop a deep learning based semantic communication system for speech transmission, named DeepSC-ST. We take the speech recognition and speech synthesis as the transmission tasks of the communication system, respectively. First, the speech recognition-related semantic features are extracted for transmission by a joint semantic-channel encoder and the text is recovered at the receiver based on the received semantic features, which significantly reduces the required amount of data transmission without performance degradation. Then, we perform speech synthesis at the receiver, which dedicates to re-generate the speech signals by feeding the recognized text and the speaker information into a neural network module. To enable the DeepSC-ST adaptive to dynamic channel environments, we identify a robust model to cope with different channel conditions. According to the simulation results, the proposed DeepSC-ST significantly outperforms conventional communication systems and existing DL-enabled communication systems, especially in the low signal-to-noise ratio (SNR) regime. A software demonstration is further developed as a proof-of-concept of the DeepSC-ST.
The Impact of Human Resources Agility on Job Performance in—Islamic Banks Operating in Jordan

The study aimed to identify the impact of human resource agility on job performance in Islamic banks operating in Jordan. The study population consists of all managers in the upper management, middle management, and lower management in these banks. The study adopted the stratified random sample that is suitable for the managers in the upper, middle management and the lower management, who are (148) respondents. To achieve the goals of the study and test its hypotheses, the researcher used the descriptive analytical approach, and collected data with a questionnaire used by the researcher as a main performance of collecting information. Smart PLS program was used to test study hypotheses. The study results showed a statistically significant effect of human resource agility on job performance in Islamic banks operating in Jordan. Based on the results of the study, the study recommends assisting decision makers in solving problems, among which was to enhance interest in practicing adaptive capacity, creativity in problem solving, professional flexibility, and learning skills.
Efficient Visual Tracking Based on Fuzzy Inference for Intelligent Transportation Systems

Remote monitoring is an important application of intelligent transportation systems (ITSs). The combination of monitoring equipment and tracking algorithms can be used to automatically track moving targets. The tracking algorithm based on the Siamese network is both accurate and efficient, and its development potential is better than that of other algorithms. Its output is a detection map that reflects the probability that any position in the search area is the center of the target’s bounding box, and the maximum value of the detection map is the center of the target’s bounding box predicted by the algorithm. Owing to partial occlusion, target deformation, out-of-view, and background clutter, local maxima in the detection map may also be the center of the target’s bounding box. A tracker’s ability to make accurate judgments is currently limited. Furthermore, previous trackers extracted only the target features in the initial frame as the matching template. Although this matching template is highly reliable, it cannot effectively combine the target features available in the subsequent frames. Therefore, in this study, fuzzy inference is introduced into the tracking process to analyze the reliability of the detection map. When this map is reliable, the target feature of the search area is transformed into a substitute template; otherwise, multiple substitute templates are selected from the template pool for parallel matching as per the set rules. The optimal result is selected from multiple detection results, based on the priority of the detection results when the initial frame is used as the matching template. Experimental results on multiple datasets show that the proposed algorithm is superior to other similar algorithms in terms of multiple assessment metrics and can improve the robustness of remote monitoring tasks in ITSs.
Financial Crimes in Web3-Empowered Metaverse: Taxonomy, Countermeasures, and Opportunities

At present, the concept of metaverse has sparked widespread attention from the public to major industries. With the rapid development of blockchain and Web3 technologies, the decentralized metaverse ecology has attracted a large influx of users and capital. Due to the lack of industry standards and regulatory rules, the Web3-empowered metaverse ecosystem has witnessed a variety of financial crimes, such as scams, code exploit, wash trading, money laundering, and illegal services and shops. To this end, it is especially urgent and critical to summarize and classify the financial security threats on the Web3-empowered metaverse in order to maintain the long-term healthy development of its ecology. In this paper, we first outline the background, foundation, and applications of the Web3 metaverse. Then, we provide a comprehensive overview and taxonomy of the security risks and financial crimes that have emerged since the development of the decentralized metaverse. For each financial crime, we focus on three issues: a) existing definitions, b) relevant cases and analysis, and c) existing academic research on this type of crime. Next, from the perspective of academic research and government policy, we summarize the current anti-crime measurements and technologies in the metaverse. Finally, we discuss the opportunities and challenges in behavioral mining and the potential regulation of financial activities in the metaverse. The overview of this paper is expected to help readers better understand the potential security threats in this emerging ecology, and to provide insights and references for financial crime fighting.
Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model

Seniors who live alone at home are at risk of falling and injuring themselves and, thus, may need a mobile robot that monitors and recognizes their poses automatically. Even though deep learning methods are actively evolving in this area, they have limitations in estimating poses that are absent or rare in training datasets. For a lightweight approach, an off-the-shelf 2D pose estimation method, a more sophisticated humanoid model, and a fast optimization method are combined to estimate joint angles for 3D pose estimation. As a novel idea, the depth ambiguity problem of 3D pose estimation is solved by adding a loss function deviation of the center of mass from the center of the supporting feet and penalty functions concerning appropriate joint angle rotation range. To verify the proposed pose estimation method, six daily poses were estimated with a mean joint coordinate difference of 0.097 m and an average angle difference per joint of 10.017 degrees. In addition, to confirm practicality, videos of exercise activities and a scene of a person falling were filmed, and the joint angle trajectories were produced as the 3D estimation results. The optimized execution time per frame was measured at 0.033 s on a single-board computer (SBC) without GPU, showing the feasibility of the proposed method as a real-time system.
Vision Transformers in Image Restoration: A Survey

The Vision Transformer (ViT) architecture has been remarkably successful in image restoration. For a while, Convolutional Neural Networks (CNN) predominated in most computer vision tasks. Now, both CNN and ViT are efficient approaches that demonstrate powerful capabilities to restore a better version of an image given in a low-quality format. In this study, the efficiency of ViT in image restoration is studied extensively. The ViT architectures are classified for every task of image restoration. Seven image restoration tasks are considered: Image Super-Resolution, Image Denoising, General Image Enhancement, JPEG Compression Artifact Reduction, Image Deblurring, Removing Adverse Weather Conditions, and Image Dehazing. The outcomes, the advantages, the limitations, and the possible areas for future research are detailed. Overall, it is noted that incorporating ViT in the new architectures for image restoration is becoming a rule. This is due to some advantages compared to CNN, such as better efficiency, especially when more data are fed to the network, robustness in feature extraction, and a better feature learning approach that sees better the variances and characteristics of the input. Nevertheless, some drawbacks exist, such as the need for more data to show the benefits of ViT over CNN, the increased computational cost due to the complexity of the self-attention block, a more challenging training process, and the lack of interpretability. These drawbacks represent the future research direction that should be targeted to increase the efficiency of ViT in the image restoration domain.
Evaluating explainability for graph neural networks

As explanations are increasingly used to understand the behavior of graph neural networks (GNNs), evaluating the quality and reliability of GNN explanations is crucial. However, assessing the quality of GNN explanations is challenging as existing graph datasets have no or unreliable ground-truth explanations. Here, we introduce a synthetic graph data generator, SHAPEGGEN, which can generate a variety of benchmark datasets (e.g., varying graph sizes, degree distributions, homophilic vs. heterophilic graphs) accompanied by ground-truth explanations. The flexibility to generate diverse synthetic datasets and corresponding ground-truth explanations allows SHAPEGGEN to mimic the data in various real-world areas. We include SHAPEGGEN and several real-world graph datasets in a graph explainability library, GRAPHXAI. In addition to synthetic and real-world graph datasets with ground-truth explanations, GRAPHXAI provides data loaders, data processing functions, visualizers, GNN model implementations, and evaluation metrics to benchmark GNN explainability methods.
TPH-YOLOv5++: Boosting Object Detection on Drone-Captured Scenarios with Cross-Layer Asymmetric Transformer

Object detection in drone-captured images is a popular task in recent years. As drones always navigate at different altitudes, the object scale varies considerably, which burdens the optimization of models. Moreover, high-speed and low-altitude flight cause motion blur on densely packed objects, which leads to great challenges. To solve the two issues mentioned above, based on YOLOv5, we add an additional prediction head to detect tiny-scale objects and replace CNN-based prediction heads with transformer prediction heads (TPH), constructing the TPH-YOLOv5 model. TPH-YOLOv5++ is proposed to significantly reduce the computational cost and improve the detection speed of TPH-YOLOv5. In TPH-YOLOv5++, cross-layer asymmetric transformer (CA-Trans) is designed to replace the additional prediction head while maintain the knowledge of this head. By using a sparse local attention (SLA) module, the asymmetric information between the additional head and other heads can be captured efficiently, enriching the features of other heads. In the VisDrone Challenge 2021, TPH-YOLOv5 won 4th place and achieved well-matched results with the 1st place model (AP 39.43%). Based on the TPH-YOLOv5 and CA-Trans module, TPH-YOLOv5++ can further increase efficiency while achieving comparable and better results.
Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey

Deep reinforcement learning has produced many success stories in recent years. Some example fields in which these successes have taken place include mathematics, games, health care, and robotics. In this paper, we are especially interested in multi-agent deep reinforcement learning, where multiple agents present in the environment not only learn from their own experiences but also from each other and its applications in multi-robot systems. In many real-world scenarios, one robot might not be enough to complete the given task on its own, and, therefore, we might need to deploy multiple robots who work together towards a common global objective of finishing the task. Although multi-agent deep reinforcement learning and its applications in multi-robot systems are of tremendous significance from theoretical and applied standpoints, the latest survey in this domain dates to 2004 albeit for traditional learning applications as deep reinforcement learning was not invented. We classify the reviewed papers in our survey primarily based on their multi-robot applications. Our survey also discusses a few challenges that the current research in this domain faces and provides a potential list of future applications involving multi-robot systems that can benefit from advances in multi-agent deep reinforcement learning.
A lightweight ship target detection model based on improved YOLOv5s algorithm

Real-time and accurate detection of ships plays a vital role in ensuring navigation safety and ship supervision. Aiming at the problems of large parameters, large computation quantity, poor real-time performance, and high requirements for memory and computing power of the current ship detection model, this paper proposes a ship target detection algorithm MC-YOLOv5s based on YOLOv5s. First, the MobileNetV3-Small lightweight network is used to replace the original feature extraction backbone network of YOLOv5s to improve the detection speed of the algorithm. And then, a more efficient CNeB is designed based on the ConvNeXt-Block module of the ConvNeXt network to replace the original feature fusion module of YOLOv5s, which improves the spatial interaction ability of feature information and further reduces the complexity of the model. The experimental results obtained from the training and verification of the MC-YOLOv5s algorithm show that, compared with the original YOLOv5s algorithm, MC-YOLOv5s reduces the number of parameters by 6.98 MB and increases the mAP by about 3.4%. Even compared with other lightweight detection models, the improved model proposed in this paper still has better detection performance. The MC-YOLOv5s has been verified in the ship visual inspection and has great application potential. The code and models are publicly available at https://github.com/sakura994479727/datas.
A novel hybrid arithmetic optimization algorithm for solving constrained optimization problems

The present study aims to optimize the engineering design and manufacturing problems with a novel hybrid optimizer named: AOA-NM (Arithmetic optimization-Nelder mead). To overcome the local optima trap shortcoming and improve the solution quality of a recently introduced arithmetic optimization algorithm (AOA), the Nelder–Mead local search methodology has been incorporated into the basic AOA framework. The objective of the proposed hybridization approach was to facilitate the refinement of the exploration–exploitation behaviour of the AOA search. In the numerical validation stage, numerous multidimensional benchmarks from the CEC2020 were used as challenging testing functions to investigate the suggested AOA-NM optimizer. To investigate the viability of the proposed hybridized algorithm in real-world applications, it is investigated for ten constrained engineering design problems, and the performance was contrasted with other distinguished metaheuristics extracted from the literature. Additionally, a hands-on manufacturing problem of milling process parameter optimization and vehicle structure shape optimization is posed and solved at the forefront to evaluate both AOA and AOA-NM efficacy. The proficiency of the AOA-NM algorithm, in terms of both solution quality and stability, is confirmed by performed comparative analysis and found to be robust in handling challenging practical issues.
Multi-Scale Ship Detection Algorithm Based on YOLOv7 for Complex Scene SAR Images

Recently, deep learning techniques have been extensively used to detect ships in synthetic aperture radar (SAR) images. The majority of modern algorithms can achieve successful ship detection outcomes when working with multiple-scale ships on a large sea surface. However, there are still issues, such as missed detection and incorrect identification when performing multi-scale ship object detection operations in SAR images of complex scenes. To solve these problems, this paper proposes a complex scenes multi-scale ship detection model, according to YOLOv7, called CSD-YOLO. First, this paper suggests an SAS-FPN module that combines atrous spatial pyramid pooling and shuffle attention, allowing the model to focus on important information and ignore irrelevant information, reduce the feature loss of small ships, and simultaneously fuse the feature maps of ship targets on various SAR image scales, thereby improving detection accuracy and the model’s capacity to detect objects at several scales. The model’s optimization is then improved with the aid of the SIoU loss function. Finally, thorough tests on the HRSID and SSDD datasets are presented to support our methodology. CSD-YOLO achieves better detection performance than the baseline YOLOv7, with a 98.01% detection accuracy, a 96.18% recall, and a mean average precision (mAP) of 98.60% on SSDD. In addition, in comparative experiments with other deep learning-based methods, in terms of overall performance, CSD-YOLO still performs better.
Detection of non‐suicidal self‐injury based on spatiotemporal features of indoor activities

Abstract Non‐suicide self‐injury (NSSI) can be dangerous and difficult for guardians or caregivers to detect in time. NSSI refers to when people hurt themselves even though they have no wish to cause critical or long‐lasting hurt. To timely identify and effectively prevent NSSI in order to reduce the suicide rates of patients with a potential suicide risk, the detection of NSSI based on the spatiotemporal features of indoor activities is proposed. Firstly, an NSSI behaviour dataset is provided, and it includes four categories that can be used for scientific research on NSSI evaluation. Secondly, an NSSI detection algorithm based on the spatiotemporal features of indoor activities (NssiDetection) is proposed. NssiDetection calculates the human bounding box by using an object detection model and employs a behaviour detection model to extract the temporal and spatial features of NSSI behaviour. Thirdly, the optimal combination schemes of NssiDetection is investigated by checking its performance with different behaviour detection methods and training strategies. Lastly, a case study is performed by implementing an NSSI behaviour detection prototype system. The prototype system has a recognition accuracy of 84.18% for NSSI actions with new backgrounds, persons, or camera angles.
Collaborative structure and feature learning for multi-view clustering

Multi-view clustering divides similar objects into the same class through using the fused multiview information. Most multi-view clustering methods obtain clustering result by only analyzing structure relationship among samples, ignoring the analysis of intrinsic features of each sample, while a few methods operate the original feature on the corresponding high-dimensional kernel matrices. However, noisy and redundant features of samples are inevitably mixed in original multi-view data or high-dimensional kernel matrices. To address this problem, we propose a novel multiview clustering method, which unifies structure learning and feature learning to a framework. Specifically, we obtain a consensus structure information from multiple views via sparse subspace structure learning with weight tensor nuclear norm constraint. Then our feature learning seeks projection directions to obtain data representation by data pseudo labels, which are obtained via the fused consensus structure information. Furthermore, we use the manifold regularization term to establish the relationship between data structure information and learnt data presentation. At last, the two subtasks are alternately iterated and optimized to acquire accurate structure and discriminative data presentation. Experimental results on different datasets validate the proposed method is superior to the state-of-the-art methods.