Reviewerly/exHarmony huggingface dataset:
Hybrid Representation and Decision Fusion towards Visual-textual Sentiment

The rising use of online media has changed social customs of the public. Users have become gradually accustomed to sharing daily experiences and publishing personal opinions on social networks. Social data carrying with emotions and attitudes have provided significant decision support for numerous tasks in sentiment analysis. Conventional sentiment analysis methods only concern about textual modality and are vulnerable to the multimodal scenario, while common multimodal approaches only focus on the interactive relationship between modalities without considering unique intra-modal information. A hybrid fusion network is proposed in this work to capture both the inter-modal and intra-modal features. First, in the intermediate fusion stage, a multi-head visual attention is proposed to extract accurate semantic and sentimental information from textual embedding representations with the assistance of visual features. Then, multiple base classifiers are trained to learn independent and diverse discriminative information from different modal representations in the late fusion stage. The final decision is determined based on fusing the decision supports from base classifiers via a decision fusion method. To improve the generalization of our hybrid fusion network, a similarity loss is employed to inject decision diversity into the whole model. Empirical results on multimodal datasets have demonstrated the proposed model achieves a higher accuracy and better generalization compared with baselines for multimodal sentiment analysis.
Machine learning and deep learning approaches in IoT

The internet is a booming sector for exchanging information because of all the gadgets in today's world. Attacks on Internet of Things (IoT) devices are alarming as these devices evolve. The two primary areas of the IoT that should be secure in terms of authentication, authorization, and data privacy are the IoMT (Internet of Medical Things) and the IoV (Internet of Vehicles). IoMT and IoV devices monitor real-time healthcare and traffic trends to protect an individual's life. With the proliferation of these devices comes a rise in security assaults and threats, necessitating the deployment of an IPS (intrusion prevention system) for these systems. As a result, machine learning and deep learning technologies are utilized to identify and control security in IoMT and IoV devices. This research study aims to investigate the research fields of current IoT security research trends. Papers about the domain were searched, and the top 50 papers were selected. In addition, research objectives are specified concerning the problem, which leads to research questions. After evaluating the associated research, data is retrieved from digital archives. Furthermore, based on the findings of this SLR, a taxonomy of IoT subdomains has been given. This article also identifies the difficult areas and suggests ideas for further research in the IoT.
Visual Sorting of Express Packages Based on the Multi-Dimensional Fusion Method under Complex Logistics Sorting

Visual sorting of express packages is faced with many problems such as the various types, complex status, and the changeable detection environment, resulting in low sorting efficiency. In order to improve the sorting efficiency of packages under complex logistics sorting, a multi-dimensional fusion method (MDFM) for visual sorting in actual complex scenes is proposed. In MDFM, the Mask R-CNN is designed and applied to detect and recognize different kinds of express packages in complex scenes. Combined with the boundary information of 2D instance segmentation from Mask R-CNN, the 3D point cloud data of grasping surface is accurately filtered and fitted to determining the optimal grasping position and sorting vector. The images of box, bag, and envelope, which are the most common types of express packages in logistics transportation, are collected and the dataset is made. The experiments with Mask R-CNN and robot sorting were carried out. The results show that Mask R-CNN achieves better results in object detection and instance segmentation on the express packages, and the robot sorting success rate by the MDFM reaches 97.2%, improving 2.9, 7.5, and 8.0 percentage points, respectively, compared to baseline methods. The MDFM is suitable for complex and diverse actual logistics sorting scenes, and improves the efficiency of logistics sorting, which has great application value.
Experience Replay Optimisation via ATSC and TSC for Performance Stability in Deep RL

Catastrophic forgetting is a significant challenge in deep reinforcement learning (RL). To address this problem, researchers introduce the experience replay (ER) concept to complement the training of a deep RL agent. However, the buffer size, experience selection, and experience retention strategies adopted for the ER can negatively affect the agent’s performance stability, especially for complex continuous state action problems. This paper investigates how to address the stability problem using an enhanced ER method that combines a replay policy network, a dual memory, and an alternating transition selection control (ATSC) mechanism. Two frameworks were designed: an experience replay optimisation via alternating transition selection control (ERO-ATSC) without a transition storage control (TSC) and an ERO-ATSC with a TSC. The first is a hybrid of experience replay optimisation (ERO) and dual-memory experience replay (DER) and the second, which has two versions of its kind, integrates a transition storage control (TSC) into the first framework. After comprehensive experimental evaluations of the frameworks on the pendulum-v0 environment and across multiple buffer sizes, retention strategies, and sampling ratios, the reward version of ERO-ATSC with a TSC exhibits superior performance over the first framework and other novel methods, such as the deep deterministic policy gradient (DDPG) and ERO.
Kcr-FLAT: A Chinese-Named Entity Recognition Model with Enhanced Semantic Information

The performance of Chinese-named entity recognition (NER) has improved via word enhancement or new frameworks that incorporate various types of external data. However, for Chinese NER, syntactic composition (in sentence level) and inner regularity (in character-level) have rarely been studied. Chinese characters are highly sensitive to sentential syntactic data. The same Chinese character sequence can be decomposed into different combinations of words according to how they are used and placed in the context. In addition, the same type of entities usually have the same naming rules due to the specificity of the Chinese language structure. This paper presents a Kcr-FLAT to improve the performance of Chinese NER with enhanced semantic information. Specifically, we first extract different types of syntactic data, functionalize the syntactic information by a key-value memory network (KVMN), and fuse them by attention mechanism. Then the syntactic information and lexical information are integrated by a cross-transformer. Finally, we use an inner regularity perception module to capture the internal regularity of each entity for better entity type prediction. The experimental results show that with F1 scores as the evaluation index, the proposed model obtains 96.51%, 96.81%, and 70.12% accuracy rates on MSRA, resume, and Weibo datasets, respectively.
A Novel Swarm Intelligence Algorithm with a Parasitism-Relation-Based Structure for Mobile Robot Path Planning

A multi-swarm-evolutionary structure based on the parasitic relationship in the biosphere is proposed in this paper and, according to the conception, the Para-PSO-ABC algorithm (ParaPA), combined with merits of the modified particle swarm optimization (MPSO) and artificial bee colony algorithm (ABC), is conducted with the multimodal routing strategy to enhance the safety and the cost issue for the mobile robot path planning problem. The evolution is divided into three stages, where the first is the independent evolutionary stage, with the same evolution strategies for each swarm. The second is the fusion stage, in which individuals are evolved hierarchically in the parasitism structure. Finally, in the interaction stage, a multi-swarm-elite strategy is used to filter the information through a predefined cross function among swarms. Meanwhile, the segment obstacle-avoiding strategy is proposed to accelerate the searching speed with two fitness functions. The best path is selected according to the performance on the safety and consumption issues. The introduced algorithm is examined with different obstacle allocations and simulated in the real routing environment compared with some typical algorithms. The results verify the productiveness of the parasitism-relation-based structure and the stage-based evolution strategy in path planning.
System of six-level atom interacting with a quantized field in the existence of time-varying coupling

In the present manuscript, we extend the Jaynes-Cummings model in the framework of six-level atom (LA) system in the presence of a quantized field describes by an ordinary coherent state or a superposition state. We investigate the dynamics of the population, coherence and quantum entanglement in terms of the system parameters. We analyze the time variation of the quantifiers in the presence and absence of time-varying coupling of the six-LA–field system and make comparison among the quantifiers. Furthermore, we show how the quantifiers can be controlled according to the values of the system parameters. The obtained results illustrate that the proposed system provides interesting values of the quantum coherence and entanglement which can be considered as good candidates for executing quantum optics and information schemes.
Attitudes About Artificially Intelligent Interactive Voice Response Systems Using Amazon Alexa in Cardiovascular Clinics: Insights from the VOICE-COVID-19 Study

The acceptability of artificially intelligent interactive voice response (AI-IVR) systems in cardiovascular research settings is unclear. As a result, we evaluated peoples' attitudes regarding the Amazon Echo Show 8 device when used for electronic data capture in cardiovascular clinics. Participants were recruited following the Voice-Based Screening for SARS-CoV-2 Exposure in Cardiovascular clinics study. Overall, 215 people enrolled and underwent screening (mean age 46.1; 55% females) in the VOICE-COVID study and 58 people consented to participate in a post-screening survey. Following thematic analysis, four key themes affecting AI-IVR acceptability were identified. These were difficulties with communication (44.8%), limitations with available interaction modalities (41.4%), barriers with the development of therapeutic relationships (25.9%), and concerns with universality and accessibility (8.6%). While there are potential concerns with the use of AI-IVR technologies, these systems appeared to be well accepted in cardiovascular clinics. Increased development of these technologies could significantly improve healthcare access and efficiency.
Mission planning on preference-based expression trees using heuristics-assisted evolutionary computation

The mission planning problem has so far been solved by using temporal logic and classic planning approaches that have an exponential computational complexity and do not always account for the transition costs. Variants of the Vehicle Routing Problem can use meta-heuristics to give a near-optimal solution; however, have a limited representation capability. In this paper the problem of giving complex instructions (mission) to a robot in the form of an expression tree with AND, OR and THEN operators is studied. This allows users to instruct the robots for everyday tasks like getting a variety of items (modeled by AND) with a few choices (modeled by OR), the picks having temporal constraints (modeled by THEN). Preferences can be added to make the robot solve a sub-mission earlier than another sub-mission. The paper first proposes an algorithm to make a greedy solution by a single parse of the expression tree. Thereafter, the solution is optimized by using a Genetic Algorithm that optimizes the branch to be taken by the OR operator and the ordering of operations. Experimental results are demonstrated on the Pioneer LX robot programmed using the Robot Operating System. The experimental results further show that the proposed approach significantly beats a Genetic Algorithm solution and a Dynamic Programming greedy solution for problem sizes that cannot be solved by using the model verification and exhaustive search techniques.
Bringing Generalization to Deep Multi-View Pedestrian Detection

Multi-View Detection (MVD) is highly effective for occlusion reasoning in a crowded environment. While recent works using deep learning have made significant ad-vances in the field, they have overlooked the generalization aspect, which makes them impractical for real-world deployment. The key novelty of our work is to formalize three critical forms of generalization and propose experiments to evaluate them: generalization with i) a varying number of cameras, ii) varying camera positions, and fi-nally, iii) to new scenes. We find that existing state-of-the-art models show poor generalization by overfitting to a single scene and camera configuration. To address the concerns: (a) we propose a novel Generalized MVD (GMVD) dataset, assimilating diverse scenes with changing daytime, camera configurations, and a varying number of cameras, and (b) we discuss the properties essential to bring gener-alization to MVD and propose a barebones model incorpo-rating them. We present comprehensive set of experiments on WildTrack, MultiViewX and the GMVD datasets to moti-vate the necessity to evaluate the generalization abilities of MVD methods and to demonstrate the efficacy of the proposed approach. The code and dataset are available at https://github.com/jeetv/GMVD.
DepthCue: Restoration of Underwater Images Using Monocular Depth as a Clue

In this paper, we perform restoration of underwater images by considering principles of the image formation model in deep neural networks. Typically, underwater images suffer from blur, color loss and other degradations due to the scattering and absorption of light in water as a medium. Quality of restoration is sensitive to depth as scattering and absorption of light increases with depth and introduces a considerable amount of degradation. However, from literature we infer, recent restoration frameworks do not consider the influence of depth on restoration of under-water images. Towards this, we propose to consider depth as a clue for restoration considering relative distance of objects in the scene. We introduce depth with different scales as a clue for learning restoration and term the proposed architecture as DepthCue. We foresee to facilitate the restoration by eliminating the effect of degradations like lost color, blur and noise. We demonstrate our results on benchmark datasets and compare with the state-of-the-art restoration techniques using various quality metrics.
The Monocular Depth Estimation Challenge

This paper summarizes the results of the first Monocular Depth Estimation Challenge (MDEC) organized at WACV2023. This challenge evaluated the progress of self-supervised monocular depth estimation on the challenging SYNS-Patches dataset. The challenge was organized on CodaLab and received submissions from 4 valid teams. Participants were provided a devkit containing updated reference implementations for 16 State-of-the-Art algorithms and 4 novel techniques. The threshold for acceptance for novel techniques was to outperform every one of the 16 SotA baselines. All participants outperformed the baseline in traditional metrics such as MAE or AbsRel. However, pointcloud reconstruction metrics were challenging to improve upon. We found predictions were characterized by interpolation artefacts at object boundaries and errors in relative object positioning. We hope this challenge is a valuable contribution to the community and encourage authors to participate in future editions.
An Efficient Approach for Underwater Image Improvement: Deblurring, Dehazing, and Color Correction

As remotely operated underwater vehicles (ROV) and static underwater video and image collection platforms become more prevalent, there is a significant need for effective ways to increase the quality of underwater images at faster than real-time speeds. To this end, we present a novel state-of-the-art end-to-end deep learning architecture for underwater image enhancement focused on solving key image degradations related to blur, haze, and color casts and inference efficiency. Our proposed architecture builds from a minimal encoder-decoder structure to address these main underwater image degradations while maintaining efficiency. We use the discrete wavelet transform skip connections and channel attention modules to address haze and color corrections while preserving model efficiency. Our minimal architecture operates at 40 frames per second while scoring a structural similarity index (SSIM) of 0.8703 on the underwater image enhancement benchmark (UIEDB) dataset. These results show our method to be twice as fast as the previous state-of-the-art. We also present a variation of our proposed method with a second parallel deblurring branch for even more significant image improvement, which achieves an improved SSIM of 0.8802 while operating more efficiently than almost all comparable methods. The source code is available at https://github.com/alejorico98/underwater_ddc
Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysis

As tools for content editing mature, and artificial intelligence (AI) based algorithms for synthesizing media grow, the presence of manipulated content across online media is increasing. This phenomenon causes the spread of misinformation, creating a greater need to distinguish between “real” and “manipulated” content. To this end, we present Videosham, a dataset consisting of 826 videos (413 real and 413 manipulated). Many of the existing deepfake datasets focus exclusively on two types of facial manipulations-swapping with a different subject's face or altering the existing face. Videosham, on the other hand, contains more diverse, context-rich, and human-centric, high-resolution videos manipulated using a combination of 6 different spatial and temporal attacks. Our analysis shows that state-of-the-art manipulation detection algorithms only work for a few specific attacks and do not scale well on Videosham. We performed a user study on Amazon Mechanical Turk with 1200 participants to understand if they can differentiate between the real and manipulated videos in Videosham. Finally, we dig deeper into the strengths and weaknesses of performances by humans and SOTA-algorithms to identify gaps that need to be filled with better AI algorithms. We present the dataset here <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> VideoSham dataset link..
Discriminative Sampling of Proposals in Self-Supervised Transformers for Weakly Supervised Object Localization

Drones are employed in a growing number of visual recognition applications. A recent development in cell tower inspection is drone-based asset surveillance, where the autonomous flight of a drone is guided by localizing objects of interest in successive aerial images. In this paper, we propose a method to train deep weakly-supervised object localization (WSOL) models based only on image-class labels to locate object with high confidence. To train our localizer, pseudo labels are efficiently harvested from a self-supervised vision transformers (SSTs). However, since SSTs decompose the scene into multiple maps containing various object parts, and do not rely on any explicit super-visory signal, they cannot distinguish between the object of interest and other objects, as required WSOL. To address this issue, we propose leveraging the multiple maps generated by the different transformer heads to acquire pseudo-labels for training a deep WSOL model. In particular, a new Discriminative Proposals Sampling (DiPS) method is introduced that relies on a CNN classifier to identify discriminative regions. Then, foreground and background pixels are sampled from these regions in order to train a WSOL model for generating activation maps that can accurately localize objects belonging to a specific class. Empirical results <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Our code is available: https://github.com/shakeebmurtaza/dips on the challenging TelDrone dataset indicate that our proposed approach can outperform state-of-art methods over a wide range of threshold values over produced maps. We also computed results on CUB dataset, showing that our method can be adapted for other tasks.
Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Dataset

Face recognition technology has advanced significantly in recent years due largely to the availability of large and increasingly complex training datasets for use in deep learning models. These datasets, however, typically comprise images scraped from news sites or social media plat-forms and, therefore, have limited utility in more advanced security, forensics, and military applications. These applications require lower resolution, longer ranges, and ele-vated viewpoints. To meet these critical needs, we collected and curated the first and second subsets of a large multi-modal biometric dataset designed for use in the research and development (R&D) of biometric recognition technolo-gies under extremely challenging conditions. Thus far, the dataset includes more than 350,000 still images and over 1,300 hours of video footage of approximately 1,000 sub-jects. To collect this data, we used Nikon DSLR cameras, a variety of commercial surveillance cameras, specialized long-rage R&D cameras, and Group 1 and Group 2 UAV platforms. The goal is to support the development of algorithms capable of accurately recognizing people at ranges up to 1,000 m and from high angles of elevation. These ad-vances will include improvements to the state of the art in face recognition and will support new research in the area of whole-body recognition using methods based on gait and anthropometry. This paper describes methods used to col-lect and curate the dataset, and the dataset's characteristics at the current stage.
A Principal Component Analysis-Based Approach for Single Morphing Attack Detection

This paper proposes an explicit method for single face image morphing attack detection, using an RGB decomposition based on Principal Component Analysis from texture patterns. Handcrafted detection algorithms can be advantageous over deep learning-based methods as they constitute increased explainability, showcased in this work by visualizing relevant face areas for morphing attack detection. Such information can be relevant for deployed systems in real-world scenarios with humans in the loop. The morphing detection capability of the proposed method is evaluated extensively across three datasets and six morphing algorithms in single, cross-dataset and cross-morphed scenarios and compared to a fine-tuned MobileNetV2 architecture. The results show how single image morphing attack detection remains challenging, especially in cross-domain scenarios involving realistic diversity of morphing algorithms, including StyleGAN-based approaches. In such conditions, the proposed method can be as good or even better than the evaluated MobileNetV2 approach.