Reviewerly/exHarmony huggingface dataset:
Model-Agnostic Multi-Agent Perception Framework

Existing multi-agent perception systems assume that every agent utilizes the same model with identical parameters and architecture. The performance can be degraded with different perception models due to the mismatch in their confidence scores. In this work, we propose a model-agnostic multi-agent perception framework to reduce the negative effect caused by the model discrepancies without sharing the model information. Specifically, we propose a confidence calibrator that can eliminate the prediction confidence score bias. Each agent performs such calibration independently on a standard public database to protect intellectual property. We also propose a corresponding bounding box aggregation algorithm that considers the confidence scores and the spatial agreement of neighboring boxes. Our experiments shed light on the necessity of model calibration across different agents, and the results show that the proposed framework improves the baseline 3D object detection performance of heterogeneous agents. The code can be found at this url.
UDepth: Fast Monocular Depth Estimation for Visually-guided Underwater Robots

In this paper, we present a fast monocular depth estimation method for enabling 3D perception capabilities of low-cost underwater robots. We formulate a novel end-to-end deep visual learning pipeline named UDepth, which incorporates domain knowledge of image formation characteristics of natural underwater scenes. First, we adapt a new input space from raw RGB image space by exploiting underwater light attenuation prior, and then devise a least-squared formulation for coarse pixel-wise depth prediction. Subsequently, we extend this into a domain projection loss that guides the end-to-end learning of UDepth on over 9K RGB-D training samples. UDepth is designed with a computationally light MobileNetV2 backbone and a Transformer-based optimizer for ensuring fast inference rates on embedded systems. By domain-aware design choices and through comprehensive experimental analyses, we demonstrate that it is possible to achieve state-of-the-art depth estimation performance while ensuring a small computational footprint. Specifically, with 70 % −80 % less network parameters than existing benchmarks, UDepth achieves comparable and often better depth estimation performance. While the full model offers over 66 FPS (13 FPS) inference rates on a single GPU (CPU core), our domain projection for coarse depth prediction runs at 51.5 FPS rates on single-board Jetson TX2s. The inference pipelines are available at https://github.com/uf-robopi/UDepth.
Surgical-VQLA:Transformer with Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery

Despite the availability of computer-aided simulators and recorded videos of surgical procedures, junior residents still heavily rely on experts to answer their queries. However, expert surgeons are often overloaded with clinical and academic workloads and limit their time in answering. For this purpose, we develop a surgical question-answering system to facilitate robot-assisted surgical scene and activity understanding from recorded videos. Most of the existing visual question answering (VQA) methods require an object detector and regions based feature extractor to extract visual features and fuse them with the embedded text of the question for answer generation. However, (i) surgical object detection model is scarce due to smaller datasets and lack of bounding box annotation; (ii) current fusion strategy of heterogeneous modalities like text and image is naive; (iii) the localized answering is missing, which is crucial in complex surgical scenarios. In this paper, we propose Visual Question Localized-Answering in Robotic Surgery (Surgical-VQLA) to localize the specific surgical area during the answer prediction. To deal with the fusion of the heterogeneous modalities, we design gated vision-language embedding (GVLE) to build input patches for the Language Vision Transformer (LViT) to predict the answer. To get localization, we add the detection head in parallel with the prediction head of the LViT. We also integrate generalized intersection over union (GIoU) loss to boost localization performance by preserving the accuracy of the question-answering model. We annotate two datasets of VQLA by utilizing publicly available surgical videos from EndoVis-17 and 18 of the MICCAI challenges. Our validation results suggest that Surgical-VQLA can better understand the surgical scene and localized the specific area related to the question-answering. GVLE presents an efficient language-vision embedding technique by showing superior performance over the existing benchmarks.
Resumate: A Prototype to Enhance Recruitment Process with NLP based Resume Parsing

The process of reviewing resumes takes time. The unstructured written language can be understood and parsed by natural language processing and machine learning, which can then extract the required information. The goal is to teach the computer to analyze written papers similarly to a human. For assessing and analyzing the unstructured data, various methods including named entity recognition, tokenization, text classification, and approaches of other natural language processing techniques have been explored. A user uploads a resume into the program, which takes it and converts it into a standard text format before parsing it for the necessary information and organizing the extracted data in a standard defined format. After extraction and preprocessing, a predictive model is trained is better and categorize resumes for use in various applications. The k-fold cross validation with k=5 outcome in this case reveals that SVM(0.58 - 0.65) and XG Boost(0.65 - 0.73) have higher ranges of training performance than all other models. Here, the XG Boost outperforms previous trained models, accurately classifying the newly created resume into the target category. The proposed work is novel and presents one of its kind methodology.
Scene Text Segmentation via Multi-Task Cascade Transformer With Paired Data Synthesis

The scene text segmentation task provides a wide range of practical applications. However, the number of images in the available datasets for scene text segmentation is not large enough to effectively train deep learning-based models, leading to limited performance. To solve this problem, we employ paired data generation to secure sufficient data samples for text segmentation via Text Image-conditional GANs. Furthermore, existing models implicitly model text attributes such as size, layout, font, and structure, which hinders their performance. To remedy this, we propose a Multi-task Cascade Transformer network that explicitly learns these attributes using large volumes of generated synthetic data. The transformer-based network includes two auxiliary tasks and one main task for text segmentation. The auxiliary tasks help the network learn text regions to focus on, as well as the structure of the text through different words and fonts, to support the main task. To bridge the gap between different datasets, we train the proposed network on paired synthetic data before fine-tuning it on real data. Our experiments on publicly available scene text segmentation datasets show that our method outperforms existing methods.
Sparse R-CNN: An End-to-End Framework for Object Detection

Object detection serves as one of most fundamental computer vision tasks. Existing works on object detection heavily rely on dense object candidates, such as k anchor boxes pre-defined on all grids of an image feature map of size H×W. In this paper, we present Sparse R-CNN, a very simple and sparse method for object detection in images. In our method, a fixed sparse set of learned object proposals ( N in total) are provided to the object recognition head to perform classification and localization. By replacing HWk (up to hundreds of thousands) hand-designed object candidates with N (e.g., 100) learnable proposals, Sparse R-CNN makes all efforts related to object candidates design and one-to-many label assignment completely obsolete. More importantly, Sparse R-CNN directly outputs predictions without the non-maximum suppression (NMS) post-processing procedure. Thus, it establishes an end-to-end object detection framework. Sparse R-CNN demonstrates highly competitive accuracy, run-time and training convergence performance with the well-established detector baselines on the challenging COCO dataset and CrowdHuman dataset. We hope that our work can inspire re-thinking the convention of dense prior in object detectors and designing new high-performance detectors.
Real-time underwater polarization imaging without relying on background

Polarization imaging has excellent potential for use in underwater application scenarios due to efficient elimination of backscattered light. In most studies, two images that contain the maximum and minimum visible backscatter are obtained by finding the brightest and darkest moments during the rotation of the polarizer. In addition, the background region must be included in the image. To prevent these limitations, this study directly generates the optimal image pair via the Stokes vector. These images are then transformed into the frequency domain, and the global distribution of the degree of polarization of backscattered light is obtained using low-pass filtering. Recovery of the clear scene requires neither relying on background nor a search for the brightest and darkest images, significantly enhancing the real-time characteristics of the imaging process. Because of this, imaging of moving targets comes true. Additionally, experimental results at continuously varying distances verify the effectiveness of the proposed method for targets with different polarization properties. Especially, the imaging distance can be increased by 30% compared to the conventional active polarization imaging model at the same level of contrast.
Lexicon annotation in sentiment analysis for dialectal Arabic: Systematic review of current trends and future directions

Due to the vast volumes of newly streamed data on the Internet and social media, the use of sentiment analysis (SA) to extract information and analyze people's opinions has become a trendy topic. Yet, the majority of research are attributed to the English language, despite the fact that other languages, such as Arabic, are among the most popular on the Internet. Considering the availability of numerous dialects of this language and how their data were annotated and processed, the absence of research in this field is evident. Understanding these initiatives merits a great deal of attention in Arabic SA research. To the best of our knowledge, this domain has not been considered before, and thus the aim of this study is to perform a systematic review with regard to SA and data annotations for Arabic dialects published between 2015 and 2023. The outcomes of this research offer a refined taxonomy of data annotation methods classified into three categories: (1) manual, (2) automatic, and (3) hybrid methods. In addition, a discussion of the research challenges, motivations, and recommendations is presented with detailed taxonomy analysis of current research trends, and from this, we identify new research gaps and propose new research implications and future directions that will encourage more scholars to contribute to Arabic SA research, facilitate more successful multilingual SA applications, and provide insights regarding Arabic SA in different contexts.
A hierarchical multi-leadership sine cosine algorithm to dissolving global optimization and data classification: The COVID-19 case study

The Sine Cosine Algorithm (SCA) is an outstanding optimizer that is appreciably used to dissolve complicated real-world problems. Nevertheless, this algorithm lacks sufficient population diversification and a sufficient balance between exploration and exploitation. So, effective techniques are required to tackle the SCA's fundamental shortcomings. Accordingly, the present paper suggests an improved version of SCA called Hierarchical Multi-Leadership SCA (HMLSCA) which uses an effective hierarchical multi-leadership search mechanism to lead the search process on multiple paths. The efficiency of the HMLSCA has been appraised and compared with a set of famous metaheuristic algorithms to dissolve the classical eighteen benchmark functions and thirty CEC 2017 test suites. The results demonstrate that the HMLSCA outperforms all compared algorithms and that the proposed algorithm provided a promising efficiency. Moreover, the HMLSCA was applied to handle the medicine data classification by optimizing the support vector machine's (SVM) parameters and feature weighting in eight datasets. The experiential outcomes verify the productivity of the HMLSCA with the highest classification accuracy and a gain scoring 1.00 Friedman mean rank versus the other evaluated metaheuristic algorithms. Furthermore, the proposed algorithm was used to diagnose COVID-19, in which it attained the topmost accuracy of 98% in diagnosing the infection on the COVID-19 dataset, which proves the performance of the proposed search strategy.
Target tracking algorithm based on occlusion prediction

Tracking failure caused by occlusion is one of the most challenging problems in visual tracking tasks. The current Siamese network-based object tracking algorithms use the maximum confidence strategy and center cropping mechanism, which may lead to partial or complete loss of target tracking. To address this problem, we propose an occlusion prediction-based tracking method, which uses an occlusion prediction branch to evaluate the occlusion degree of the target in four directions: up, down, left and right, and combines information fusion to predict the target position, and uses dynamic background suppression to reduce the interference of similar targets. First, we use dynamic background suppression to preprocess the input image, weaken the interference around the target, then use the target location information and regression information output by the Siamese network to obtain the candidate target position, and finally use the occlusion prediction information for information fusion to obtain the final target position. Although our method brings additional computational burden, it can still achieve real-time performance. Experiments show that our proposed method can effectively improve the robustness of Siamese tracking algorithms under different occlusion conditions. In the interference test based on OTB100, our method improves the tracking success rate by about 7% and the precision by about 10% for SiamBan and SiamGat algorithms respectively; in the interference test based on VOT2018, our method can effectively reduce the tracking loss frequency of Siamban and SiamGat algorithms, improving the EAO(Expected Average Overlap) by about 10%.
Cross-directional consistency network with adaptive layer normalization for multi-spectral vehicle re-identification and a high-quality benchmark

To tackle the challenge of vehicle re-identification (Re-ID) in complex lighting environments and diverse scenes, multi-spectral sources like visible and infrared information are taken into consideration due to their excellent complementary advantages. However, multi-spectral vehicle Re-ID suffers cross-modality discrepancy caused by heterogeneous properties of different modalities as well as a big challenge of the diverse appearance with different views in each identity. Meanwhile, diverse environmental interference leads to heavy sample distributional discrepancy in each modality. In this work, we propose a novel cross-directional consistency network (CCNet) to simultaneously overcome the discrepancies from both modality and sample aspects. In particular, we design a new cross-directional center loss (Lcdc) to pull the modality centers of each identity close to mitigate cross-modality discrepancy, while the sample centers of each identity close to alleviate the sample discrepancy. Such a strategy can generate discriminative multi-spectral feature representations for vehicle Re-ID. In addition, we design an adaptive layer normalization unit (ALNU) to dynamically adjust individual feature distribution to handle distributional discrepancy of intra-modality features for robust learning. To provide a comprehensive evaluation platform, we create a high-quality RGB-NIR-TIR multi-spectral vehicle Re-ID benchmark (MSVR310), including 310 different vehicles from a broad range of viewpoints, time spans and environmental complexities. Comprehensive experiments on both created and public datasets demonstrate the effectiveness of the proposed approach comparing to the state-of-the-art methods. The dataset and code will be released for free academic usage at https://github.com/superlollipop123/Cross-directional-Center-Network-and-MSVR310.
A transformative solution for construction safety: Blockchain-based system for accident information management

Effective management of accident information is a crucial component of safety management within the construction industry, as it reflects the safety performance of the company and allows them to identify the root causes of accidents and prevent similar accidents in the future. However, existing safety information systems provide self-owned, isolated, and centralized environments and fail to present a secure, transparent, and trustworthy platform for monitoring and management of accident information. To address these issues, this paper presents a novel decentralized blockchain-based system for accident/incident information management of construction projects. The proposed system leverages the benefits and advantages of blockchain, smart contracts, and decentralized IPFS storage to address the security transparency, tampering, and trustworthiness issues of the conventional approaches. The proposed system is simulated by using real-world construction accident data to demonstrate how blockchain technology can provide a novel solution to assure security, transparency, authenticity, availability, and immutability of the accident/incident data for improving safety management.
Optimizing the information extracted by a single qubit measurement

We consider a quantum computation that only extracts one bit of information per $N$-qubit quantum state preparation. This is relevant for error mitigation schemes where the remainder of the system is measured to detect errors. We optimize the estimation of the expectation value of an operator by its linear decomposition into bitwise-measurable terms. We prove that optimal decompositions must be in terms of reflections with eigenvalues $\ifmmode\pm\else\textpm\fi{}1$. We find the optimal reflection decomposition of a fast-forwardable operator, and show a numerical improvement over a simple Pauli decomposition by a factor ${N}^{0.7}$.
Unsupervised feature extraction of aerial images for clustering and understanding hazardous road segments

Aerial image data are becoming more widely available, and analysis techniques based on supervised learning are advancing their use in a wide variety of remote sensing contexts. However, supervised learning requires training datasets which are not always available or easy to construct with aerial imagery. In this respect, unsupervised machine learning techniques present important advantages. This work presents a novel pipeline to demonstrate how available aerial imagery can be used to better the provision of services related to the built environment, using the case study of road traffic collisions (RTCs) across three cities in the UK. In this paper, we show how aerial imagery can be leveraged to extract latent features of the built environment from the purely visual representation of top-down images. With these latent image features in hand to represent the urban structure, this work then demonstrates how hazardous road segments can be clustered to provide a data-augmented aid for road safety experts to enhance their nuanced understanding of how and where different types of RTCs occur.
Supervised Gradual Machine Learning for Aspect-Term Sentiment Analysis

Abstract Recent work has shown that Aspect-Term Sentiment Analysis (ATSA) can be effectively performed by Gradual Machine Learning (GML). However, the performance of the current unsupervised solution is limited by inaccurate and insufficient knowledge conveyance. In this paper, we propose a supervised GML approach for ATSA, which can effectively exploit labeled training data to improve knowledge conveyance. It leverages binary polarity relations between instances, which can be either similar or opposite, to enable supervised knowledge conveyance. Besides the explicit polarity relations indicated by discourse structures, it also separately supervises a polarity classification DNN and a binary Siamese network to extract implicit polarity relations. The proposed approach fulfills knowledge conveyance by modeling detected relations as binary features in a factor graph. Our extensive experiments on real benchmark data show that it achieves the state-of-the-art performance across all the test workloads. Our work demonstrates clearly that, in collaboration with DNN for feature extraction, GML outperforms pure DNN solutions.
CASSOCK: Viable Backdoor Attacks against DNN in the Wall of Source-Specific Backdoor Defenses

As a critical threat to deep neural networks (DNNs), backdoor attacks can be categorized into two types, i.e., source-agnostic backdoor attacks (SABAs) and source-specific backdoor attacks (SSBAs). Compared to traditional SABAs, SSBAs are more advanced in that they have superior stealthier in bypassing mainstream countermeasures that are effective against SABAs. Nonetheless, existing SSBAs suffer from two major limitations. First, they can hardly achieve a good trade-off between ASR (attack success rate) and FPR (false positive rate). Besides, they can be effectively detected by the state-of-the-art (SOTA) countermeasures (e.g., SCAn [40]).
Unraveling the Effects of Voice Assistant Interactions on Digital Engagement: The Moderating Role of Adult Playfulness

The definition and dimensions of interaction are changing as artificial intelligence (AI) technologies advance. Drawing on the mind perception theory (MPT), the authors conducted an online survey to investigate the influential mechanism of the interactions with smart voice assistants (SVAs) on digital engagement via competence and warmth in the smart Internet of Things era. Firstly, they conceptualized and operationalized the seven sub-dimensions of voice assistant (VA) interaction, investigated their effects on competence and warmth perception of SVAs’ mind perceptions, and explored the psychological mechanisms of VA interaction on digital engagement in SVAs. Secondly, it was one of the first studies to unravel the moderating effects of adult playfulness in the field of VA interaction. The results showed that: (1) VA interaction is a multidimensional construct, and VA interactions positively affect competence perception; (2) personalization, responsiveness, humanness, and communication positively impact warmth perception; (3) VA interactions positively impact digital engagement via competence and warmth perception; (4) adult playfulness moderates the effects of VA interactions on digital engagement. The theoretical and practical implications were discussed.