Reviewerly/exHarmony huggingface dataset:
Towards explainable deep visual saliency models

Deep neural networks have shown their profound impact on achieving human-level performance in visual saliency prediction. However, it is still unclear how they learn their task and what it means in terms of understanding human visual system. In this work, we propose a framework to derive explainable saliency models from their corresponding deep architectures. Mainly, we explain a deep saliency model by understanding its four different aspects: (1) intermediate activation maps of deep layers, (2) biologically plausible Log-Gabor (LG) filters for salient region identification, (3) positional biased behavior of Log-Gabor filters and (4) processing of color information by establishing a relevance with human visual system. We consider four state-of-the-art (SOTA) deep saliency models, namely CMRNet, UNISAL, DeepGaze IIE, and MSI-Net for their interpretation using our proposed framework. We observe that explainable models perform way better than the classical SOTA models. We also find that CMRNet transforms the input RGB space to a representation after the input layer, which is very close to YUV space of a color image. Then, we discuss about the biological consideration and relevance of our framework for its possible anatomical substratum of visual attention. We find a good correlation between components of HVS and the base operations of the proposed technique. Hence, we say that this generic explainable framework provides a new perspective to see relationship between classical methods/human visual system and DNN based ones.
Automating Method Naming with Context-Aware Prompt-Tuning

Method names are crucial to program comprehension and maintenance. Recently, many approaches have been proposed to automatically recommend method names and detect inconsistent names. Despite promising, their results are still suboptimal considering the three following drawbacks: 1) These models are mostly trained from scratch, learning two different objectives simultaneously. The misalignment between two objectives will negatively affect training efficiency and model performance. 2) The enclosing class context is not fully exploited, making it difficult to learn the abstract functionality of the method. 3) Current method name consistency checking methods follow a generate-then-compare process, which restricts the accuracy as they highly rely on the quality of generated names and face difficulty measuring the semantic consistency.In this paper, we propose an approach named AUMENA to AUtomate MEthod NAming tasks with context-aware prompt-tuning. Unlike existing deep learning based approaches, our model first learns the contextualized representation(i.e., class attributes) of programming language and natural language through the pre-training model, then fully exploits the capacity and knowledge of large language model with prompt-tuning to precisely detect inconsistent method names and recommend more accurate names. To better identify semantically consistent names, we model the method name consistency checking task as a two-class classification problem, avoiding the limitation of previous generate-then-compare consistency checking approaches. Experiment results reflect that AUMENA scores 68.6%, 72.0%, 73.6%, 84.7% on four datasets of method name recommendation, surpassing the state-of-the-art baseline by 8.5%, 18.4%, 11.0%, 12.0%, respectively. And our approach scores 80.8% accuracy on method name consistency checking, reaching an 5.5% outperformance. All data and trained models are publicly available.
Robust table structure recognition with dynamic queries enhanced detection transformer

We present a new table structure recognition (TSR) approach, called TSRFormer, to robustly recognize the structures of complex tables with geometrical distortions from various table images. Unlike previous methods, we formulate table separation line prediction as a line regression problem instead of an image segmentation problem and propose a new two-stage dynamic queries enhanced DETR based separation line regression approach, named DQ-DETR, to predict separation lines from table images directly. Compared to Vallina DETR, we propose three improvements in DQ-DETR to make the two-stage DETR framework work efficiently and effectively for the separation line prediction task: 1) A new query design, named Dynamic Query, to decouple single line query into separable point queries which could intuitively improve the localization accuracy for regression tasks; 2) A dynamic queries based progressive line regression approach to progressively regressing points on the line which further enhances localization accuracy for distorted tables; 3) A prior-enhanced matching strategy to solve the slow convergence issue of DETR. After separation line prediction, a simple relation network based cell merging module is used to recover spanning cells. With these new techniques, our TSRFormer achieves state-of-the-art performance on several benchmark datasets, including SciTSR, PubTabNet, WTW, FinTabNet, and cTDaR TrackB2-Modern. Furthermore, we have validated the robustness and high localization accuracy of our approach to tables with complex structures, borderless cells, large blank spaces, empty or spanning cells as well as distorted or even curved shapes on a more challenging real-world in-house dataset.
Cooperative Perception With Learning-Based V2V Communications

Cooperative perception has been widely used in autonomous driving to alleviate the inherent limitation of single automated vehicle perception. To enable cooperation, vehicle-to-vehicle (V2V) communication plays an indispensable role. This work analyzes the performance of cooperative perception accounting for communications channel impairments. Different fusion methods and channel impairments are evaluated. A new late fusion scheme is proposed to leverage the robustness of intermediate features. In order to compress the data size incurred by cooperation, a convolution neural network-based autoencoder is adopted. Numerical results demonstrate that intermediate fusion is more robust to channel impairments than early fusion and late fusion, when the SNR is greater than 0 dB. Also, the proposed fusion scheme outperforms the conventional late fusion using detection outputs, and autoencoder provides a good compromise between detection accuracy and bandwidth usage.
DeepArc: Modularizing Neural Networks for the Model Maintenance

Neural networks are an emerging data-driven programming paradigm widely used in many areas. Unlike traditional software systems consisting of decomposable modules, a neural network is usually delivered as a monolithic package, raising challenges for some maintenance tasks such as model restructure and re-adaption. In this work, we propose DeepArc, a novel modularization method for neural networks, to reduce the cost of model maintenance tasks. Specifically, DeepArc decomposes a neural network into several consecutive modules, each of which encapsulates consecutive layers with similar semantics. The network modularization facilitates practical tasks such as refactoring the model to preserve existing features (e.g., model compression) and enhancing the model with new features (e.g., fitting new samples). The modularization and encapsulation allow us to restructure or retrain the model by only pruning and tuning a few localized neurons and layers. Our experiments show that (1) DeepArc can boost the runtime efficiency of the state-of-the-art model compression techniques by 14.8%; (2) compared to the traditional model retraining, DeepArc only needs to train less than 20% of the neurons on average to fit adversarial samples and repair under-performing models, leading to 32.85% faster training performance while achieving similar model prediction performance.
Improving Small-Scale Human Action Recognition Performance Using a 3D Heatmap Volume

In recent years, skeleton-based human action recognition has garnered significant research attention, with proposed recognition or segmentation methods typically validated on large-scale coarse-grained action datasets. However, there remains a lack of research on the recognition of small-scale fine-grained human actions using deep learning methods, which have greater practical significance. To address this gap, we propose a novel approach based on heatmap-based pseudo videos and a unified, general model applicable to all modality datasets. Leveraging anthropometric kinematics as prior information, we extract common human motion features among datasets through an ad hoc pre-trained model. To overcome joint mismatch issues, we partition the human skeleton into five parts, a simple yet effective technique for information sharing. Our approach is evaluated on two datasets, including the public Nursing Activities and our self-built Tai Chi Action dataset. Results from linear evaluation protocol and fine-tuned evaluation demonstrate that our pre-trained model effectively captures common motion features among human actions and achieves steady and precise accuracy across all training settings, while mitigating network overfitting. Notably, our model outperforms state-of-the-art models in recognition accuracy when fusing joint and limb modality features along the channel dimension.
COVID-19 Contact Tracing Experience at a Tertiary Healthcare Center in Saudi Arabia

IntroductionAs of May 2023, the end of the pandemic, the cumulative number of COVID-19 cases reached more than 841,000 cases.Healthcare workers (HCWs) especially have been at the frontline during this pandemic and are at a higher chance of contracting COVID-19.Approximately half of all high-risk exposures were to healthcare personnel with COVID-19.While several tools for contact tracing were developed for the general public, contact tracing for infectious diseases in the healthcare setting is limited, and global testing of HCWs, or in-hospital digital tracing, is not performed in most facilities.The King Faisal Specialist Hospital and Research Center (KFSH&RC) collaborated with the Infection Control and Health Information Technology Affairs (HITA) to create an automated COVID-19 contact tracing tool specifically for HCWs who worked at the institute.This study aims to describe the contact tracing experience at KFSH&RC. MethodsA retrospective study was conducted to describe the use of an automated tool that was developed to assist in the contact tracing process and that was to be used by KFSH&RC employees who had been in contact with a COVID-19-positive individual.This tool is utilized for the early identification of possible COVID-19 cases and risk stratification of the exposed individuals.The tool can be accessed through the KFSH&RC website; it also collects information about the COVID-19 exposure rate among the different departments such as administration, capital projects/facilities, and healthcare at a tertiary care hospital in Riyadh, Saudi Arabia ResultsThe tool has been utilized 7,353 times by contact cases.Approximately 7% of those tested later developed a COVID-19 infection.When assessing the positivity rates per department, The Environmental Services Department had the highest positivity rate of 28.21%, followed by Health Information Technology and Analytics (HITA), and then the Central Transportation Department. ConclusionThis study acts as the first of its kind to describe the successful use of the healthcare contact tracing system in one of Saudi Arabia's largest hospitals (KFSH&RC) and describe the infection trends in different departments of the hospital.Through the tracing system, the departments with the highest COVID-19 infection occurrences at the hospital were identified in a timely manner, and safety protocols were implemented.
Development of an Automated Scoring Model Using SentenceTransformers for Discussion Forums in Online Learning Environments

Due to the limitations of public datasets, research on automatic essay scoring in Indonesian has been restrained and resulted in suboptimal accuracy. In general, the main goal of the essay scoring system is to improve execution time, which is usually done manually with human judgment. This study uses a discussion forum in online learning to generate an assessment between the responses and the lecturer's rubric in the automated essay scoring. A SentenceTransformers pre-trained model that can construct the highest vector embedding was proposed to identify the semantic meaning between the responses and the lecturer's rubric. The effectiveness of monolingual and multilingual models was compared. This research aims to determine the model's effectiveness and the appropriate model for the Automated Essay Scoring (AES) used in paired sentence Natural Language Processing tasks. The distiluse-base-multilingual-cased-v1 model, which employed the Pearson correlation method, obtained the highest performance. Specifically, it obtained a correlation value of 0.63 and a mean absolute error (MAE) score of 0.70. It indicates that the overall prediction result is enhanced when compared to the earlier regression task research.
Dual-Template Siamese Network with Attention Feature Fusion for Object Tracking

In order to alleviate the adverse effects resulted from complex scenes for object tracking, such as fast movement, mottled background, interference of similar objects, and occlusion etc., an algorithm using dualtemplate Siamese network with attention feature fusion, named SiamDT, is proposed in this paper.The main idea include that the original ResNet-50 network is improved to extract deep semantic information and shallow spatial information, which are effectively fused using the attention mechanism to achieve accurate feature representation of objects.In addition, a template branch is added to the traditional Siamese network in which a dynamic template is generated together with the first frame image to solve the problems of template failure and model drift.Experimental results on OTB100 dataset and VOT2018 dataset show that the proposed approach obtains the excellent performance compared with the state-of-the-art tracking algorithms, which verifies the feasibility and effectiveness of the proposed approach.
Opportunities and Challenges of Quantum Computing for Engineering Optimization

Abstract Quantum computing as the emerging paradigm for scientific computing has attracted significant research attention in the past decade. Quantum algorithms to solve the problems of linear systems, eigenvalue, optimization, machine learning, and others have been developed. The main advantage of utilizing quantum computer to solve optimization problems is that quantum superposition allows for massive parallel searching of solutions. This article provides an overview of fundamental quantum algorithms that can be utilized in solving optimization problems, including Grover search, quantum phase estimation, quantum annealing, quantum approximate optimization algorithm, variational quantum eigensolver, and quantum walk. A review of recent applications of quantum optimization methods for engineering design, including materials design and topology optimization, is also given. The challenges to develop scalable and reliable quantum algorithms for engineering optimization are discussed.
Protect and Extend - Using GANs for Synthetic Data Generation of Time-Series Medical Records

Preservation of private user data is of paramount importance for high Quality of Experience (QoE) and acceptability, particularly with services treating sensitive data, such as IT-based health services. Whereas anonymization techniques were shown to be prone to data re-identification, synthetic data generation has gradually replaced anonymization since it is relatively less time and resource-consuming and more robust to data leakage. Generative Adversarial Networks (GANs) have been used for generating synthetic datasets, especially GAN frameworks adhering to the differential privacy phenomena. This research compares state-of-the-art GAN-based models for synthetic data generation to generate time-series synthetic medical records of dementia patients which can be distributed without privacy concerns. Predictive modeling, autocorrelation, and distribution analysis are used to assess the Quality of Generating (QoG) of the generated data. The privacy preservation of the respective models is assessed by applying membership inference attacks to determine potential data leakage risks. Our experiments indicate the superiority of the privacy-preserving GAN (PPGAN) model over other models regarding privacy preservation while maintaining an acceptable level of QoG. The presented results can support better data protection for medical use cases in the future.
A Blockchain-Based Trustworthy Model Evaluation Framework for Deep Learning and Its Application in Moving Object Segmentation

Model evaluation is critical in deep learning. However, the traditional model evaluation approach is susceptible to issues of untrustworthiness, including insecure data and model sharing, insecure model training, incorrect model evaluation, centralized model evaluation, and evaluation results that can be tampered easily. To minimize these untrustworthiness issues, this paper proposes a blockchain-based model evaluation framework. The framework consists of an access control layer, a storage layer, a model training layer, and a model evaluation layer. The access control layer facilitates secure resource sharing. To achieve fine-grained and flexible access control, an attribute-based access control model combining the idea of a role-based access control model is adopted. A smart contract is designed to manage the access control policies stored in the blockchain ledger. The storage layer ensures efficient and secure storage of resources. Resource files are stored in the IPFS, with the encrypted results of their index addresses recorded in the blockchain ledger. Another smart contract is designed to achieve decentralized and efficient management of resource records. The model training layer performs training on users’ servers, and, to ensure security, the training data must have records in the blockchain. The model evaluation layer utilizes the recorded data to evaluate the recorded models. A method in the smart contract of the storage layer is designed to enable evaluation, with scores automatically uploaded as a resource attribute. The proposed framework is applied to deep learning-based motion object segmentation, demonstrating its key functionalities. Furthermore, we validated the storage strategy adopted by the framework, and the trustworthiness of the framework is also analyzed.
StarSRGAN: Improving Real-World Blind Super-Resolution

The aim of blind super-resolution (SR) in computer vision is to improve the resolution of an image without prior knowledge of the degradation process that caused the image to be low-resolution. The State of the Art (SOTA) model Real-ESRGAN has advanced perceptual loss and produced visually compelling outcomes using more complex degradation models to simulate real-world degradations. However, there is still room to improve the super-resolved quality of Real-ESRGAN by implementing recent techniques. This research paper introduces StarSRGAN, a novel GAN model designed for blind super-resolution tasks that utilize 5 various architectures. Our model provides new SOTA performance with roughly 10% better on the MANIQA and AHIQ measures, as demonstrated by experimental comparisons with Real-ESRGAN. In addition, as a compact version, StarSRGAN Lite provides approximately 7.5 times faster reconstruction speed (real-time upsampling from 540p to 4K) but can still keep nearly 90% of image quality, thereby facilitating the development of a real-time SR experience for future research. Our codes are released at https://github.com/kynthesis/StarSRGAN.
Operationally-based program equivalence proofs using LCTRSs

We propose an operationally-based framework for deductive proofs of program equivalence. It is based on encoding the language semantics as logically constrained term rewriting systems (LCTRSs) and the two programs as terms. As a novelty of our method, we show that it enables relational reasoning about programs in various settings, which are encoded in the operational semantics. For example, we show how our method can be used to prove programs that are equivalent when considering an unbounded stack, but where the equivalence fails to hold if the stack is bounded. We also show how to formalize read-sets and write-sets of symbolic expressions and statements by extending the operational semantics in a conservative way. This enables the relational verification of program schemas, which we exploit to prove compiler optimizations that cannot be handled by other tools. Our method requires an extension of standard LCTRSs with axiomatized symbols, which generate new research questions. We also present a prototype implementation that proves the practical feasibility of our approach.
Deep perceptual hashing algorithms with hidden dual purpose: when client-side scanning does facial recognition

End-to-end encryption (E2EE) provides strong technical protections to individuals from interferences. Governments and law enforcement agencies around the world have however raised concerns that E2EE also allows illegal content to be shared undetected. Client-side scanning (CSS), using perceptual hashing (PH) to detect known illegal content before it is shared, is seen as a promising solution to prevent the diffusion of illegal content while preserving encryption. While these proposals raise strong privacy concerns, proponents of the solutions have argued that the risk is limited as the technology has a limited scope: detecting known illegal content. In this paper, we show that modern perceptual hashing algorithms are actually fairly flexible pieces of technology and that this flexibility could be used by an adversary to add a secondary hidden feature to a client-side scanning system. More specifically, we show that an adversary providing the PH algorithm can "hide" a secondary purpose of face recognition of a target individual alongside its primary purpose of image copy detection. We first propose a procedure to train a dual-purpose deep perceptual hashing model by jointly optimizing for both the image copy detection and the targeted facial recognition task. Second, we extensively evaluate our dual-purpose model and show it to be able to reliably identify a target individual 67% of the time while not impacting its performance at detecting illegal content. We also show that our model is neither a general face detection nor a facial recognition model, allowing its secondary purpose to be hidden. Finally, we show that the secondary purpose can be enabled by adding a single illegal looking image to the database. Taken together, our results raise concerns that a deep perceptual hashing-based CSS system could turn billions of user devices into tools to locate targeted individuals.
Research on Path Planning and Trajectory Tracking of Autonomous Vehicle

With the rapid development of the world economy and science and technology, the intelligent era has kicked off, and the development and application of automatic driving technology has also attracted people's attention.Trajectory planning and tracking control, as the key technologies of autonomous driving, determine the safety and stability of autonomous vehicles during driving.Aiming at the problems of planning safety and control robustness, research on structured road obstacle avoidance trajectory planning and trajectory tracking control algorithm of autonomous vehicles can effectively improve the performance of the obstacle avoidance system of autonomous vehicles.In this paper, the obstacle avoidance path planning problem of autonomous vehicle is studied, and in order to reduce the difficulty of system modeling and improve the controller performance, the trajectory tracking control is decoupled into horizontal control and vertical control, and the research is carried out respectively.