Reviewerly/exHarmony huggingface dataset:
Image compression-hiding algorithm based on compressive sensing and integer wavelet transformation

In this paper, a three-dimensional chaotic system is proposed. Based on this chaotic system and two-dimensional compressive sensing, an asymmetric visually meaningful image compression-hiding algorithm is presented. Firstly, in the keystream generation stage, a novel parameter transformation model is constructed to pick up the feature information from the plain image as the plaintext key. Then, Rivest-Shamir-Adleman algorithm is employed to encrypt the plaintext key into the ciphertext key seen as public key. Before generating the initial values for the chaotic system, a new initial value getting model is designed to transform both the plaintext and the ciphertext keys. After solving the chaotic system, the keystream is produced which is then used in the image encryption process. Secondly, in the compression and encryption phase, a novel keystream pre-processing model is built to generate new sequences with a confusion performed on the plain image. Then, a newly constructed measurement matrix is designed to do two-dimensional compressive sensing on confusing the image to get measurements. Before obtaining the cipher image, a double diffusion operation is applied on these measurements. Thirdly, in the image hiding stage, the carrier image is performed by integer wavelet transformation to obtain coefficient matrices. Then, the cipher image is decomposed in decimal, getting the ones, tens and hundreds of pixels to form three bit matrices, of which are embedded into the three medium-high coefficient matrices of integer wavelet transformation, respectively. Finally, after performing inverse integer wavelet transformation, the carrier image containing secrets, i.e., visually meaningful encrypted image, is obtained. Experimental results also show that at a low compression ratio of 0.25, the normalized correlation coefficient between the original plain image and the recovered image is almost equal to one, while the peak signal-to-noise ratio between the carrier image containing secrets and its original carrier can reach as high as 42 dB. In addition, the proposed image compression-hiding algorithm performs good ability consideing the brute force attack and the cropping attack.
A novel improved lemurs optimization algorithm for feature selection problems

The irrelevant and repeated features in high-dimensional datasets can negatively affect the final performance and accuracy of classification-based models. Therefore, feature selection (FS) techniques can be used to determine the most optimal relevant features. In this paper, we fuse a new enhanced model from Lemurs Optimization (LO) algorithm, called Enhanced Lemurs Optimization (ELO). We combine Opposition Based Learning (OBL) and Local Search Algorithm (LSA) to address exploration and exploitation challenges, respectively. Our proposed ELO algorithm incorporates U-shaped and Sigmoid transfer functions during the position update step, leading to improved accuracy and convergence. These new deployments based on the U-shaped and Sigmoid transfer functions are called ELO-U and ELO-S algorithms, respectively. The performance of all three new versions of our proposed optimization algorithms (ELO, ELO-U, and ELO-S) has been evaluated using 21 UCI datasets in different fields and sizes. Moreover, their results are also compared to other competitive algorithms. The evaluation process included several measurements such as fitness value, an average of selected features, and average accuracy. Experimental results demonstrate that our proposed ELO-U algorithm achieves the best average accuracy of 91.03%. Statistical analysis using Friedman and Wilcoxon tests confirms the superiority of ELO-U over other competitors.
A survey on conventional and learning‐based methods for multi‐view stereo

Abstract 3D reconstruction of scenes using multiple images, relying on robust correspondence search and depth estimation, has been thoroughly studied for the two‐view and multi‐view scenarios in recent years. Multi‐view stereo (MVS) algorithms aim to generate a rich, dense 3D model of the scene in the form of a dense point cloud or a triangulated mesh. In a typical MVS pipeline, the robust estimations for the camera poses along with the sparse points obtained from structure from motion (SfM) are used as input. During this process, the depth of generally every pixel of the scene is to be calculated. Several methods, either conventional or, more recently, learning‐based have been developed for solving the correspondence search problem. A vast amount of research exists in the literature using local, global or semi‐global stereomatching approaches, with the PatchMatch algorithm being among the most popular and efficient conventional ones in the last decade. Yet, and despite the widespread evolution of the algorithms, yielding complete, accurate and aesthetically pleasing 3D representations of a scene remains an open issue in real‐world and large‐scale photogrammetric applications. This work aims to provide a concrete survey on the most widely used MVS methods, investigating underlying concepts and challenges. To this end, the theoretical background and relative literature are discussed for both conventional and learning‐based approaches, with a particular focus on close‐range 3D reconstruction applications.
Comparative performance of ensemble machine learning for Arabic cyberbullying and offensive language detection

Abstract Since cyberbullying impacts both individual victims and entire society, research on abusive language and its detection has attracted attention in recent years. Because social media sites like Facebook, Instagram, Twitter, and others are so widely accessible, hate speech, bullying, sexism, racism, aggressive material, harassment, poisonous comments, and other types of abuse have all substantially increased. Due to the critical requirement to detect, regulate, and limit the spread of harmful content on social networking sites, we conducted this study to automate the detection of offensive language or cyberbullying. We created a new Arabic balanced data set to be used in the offensive detection process because having a balanced data set for a model would result in improved accuracy models. Recently, the performance of single classifiers has been improved using ensemble machine learning. The purpose of this study is to examine the effectiveness of several single and ensemble machine learning algorithms in identifying Arabic text that contains foul language and cyberbullying. Applying them to three Arabic datasets, we have selected three machine learning classifiers and three ensemble models for this aim. Two of them are offensive datasets that are readily accessible in the public, while the third one was created. The results showed that the single learner machine learning strategy is inferior to the ensemble machine learning methodology. Voting performs is the best performing trained ensemble machine learning classifier, outperforming the best single learner classifier (65.1%, 76.2%, and 98%) for the same datasets with accuracy scores of (71.1%, 76.7%, and 98.5%) for each of the three datasets used. Finally, we improve the voting technique’s performance through hyperparameter tuning on the Arabic cyberbullying data set.
Sistem Pakar Penyakit Telinga Menggunakan Metode Naïve Bayes

An expert system is a type of artificial intelligence application that is used to tackle complex problems that require specialized knowledge. Expert systems can be used in a variety of disciplines, including healthcare, finance, and manufacturing. The aim of this study was to apply the Nave Bayes approach in a website-based ear illness diagnostic system and to determine its accuracy in an expert system for diagnosing ear disease. The naive Bayes approach is implemented in this research because it may assume that each symptom is independent of one another and can thus be used to assess the probability of a condition based on the symptoms that emerge. The results of this study show that the expert system for diagnosing ear disease using the Nave Bayes method is built on a website using the PHP programming language and the database maintained by MySQL, and this application has been tested 10 times, with 9 test data appropriate and 1 test data not appropriate. As a result of testing this application, the accuracy value obtained is 90%.
Open Set Classification of GAN-based Image Manipulations via a ViT-based Hybrid Architecture

Classification of AI-manipulated content is receiving great attention, for distinguishing different types of manipulations. Most of the methods developed so far fail in the open-set scenario, that is when the algorithm used for the manipulation is not represented by the training set. In this paper, we focus on the classification of synthetic face generation and manipulation in open-set scenarios, and propose a method for classification with a rejection option. The proposed method combines the use of Vision Transformers (ViT) with a hybrid approach for simultaneous classification and localization. Feature map correlation is exploited by the ViT module, while a localization branch is employed as an attention mechanism to force the model to learn per-class discriminative features associated with the forgery when the manipulation is performed locally in the image. Rejection is performed by considering several strategies and analyzing the model output layers. The effectiveness of the proposed method is assessed for the task of classification of facial attribute editing and GAN attribution.
TSRFormer: Transformer Based Two-stage Refinement for Single Image Shadow Removal

Single-image shadow removal aims to remove undesired shadow information from captured images. With the development of deep convolutional neural networks, several methods have been proposed to achieve promising performance in shadow removal. However, they still struggle with limited performance due to the non-homogeneous intensity distribution of the shadow. To address this issue, we propose a two-stage shadow removal architecture based on the transformer called TSRFormer. The proposed architecture is divided into shadow removal and content refinement networks. These two stages adopt different transformer architectures and remove the shadow based on different information to achieve effective shadow removal. Experiments performed on challenging benchmark show that the proposed model achieves the 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> highest SSIM in the NTIRE 2023 Image Shadow Removal Challenge. The source code will be public after the acceptance of this paper.
Does Image Anonymization Impact Computer Vision Training?

Image anonymization is widely adapted in practice to comply with privacy regulations in many regions. However, anonymization often degrades the quality of the data, reducing its utility for computer vision development. In this paper, we investigate the impact of image anonymization for training computer vision models on key computer vision tasks (detection, instance segmentation, and pose estimation). Specifically, we benchmark the recognition drop on common detection datasets, where we evaluate both traditional and realistic anonymization for faces and full bodies. Our comprehensive experiments reflect that traditional image anonymization substantially impacts final model performance, particularly when anonymizing the full body. Furthermore, we find that realistic anonymization can mitigate this decrease in performance, where our experiments reflect a minimal performance drop for face anonymization. Our study demonstrates that realistic anonymization can enable privacy-preserving computer vision development with minimal performance degradation across a range of important computer vision benchmarks.
Leveraging triplet loss for unsupervised action segmentation

In this paper, we propose a novel fully unsupervised framework that learns action representations suitable for the action segmentation task from the single input video itself, without requiring any training data. Our method is a deep metric learning approach rooted in a shallow network with a triplet loss operating on similarity distributions and a novel triplet selection strategy that effectively models temporal and semantic priors to discover actions in the new representational space. Under these circumstances, we successfully recover temporal boundaries in the learned action representations with higher quality compared with existing unsupervised approaches. The proposed method is evaluated on two widely used benchmark datasets for the action segmentation task and it achieves competitive performance by applying a generic clustering algorithm on the learned representations. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Zoom-VQA: Patches, Frames and Clips Integration for Video Quality Assessment

Video quality assessment (VQA) aims to simulate the human perception of video quality, which is influenced by factors ranging from low-level color and texture details to high-level semantic content. To effectively model these complicated quality-related factors, in this paper, we decompose video into three levels (i.e., patch level, frame level, and clip level), and propose a novel Zoom-VQA architecture to perceive spatio-temporal features at different levels. It integrates three components: patch attention module, frame pyramid alignment, and clip ensemble strategy, respectively for capturing region-of-interest in the spatial dimension, multi-level information at different feature levels, and distortions distributed over the temporal dimension. Owing to the comprehensive design, Zoom-VQA obtains state-of-the-art results on four VQA benchmarks and achieves 2nd place in the NTIRE 2023 VQA challenge. Notably, Zoom-VQA has outperformed the previous best results on two subsets of LSVQ, achieving 0.8860 (+1.0%) and 0.7985 (+1.9%) of SRCC on the respective subsets. Adequate ablation studies further verify the effectiveness of each component. Codes and models are released in https://github.com/k-zha14/Zoom-VQA.
Dynamic Consensus Committee-Based for Secure Data Sharing With Authorized Multi-Receiver Searchable Encryption

Data management services provided by the public cloud can economize the enterprise’s local storage costs, and meanwhile, realize data sharing among the enterprise. Corporate users, however, usually have concerns about the security and privacy of their data stored in the public cloud. Searchable encryption has been used as a secure method for enterprise users in the public cloud to share data via keyword search for many years. Nevertheless, conventional search and encryption primitives are challenging to realize several critical functions flexibly, such as key update, user revocation, lightweight computation, and low communication overhead for users, which will impede their widespread application. Besides, it may cause single-key-exposure security concerns by using untrusted devices. To address these issues, we present a dynamic consensus committee-based for secure data sharing with authorized multi-receiver searchable encryption (called DCC-SE), which exploits the blockchain dynamic committee to eliminate a single-key-exposure problem and enables fast keyword search without pairing and dynamic user management. The proposed DCC-SE scheme enjoys minor computation and communication overheads. It maintains a constant ciphertext size as the receiver increases. Furthermore, it periodically supports secure key and ciphertext updating and has been proven secure against chosen plaintext attacks and the chosen keyword guessing attacks under the random oracle model. The performance evaluation results show that DCC-SE has more feasibility and higher efficiency than the previous public key with keyword search(PEKS) schemes through theoretical analysis and simulation studies.
NeighborTrack: Single Object Tracking by Bipartite Matching with Neighbor Tracklets and Its Applications to Sports

We propose a post-processor, called NeighborTrack, that leverages neighbor information of the tracking target to validate and improve single-object tracking (SOT) results. It requires no additional data or retraining. Instead, it uses the confidence score predicted by the backbone SOT network to automatically derive neighbor information and then uses this information to improve the tracking results. When tracking an occluded target, its appearance features are untrustworthy. However, a general siamese network often cannot tell whether the tracked object is occluded by reading the confidence score alone, because it could be misled by neighbors with high confidence scores. Our proposed NeighborTrack takes advantage of unoccluded neighbors’ information to reconfirm the tracking target and reduces false tracking when the target is occluded. For the VOT challenge dataset commonly used in short-term object tracking, we improve three famous SOT networks, Ocean, TransT, and OSTrack, by an average of 1.92% EAO and 2.11% robustness. For the mid- and long-term tracking experiments based on OSTrack, we achieve state-of-the-art 72.25% AUC on LaSOT and 75.7% AO on GOT-10K. Most of the tracking examples we have used are related to sports.
OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework

Recent developments in Artificial Intelligence techniques have enabled their successful application across a spectrum of commercial and industrial settings. However, these techniques require large volumes of data to be aggregated in a centralized manner, forestalling their applicability to scenarios wherein the data is sensitive or the cost of data transmission is prohibitive. Federated Learning alleviates these problems by decentralizing model training, thereby removing the need for data transfer and aggregation. To advance the adoption of Federated Learning, more research and development needs to be conducted to address some important open questions. In this work, we propose OpenFed, an open-source software framework for end-to-end Federated Learning. OpenFed reduces the barrier to entry for both researchers and downstream users of Federated Learning by the targeted removal of existing pain points. For researchers, OpenFed provides a framework wherein new methods can be easily implemented and fairly evaluated against an extensive suite of benchmarks. For downstream users, OpenFed allows Federated Learning to be plugged and play within different subject-matter contexts, removing the need for deep expertise in Federated Learning. The source code of OpenFed is publicly available online at https://github.com/FederalLab/OpenFed.
RadarGNN: Transformation Invariant Graph Neural Network for Radar-based Perception

A reliable perception has to be robust against challenging environmental conditions. Therefore, recent efforts focused on the use of radar sensors in addition to camera and lidar sensors for perception applications. However, the sparsity of radar point clouds and the poor data availability remain challenging for current perception methods. To address these challenges, a novel graph neural network is proposed that does not just use the information of the points themselves but also the relationships between the points. The model is designed to consider both point features and point-pair features, embedded in the edges of the graph. Furthermore, a general approach for achieving transformation invariance is proposed which is robust against unseen scenarios and also counteracts the limited data availability. The transformation invariance is achieved by an invariant data representation rather than an invariant model architecture, making it applicable to other methods. The proposed RadarGNN model outperforms all previous methods on the RadarScenes dataset. In addition, the effects of different invariances on the object detection and semantic segmentation quality are investigated. The code is made available as open-source software under https://github.com/TUMFTM/RadarGNN.
Token Merging for Fast Stable Diffusion

The landscape of image generation has been forever changed by open vocabulary diffusion models. However, at their core these models use transformers, which makes generation slow. Better implementations to increase the throughput of these transformers have emerged, but they still evaluate the entire model. In this paper, we instead speed up diffusion models by exploiting natural redundancy in generated images by merging redundant tokens. After making some diffusion-specific improvements to Token Merging (ToMe), our ToMe for Stable Diffusion can reduce the number of tokens in an existing Stable Diffusion model by up to 60% while still producing high quality images with-out any extra training. In the process, we speed up image generation by up to 2× and reduce memory consumption by up to 5.6×. Furthermore, this speed-up stacks with efficient implementations such as xFormers, minimally impacting quality while being up to 5.4× faster for large images. Code is available at https://github.com/dbolya/tomesd.
Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanations

Applying traditional post-hoc attribution methods to segmentation or object detection predictors offers only limited insights, as the obtained feature attribution maps at input level typically resemble the models’ predicted segmentation mask or bounding box. In this work, we address the need for more informative explanations for these predictors by proposing the post-hoc eXplainable Artificial Intelligence method L-CRP to generate explanations that automatically identify and visualize relevant concepts learned, recognized and used by the model during inference as well as precisely locate them in input space. Our method therefore goes beyond singular input-level attribution maps and, as an approach based on the Concept Relevance Propagation technique, is efficiently applicable to state-of-the-art black-box architectures in segmentation and object detection, such as DeepLabV3+ and YOLOv6, among others. We verify the faithfulness of our proposed technique by quantitatively comparing different concept attribution methods, and discuss the effect on explanation complexity on popular datasets such as CityScapes, Pascal VOC and MS COCO 2017. The ability to precisely locate and communicate concepts is used to reveal and verify the use of background features, thereby highlighting possible biases of the model. Code is available on https://github.com/maxdreyer/L-CRP.
Anomaly Detection with Domain Adaptation

Despite great advances have been made in the field of domain adaptation (DA), the vast majority of current methods in DA solve classical ML tasks, e.g. classification. In this paper, we study a novel research direction: semi-supervised anomaly detection with domain adaptation. Given a set of normal data from a source domain and a limited number of normal examples from a target domain, the goal is to have a well-performing anomaly detector in the target domain. We then present the Invariant Representation Anomaly Detection (IRAD) to solve this problem where we first learn to extract a domain-invariant representation. The extraction is achieved by an across-domain encoder trained together with source-specific encoders and generators by adversarial learning. An anomaly detector is then trained using the learnt representations. We evaluate IRAD extensively on anomaly detection datasets, object recognition datasets and digits benchmarks. Experimental results show that IRAD outperforms baseline models by a wide margin across different datasets. We derive a theoretical lower bound for the joint error that explains the performance decay from over-training and also an upper bound for the generalization error.