Reviewerly/exHarmony huggingface dataset:
Environment-Invariant Curriculum Relation Learning for Fine-Grained Scene Graph Generation

The scene graph generation (SGG) task is designed to identify the predicates based on the subject-object pairs. However, existing datasets generally include two imbalance cases: one is the class imbalance from the predicted predicates and another is the context imbalance from the given subject-object pairs, which presents significant challenges for SGG. Most existing methods focus on the imbalance of the predicted predicate while ignoring the imbalance of the subject-object pairs, which could not achieve satisfactory results. To address the two imbalance cases, we propose a novel Environment Invariant Curriculum Relation learning (EICR) method, which can be applied in a plug-and-play fashion to existing SGG methods. Concretely, to remove the imbalance of the subject-object pairs, we first construct different distribution environments for the subject-object pairs and learn a model invariant to the environment changes. Then, we construct a class-balanced curriculum learning strategy to balance the different environments to remove the predicate imbalance. Comprehensive experiments conducted on VG and GQA datasets demonstrate that our EICR framework can be taken as a general strategy for various SGG models, and achieve significant improvements.
Counterfactual-based Saliency Map: Towards Visual Contrastive Explanations for Neural Networks

Explaining deep models in a human-understandable way has been explored by many works that mostly explain why an input causes a corresponding prediction (i.e., Why P?). However, seldom they could handle those more complex causal questions like "Why P rather than Q?" and "Why one is P, while another is Q?", which would better help humans understand the behavior of deep models. Considering the insufficient study on such complex causal questions, we make the first attempt to explain different causal questions by contrastive explanations in a unified framework, i.e., Counterfactual Contrastive Explanation (CCE), which visually and intuitively explains the aforementioned questions via a novel positive-negative saliency-based explanation scheme. More specifically, we propose a content-aware counterfactual perturbing algorithm to stimulate contrastive examples, from which a pair of positive and negative saliency maps could be derived to contrastively explain why P (positive class) rather than Q (negative class). Beyond existing works, our counterfactual perturbation meets the principles of validity, sparsity, and data distribution closeness at the same time. In addition, by slightly adjusting the objective of perturbation, our framework can adapt to different causal questions. Extensive experimental evaluation demonstrates the effectiveness and superior performance of the proposed CCE on different benchmark metrics for interpretability, including Sanity Check, Class Deviation Score and Insertion-Deletion tests. A user study is conducted and the results show that user confidence is increasing significantly when presented with CCE compared to standard saliency map baselines.
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models

Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter- and data- efficient way, by learning the "soft prompts" to condition frozen pretraining models. Though effective, it is particularly problematic in the few-shot scenario, where prompt tuning performance is sensitive to the initialization and requires a time-consuming process to find a good initialization, thus restricting the fast adaptation ability of the pre-training models. In addition, prompt tuning could undermine the generalizability of the pre-training models, because the learnable prompt tokens are easy to overfit to the limited training samples. To address these issues, we introduce a novel Gradient-RegulAted Meta-prompt learning (GRAM) framework that jointly meta-learns an efficient soft prompt initialization for better adaptation and a lightweight gradient regulating function for strong cross-domain generalizability in a meta-learning paradigm using only the unlabeled image-text pre-training data. Rather than designing a specific prompt tuning method, our GRAM can be easily incorporated into various prompt tuning methods in a model-agnostic way, and comprehensive experiments show that GRAM brings about consistent improvement for them in several settings (i.e., few-shot learning, cross-domain generalization, cross-dataset generalization, etc.) over 11 datasets. Further, experiments show that GRAM enables the orthogonal methods of textual and visual prompt tuning to work in a mutually-enhanced way, offering better generalizability beyond the uni-modal prompt tuning methods.
One-shot Implicit Animatable Avatars with Model-based Priors

Existing neural rendering methods for creating human avatars typically either require dense input signals such as video or multi-view images, or leverage a learned prior from large-scale specific 3D human datasets such that reconstruction can be performed with sparse-view inputs. Most of these methods fail to achieve realistic reconstruction when only a single image is available. To enable the data-efficient creation of realistic anima table 3D humans, we propose ELICIT, a novel method for learning human-specific neural radiance fields from a single image. Inspired by the fact that humans can effortlessly estimate the body geometry and imagine full-body clothing from a single image, we leverage two priors in ELICIT: 3D geometry prior and visual semantic prior. Specifically, ELICIT utilizes the 3D body shape geometry prior from a skinned vertex-based template model (i.e., SMPL) and implements the visual clothing semantic prior with the CLIP-based pre-trained models. Both priors are used to jointly guide the optimization for creating plausible content in the invisible areas. Taking advantage of the CLIP models, ELICIT can use text descriptions to generate text-conditioned unseen regions. In order to further improve visual details, we propose a segmentation-based sampling strategy that locally refines different parts of the avatar. Comprehensive evaluations on multiple popular benchmarks, including ZJU-MoCAP, Human3.6M, and DeepFashion, show that ELICIT outperforms strong baseline methods of avatar creation when only a single image is available. The code is public for research purposes at https://huangyangyi.github.io/ELICIT
StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models

Content and style (C-S) disentanglement is a fundamental problem and critical challenge of style transfer. Existing approaches based on explicit definitions (e.g., Gram matrix) or implicit learning (e.g., GANs) are neither interpretable nor easy to control, resulting in entangled representations and less satisfying results. In this paper, we propose a new C-S disentangled framework for style transfer without using previous assumptions. The key insight is to explicitly extract the content information and implicitly learn the complementary style information, yielding interpretable and controllable C-S disentanglement and style transfer. A simple yet effective CLIP-based style disentanglement loss coordinated with a style reconstruction prior is introduced to disentangle C-S in the CLIP image space. By further leveraging the powerful style removal and generative ability of diffusion models, our framework achieves superior results than state of the art and flexible C-S disentanglement and trade-off control. Our work provides new insights into the C-S disentanglement in style transfer and demonstrates the potential of diffusion models for learning well-disentangled C-S characteristics.
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference

The combination of Neural Architecture Search (NAS) and quantization has proven successful in automatically designing low-FLOPs INT8 quantized neural networks (QNN). However, directly applying NAS to design accurate QNN models that achieve low latency on real-world devices leads to inferior performance. In this work, we identify that the poor INT8 latency is due to the quantization-unfriendly issue: the operator and configuration (e.g., channel width) choices in prior art search spaces lead to diverse quantization efficiency and can slow down the INT8 inference speed. To address this challenge, we propose SpaceEvo, an automatic method for designing a dedicated, quantization-friendly search space for each target hardware. The key idea of SpaceEvo is to automatically search hardware-preferred operators and configurations to construct the search space, guided by a metric called Q-T score to quantify how quantization-friendly a candidate search space is. We further train a quantized-for-all supernet over our discovered search space, enabling the searched models to be directly deployed without extra retraining or quantization. Our discovered models, SEQnet, establish new SOTA INT8 quantized accuracy under various latency constraints, achieving up to 10.1% accuracy improvement on ImageNet than prior art CNNs under the same latency. Extensive experiments on real devices show that SpaceEvo consistently outperforms manually-designed search spaces with up to 2.5× faster speed while achieving the same accuracy.
Learning from Noisy Pseudo Labels for Semi-Supervised Temporal Action Localization

Semi-Supervised Temporal Action Localization (SS-TAL) aims to improve the generalization ability of action detectors with large-scale unlabeled videos. Albeit the recent advancement, one of the major challenges still remains: noisy pseudo labels hinder efficient learning on abundant unlabeled videos, embodied as location biases and category errors. In this paper, we dive deep into such an important but understudied dilemma. To this end, we propose a unified framework, termed Noisy Pseudo-Label Learning, to handle both location biases and category errors. Specifically, our method is featured with (1) Noisy Label Ranking to rank pseudo labels based on the semantic confidence and boundary reliability, (2) Noisy Label Filtering to address the class-imbalance problem of pseudo labels caused by category errors, (3) Noisy Label Learning to penalize in-consistent boundary predictions to achieve noise-tolerant learning for heavy location biases. As a result, our method could effectively handle the label noise problem and improve the utilization of a large amount of unlabeled videos. Extensive experiments on THUMOS14 and ActivityNet v1.3 demonstrate the effectiveness of our method. The code is available at github.com/kunnxia/NPL.
Investigating Relational Reasoning Performance: A Comparative Analysis of Humans and MRNet Network

Relational reasoning is a crucial aspect of intelligent decision-making, and while artificial intelligence (AI) has shown promise in solving relational reasoning problems, its interpretability remains limited. This article explores the performance of human participants in various relational reasoning tasks using the Impartial-RAVEN dataset and compares their results with those obtained from the Multi-scale Relation Network (MRNet). Surprisingly, a remarkable similarity in performance patterns is found between humans and MRNet. Through a detailed analysis, this article uncover the factors contributing to this phenomenon. Moreover, insightful suggestions are provided to enhance the problem-solving ability of the MRNet network based on human behavior. This study sheds light on understanding human cognitive abilities in relational reasoning tasks and offers valuable insights for the development and evaluation of AI models.
Towards Fairness-aware Adversarial Network Pruning

Network pruning aims to compress models while minimizing loss in accuracy. With the increasing focus on bias in AI systems, the bias inheriting or even magnification nature of traditional network pruning methods has raised a new perspective towards fairness-aware network pruning. Straightforward pruning plus debias methods and recent designs for monitoring disparities of demographic attributes during pruning have endeavored to enhance fairness in pruning. However, neither simple assembling of two tasks nor specifically designed pruning strategies could achieve the optimal trade-off among pruning ratio, accuracy, and fairness. This paper proposes an end-to-end learnable framework for fairness-aware network pruning, which optimizes both pruning and debias tasks jointly by adversarial training against those final evaluation metrics like accuracy for pruning, and disparate impact (DI) and equalized odds (DEO) for fairness. In other words, our fairness-aware adversarial pruning method would learn to prune without any handcraft rules. Therefore, our approach could flexibly adapt to variate network structures. Exhaustive experimentation demonstrates the generalization capacity of our approach, as well as superior performance on pruning and debias simultaneously. To highlight, the proposed method could preserve the SOTA pruning performance while significantly improving fairness by around 50% as compared to traditional pruning methods.
Partition Speeds Up Learning Implicit Neural Representations Based on Exponential-Increase Hypothesis

Implicit neural representations (INRs) aim to learn a continuous function (i.e., a neural network) to represent an image, where the input and output of the function are pixel coordinates and RGB/Gray values, respectively. However, images tend to consist of many objects whose colors are not perfectly consistent, resulting in the challenge that image is actually a discontinuous piecewise function and cannot be well estimated by a continuous function. In this paper, we empirically investigate that if a neural network is enforced to fit a discontinuous piecewise function to reach a fixed small error, the time costs will increase exponentially with respect to the boundaries in the spatial domain of the target signal. We name this phenomenon the exponential-increase hypothesis. Under the exponential-increase hypothesis, learning INRs for images with many objects will converge very slowly. To address this issue, we first prove that partitioning a complex signal into several sub-regions and utilizing piecewise INRs to fit that signal can significantly speed up the convergence. Based on this fact, we introduce a simple partition mechanism to boost the performance of two INR methods for image reconstruction: one for learning INRs, and the other for learning-to-learn INRs. In both cases, we partition an image into different sub-regions and dedicate smaller networks for each part. In addition, we further propose two partition rules based on regular grids and semantic segmentation maps, respectively. Extensive experiments validate the effectiveness of the proposed partitioning methods in terms of learning INR for a single image (ordinary learning framework) and the learning-to-learn framework. Code is released here.
Remote sensing aircraft image target detection based on Yolov5-EBA

The problems of overlooked detection and false detection issues are pervasive in the aircraft target detection process, and a remote sensing image target improvement model YOLOv5s-EBA based on YOLOv5s is proposed. first, the algorithm introduces an attention network to improve the model main network to enhance the feature selection of aircraft with different resolutions; then, the path aggregation network (PAN) in the original YOLOv5s network is replaced by the bi-directional feature pyramid network to implementing the multi-scale feature blending of aircraft targets simply and quickly; Finally, we substitute CIoU with α-IoU, and the prediction accuracy of the bounding box is improved by introducing the parameter α and by adjusting the parameter α. The experimental results indicate on the homemade dataset Airplane-GDD, contrast with the original YOLOv5 algorithm, the average precision of the YOLOv5s-EBA algorithm is increased by 2.2 percentage points, and the accuracy and recall witness a boost of 2.3 and 2.7 percentage points, respectively, and there is also a noteworthy enhancement for the problem of incorrect and overlooked detections in the detection process.
Construction of marine target detection dataset and target detection methods

This paper aims to investigate the construction of a marine target detection dataset and target detection methods, focusing specifically on marine ships. To begin with, a collection of marine ship images, 41,162, is obtained through difference resources The LabelImg software is then utilized to visually select the dataset, generating the target object calibration information in the image files. This approach enhances the efficiency of dataset file creation. The dataset is converted from Pascal VOC to YOLO and COCO format. Furthermore, existing target detection algorithms are thoroughly examined, analyzing their structural characteristics as well as the strengths and weaknesses observed in practical applications. Following the preparation of the environment and dataset processing, six detection methods, YOLOv7, Faster R-CNN, RetinaNet, FCOS, CenterNet, and ATSS are selected for training and testing using the constructed marine ship dataset. The evaluation metrics of accuracy, recall rate, and average accuracy are employed to analyze and summarize the experimental results.
New technologies and geographies in performing art: A study on environments and relationships in some recent shows in Europe

Digitization has had a significant impact on live performance, specifically influencing the environments of representation and giving rise to new trends in performative language. Certain performances from the last fifteen years that have circulated in Europe are taken into consideration, experienced by audiences with various technological provisions (audio headphones, video conferencing, and 3D goggles) and outside traditional theater spaces. In light of certain theories about learning dynamics, a geo-relational approach is pursued to understand what potential formal evolutions could occur in the realm of performance with reference to the use of technologies, and in which cases the complexity of multisensory experience in traditional in-person theater has yet to be fully achieved.
DEVELOPMENTOFMETHODSFORTESTINGTHELIBRARYOFCRYPTOGRAPHICTRANSFORMATIONSONTHEEXAMPLEOFTHEMST3CRYPTOSYSTEMBASEDONGENERALIZEDSUZUKI2-GROUPS

The article proposes a methodology for testing a library of cryptographic transformations with the implementation of an improved encryption scheme on generalized Suzuki 2-groups in the MST3 cryptosystem. The need to improve existing methods of cryptosystem creation is driven by progress in quantum computer development, which possess sufficient computational power to compromise many existing public key cryptosystems. This is especially true for systems based on factorization and discrete logarithm, such as RSA and ECC. Over the last nearly 20 years, there have been proposals for using non-commutative groups to develop quantum-resistant cryptosystems. The unsolved word problem, formulated by Wagner and Magyarik, uses permutation groups and is a promising direction in cryptosystem development. Magliveras proposed logarithmic signatures, a special type of factorization applied to finite groups, and the latest version of this technology is known as MST3, based on the Suzuki group. The first implementation of the cryptosystem on the generalized Suzuki 2-group had limitations in encryption and protection against brute force attacks. Over the past years, many proposals have been made to improve the basic design. The research conducted by the authors expanded the possibilities of using public cryptography by refining parameters based on non-Abelian groups. The article demonstrates the methodology for conducting tests of the practical implementation of the library of cryptographic transformations with the implementation of an improved encryption scheme on Suzuki 2-groups, confirming its functionality.
METHOD OF LOW-RESOURCE HASHING TYPE "DATA – GENERATOR"

Creating a secure and efficient structure of a cryptographic algorithm is one of the key cryptographic tasks. Recently, cryptography for low-resource devices has attracted considerable attention of world scientists. A significant portion of the research is dedicated to the examination of block encryption methods, and conversely, there are notably fewer publicly disclosed proposals for low-resource hashing methods. Many hash functions recommended for use in low-resource devices by well-known standardization organizations are based on block encryption, which offers a sufficient level of security but demands substantial computational resources—something critical for their application in such devices. The urgency of investigating low-resource data hashing methods stems from the need to guarantee an adequate level of hash function security while minimizing computational resource usage through adjustments to the hashing process. This article reviews established approaches to constructing hash functions of varying complexities and examines the latest research and publications focused on low-resource hashing. Based on this, the structure and approach for developing a low-resource data hashing method were chosen. A novel low-resource hashing method, founded on the Merkle-Damgård construction and utilizing an iterative byte-oriented approach, is introduced. The process of low-resource hashing, according to the new method, is formally described. Statistical testing of the proposed method was conducted in accordance with NIST SP 800-22. An overview of the hardware implementation of the proposed low-resource hashing method is presented in the form of a generalized structural diagram. The complexity of the proposed hardware implementation is quantified in conventional units [GE] for hash value calculations of 128, 192, and 256 bits. A comparison of the proposed "data-generator" type hashing method with established low-resource hash functions, in terms of hardware costs, is conducted.