Reviewerly/exHarmony huggingface dataset:
Iranian Architectural Styles Recognition Using Image Processing and Deep Learning

Iranian architecture, also known as Persian architecture, encompasses the design of buildings in Iran and extends to various regions in West Asia, the Caucasus, and Central Asia. With a rich history dating back at least 5,000 BC, it boasts distinctive features and styles. Iran, located in the Middle East, has faced ongoing geopolitical challenges, including the potential for conflicts, such as those in Iraq and Afghanistan. Unfortunately, historical monuments often become unintentional casualties during wartime, suffering damage or destruction. These historical monuments hold cultural and historical significance not only for the country they belong to but for all of humanity. Therefore, it is crucial to make efforts to preserve them. In this paper, we propose the development of an automated system utilizing Deep Learning methods for the detection and recognition of historical monuments. This system can be integrated into military equipment to help identify the architectural style of a building and determine its construction date. By doing so, it can provide a critical warning to prevent the targeting of historically significant structures. To support our system, we have curated a dataset consisting of approximately 3,000 photographs showcasing six distinct styles of Iranian historical architecture. Figure 1 provides some examples of these photographs. It is worth noting that this dataset can be valuable for various scientific research projects and applications beyond our proposed system. Additionally, it offers tourists the opportunity to learn about Iranian historical monuments independently, using their mobile phones to access information about a monument’s historical period and architectural style, eliminating the need for a traditional guide. This initiative aims to safeguard the invaluable cultural heritage of Iran and neighboring regions, contributing to the collective preservation of these historical treasures.
Accident event detection from Facebook posts written in Bengali and Banglish languages

Summary Accident events refer to abrupt or unpredictable incidents, which often cause a temporary or permanent bruise. Sometimes, the damage is so severe that it permanently ruins an individual's whole life and family. Detection of these unwanted accident events is the prerequisite for preventing and lowering the rate of casualty. Most people are habituated to posting their seeings, thoughts, and beliefs on social media and often post about accident incidents. This work detects accident incidents from real‐time Facebook data and the text‐mentioned probable time and location. Previous research works in this domain have been conducted in English, mainly with Twitter data. In this work, Bengali and Banglish Facebook posts have been used to aid the emergency response team in rapidly rescuing injured personnel. For this purpose, real‐time Facebook data have been crawled and pre‐processed. After that, several layers of feature extraction including individual keyword feature and paired keyword feature have been conducted and then sentiment analysis of the posts have taken place. Individual keyword feature is further divided into common event keyword and accident‐specific event keyword. Finally, the Support Vector Machine (SVM) classifier is used to classify accident and non‐accident events. Two more classifiers, Naive Bayes (NB) and Decision Tree (DT), are used for comparison. For NB, the Bernoulli, Gaussian, and Multinomial NB are applied. The SVM method achieves slightly better accuracy in the Bengali dataset than the Banglish dataset. The SVM, Bernoulli NB, Multinomial NB, Gaussian NB, and DT classifier achieved accuracy of 80%, 81%, 74%, 80%, and 79.5% for the Bengali dataset and 78%, 77.5%, 73%, 78%, and 78.5% for the Banglish dataset, respectively.
Camouflaged Object Segmentation Based on Fractional Edge Perception

Camouflaged object detection is a challenging task because of intrinsic similarity between background and foreground. In the existing models, many confusing edge features eventually lead to bad predictions. In this paper, we propose an Interactive Task Learning Network, in which an improved fractional-order differential operator is used to calculate the gradient intensity of the image. By calculating average gradient and spatial frequency of image, it can adaptively learn the fractional-order v and extract features from different directions. Experiments on three camouflaged datasets indicate that the proposed method is effective and progressiveness.
Technical Quality-Assisted Image Aesthetics Quality Assessment

Image aesthetics assessment (IAA) aims at predicting the perceived aesthetic quality of images. Intuitively, the technical quality of an image has significant impact on its aesthetic quality, e.g., an image with noticeable distortions is not likely to have very high aesthetic quality. However, this characteristic has rarely been considered when designing modern IAA models. Motivated by this, this paper presents a new Technical Quality-assisted multi-task deep network for image Aesthetic Quality assessment, dubbed TQ4AQ. Specifically, we first extract theme-aware general aesthetic features based on the attention mechanism. Meantime, hand-crafted technical quality features are extracted from aesthetic images. Then the general aesthetic features are utilized to predict the technical quality features and the aesthetic quality simultaneously, based on which technical quality features are integrated. By this means, the aesthetic features are empowered the capability of understanding technical distortions, and more comprehensive aesthetic feature representations are obtained for IAA. Extensive experiments demonstrate the advantage of the proposed TQ4AQ model over the state-of-the-arts.
Discriminative Activation of Information Is What You Need in Image Super-Resolution Transformer

Despite the significant progress made by Transformer in image super-resolution tasks, it has not effectively utilized prior knowledge in the image frequency domain and differentiated the processing of high-frequency and low-frequency information in the image. Previous studies on image super-resolution have shown that the high-frequency and low-frequency regions of the image exhibit distinct differences during the super-resolution process. In this paper, we propose a Discriminative Information Activation Super-Resolution Transformer (DIAST) to further improve the performance of Transformer in SISR tasks by discriminating high-frequency information from low-frequency information in images and discriminating cross-window information from inside-window information efficiently. Our results demonstrate that our method can further utilize the potential of the Transformer. The codes will be available at https://github.com/qyx1999/DIAST .
Multi-domain Information Fusion for Key-Points Guided GAN Inversion

In recent years, GAN inversion has emerged as a powerful technique for bridging the gap between real and fake image domains, and it has become increasingly important for enabling pre-trained GAN models for real image editing applications. However, current GAN inversion methods are limited by network parameters and model structures, and there is still room for improvement in accurate reconstruction and latent editing tasks. In this paper, we propose a two-stage model that fine-tunes a pre-trained Masked Autoencoder in the first stage and utilizes multi-layers information fusion to obtain an initial global latent code. We then use this latent code as global queries for the subsequent cross-attention-based fusion of local key patch, key point feature, and residual image information in the second stage, guided by facial landmarks. This allows our model to better embed images in the $$W+$$ space and perform related attribute editing, achieving better results than current state-of-the-art methods. We conduct extensive experiments to demonstrate the capabilities of our model, as well as the roles of relevant modules, and study the effects of different domain information on inversion.
Dynamic Neural Networks for Adaptive Implicit Image Compression

Compression with Implicit Neural Presentations (COIN) is a neural network image compression method based on multilayer perceptron (MLP). COIN encodes an image with an MLP that maps pixel positions to RGB values matching, the weights of the MLP are quantized to obtain a code stored as an image. However, this single implicit network structure performs generally when dealing with images of multiple complexities. In this paper, we propose a novel implicit dynamic neural network to process images in a dynamic and adaptive manner. Specifically, this paper uses the Sobel operator to divide the complexity of the images and use it as a criterion to select the network width and depth adaptively. To better fit the image features, this paper concludes with further quantification of the dynamic network parameters and storage matrices. Therefore, only some of the relevant network parameters with their storage matrices are required when storing the images. In training this dynamic network, this paper uses a meta-learning approach for the multi-image compression task. Experimental results show that our method outperforms COIN and JPEG in terms of image reconstruction results for the CIFAR-10 dataset.
Image Visual Complexity Evaluation Based on Deep Ordinal Regression

Image complexity is an important indicator in computer vision that helps people to more accurately evaluate and understand visual image information. Compared with traditional regression algorithms, ordinal regression methods are better suited to handling relationships and structures among ordinal data, providing more accurate reference for image complexity assessment. Currently, IC9600 dataset has made significant progress for providing the largest image complexity dataset, and ICNet provided a baseline model to evaluate the complexity score of images, but it neglects the ordinal property of complexity scores. This paper focuses on exploring a method to evaluate image complexity based on deep ordinal regression. We propose an evaluation model (ICCORN) that combines convolutional neural network ICNet and ordinal regression approach CORN. The model firstly extracts global semantic information and local detail information, and then considers ordinal relationship between complexity scores in the prediction process. The model demonstrates a high degree of correlation with human perception, as indicated by an increased Pearson correlation coefficient of 0.955. Furthermore, other evaluation metrics have also yielded favorable results.
Low-Light Image Enhancement Based on Mutual Guidance Between Enhancing Strength and Image Appearance

The existing low light image enhancement (LLIE) methods primarily aim at adjusting the overall brightness of the image, which are prone to produce the over-enhancement issue, such as over-exposure and edge halo. Therefore, it is desirable to improve the visibility of originally dark regions of an image, while preserving the naturalness of the originally bright regions. Based on this motivation, we propose a simple but effective mutual guidance module, which builds a mutual guidance process between a pixel-wise enhancing strength map and an edge-aware lightness map. Based on this module, the image appearance information such as illumination and structure can be effectively propagated onto the enhancing strength map. By integrating this module into the ZeroDCE++ model, the over-enhancement issue like over-exposure and edge halo can be greatly alleviated. We have conducted extensive experiments to validate the effectiveness and the superiority of our model. Compared with many state-of-the-art unsupervised and supervised LLIE methods, our model achieves a much better visual effect as it consistently keeps the naturalness during the enhancement process. Our model also has better or comparable performance than its counterparts in quantitative comparison with various image quality assessment metrics.
Misalignment Insensitive Perceptual Metric for Full Reference Image Quality Assessment

Full-reference (FR) image quality assessment (IQA) is crucial in the evaluation of restored images by comparing them with pristine-quality reference images, offering invaluable insights into the effectiveness of image restoration algorithms. Recently, with the advancement of generative adversarial networks (GANs), the GAN-based restoration algorithms demonstrate excellent restoration capability. Nevertheless, these algorithms introduce local spatial misalignment between the restored and the original reference images, posing a challenge for FR-IQA. To tackle this issue, we present a Misalignment Insensitive Perceptual Metric (MIPM) that strengthens the three components of FR-IQA, namely feature extraction, difference representation and quality regression. Specifically, a Vision Transformer-based network for global feature extraction is employed. Furthermore, MIPM utilizes Local Overlapping Wasserstein Difference (LOWD) and Channel Attention Block (CAB) to provide more accurate difference representation between the features of reference and distorted images in the spatial and channel dimensions, respectively. Lastly, a hybrid loss aids in regressing scores that align better with human subjective perception. Coupled with three key improvements, our MIPM exhibits superior performance over state-of-the-art approaches on five IQA datasets, LIVE, CSIQ, TID2013, KADID-10k, and PIPAL.
Semantically Guided Bi-level Adaptation for Cross Domain Crowd Counting

Visual crowd counting has played an important role in various practical applications. However, domain gap remains a major barrier preventing models trained on the source domain (e.g., training scenes) generalize well to the target domain (e.g., unseen testing scenes). Crowd semantic information are shown to be beneficial to assist crowd counting in supervised training settings, implying the close relationship between crowd density and semantics. Nevertheless, the potential of this powerful cue has bot been fully explored in the unsupervised domain adaptation (UDA) setting. Motivated by the observation that crowd density map share domain-invariant correspondence with the crowd segmentation map, we propose to adapt this correspondence correlation from the source domain to the target domain to address the domain gap. To this end, a semantically guided task correlation layer is introduced to extract the task correspondence map, whose coherence is enforced across domains by adversarial training. To drive the adaption of earlier hidden layers directly, we further align the task correspondence correlation upon intermediate-level outputs. Extensive experiments are conducted on three benchmark datasets. The performances of our method either surpass or are on par with the counterparts, demonstrating the effectiveness of the proposed approach for cross-domain crowd counting.
DecTrans: Person Re-identification with Multifaceted Part Features via Decomposed Transformer

Utilizing part-level features provides a more detailed representation, leading to improved results in person re-identification (ReID). Yet existing works either use external tasks like pose estimation or struggle to define part features, which limit the model’s learning capability. In this work, we propose the Decomposed Transformer (DecTrans), a transformer-based person ReID framework which exploits multifaceted part features. In particular, DecTrans extracts local features using the Vision Transformer (ViT) and then maps them into latent parts through a novel Token Decomposition (TD) layer. In the TD layer, soft clustering of ViT tokens forms clusters, and each token is decomposed into components based on its similarity to all cluster centroids. Token components referencing the same cluster are then regrouped to produce part features, thereby retaining more feature details. To ensure tokens from different pedestrians but referring to the same part are sufficiently clustered together, we propose to remove id information from tokens before clustering. Besides, we also propose a simple yet efficient data augmentation named Image Graying, which has been experimentally validated when used in conjunction with the TD layer. The DecTrans achieves remarkable performance, e.g., mAP and Rank1 of $$ 70.8\% \& 87.1\%$$ , and $$ 61.6\% \& 67.7\%$$ on MSMT17 and Occluded-Duke, significantly outperforming state-of-the-arts.
A Dynamic Tracking Framework Based on Scene Perception

While recent large models have greatly improved tracking performance, not all scenes require a large and complex network. Dynamic networks can adapt the architecture to different inputs, leading to notable accuracy and computational efficiency. However, existing dynamic architectures and decision mechanisms designed for classification are not applicable to the tracking task. This paper proposes a dynamic tracking framework based on scene perception, named DynamicTrack. We classify tracking scenes into easy and hard categories, and propose a dynamic architecture with an easy-hard dual-branch to handle different scenes respectively. Unlike previous works in classification that selectively prune a subset of the backbone, complete execution of the entire backbone is necessary for tracking. Hence, we maintain two complete transformer backbones for the dual branches and vary the number of input tokens to achieve modeling at different granularities. Then, we propose a scene router that automatically selects the optimal branch for each input frame. The router directly assesses the scene complexity of features extracted by the easy branch for decision-making, without relying on the tracking head output. This enhances decision efficiency during dynamic inference. Moreover, we introduce two techniques that benefit DynamicTrack optimization, namely, the Gumbel-Softmax trick and cross-branch transmission (CBT). The former increases the stochasticity of decisions and prevents mode collapse into trivial solutions. The latter establishes information transmission between the two branches, facilitating discriminative power and learning efficiency. Extensive experiments on four benchmarks demonstrate that the proposed DynamicTrack achieves SOTA performance and accuracy-speed trade-offs.
Reliable Boundary Samples-Based Proxy Pairs for Unsupervised Person Re-identification

Contrastive learning methods based on the memory bank have shown promising results for unsupervised person re-identification. However, most methods maintain a uni-proxy for each cluster in the memory that only describes the average information but can not represent the intra-class variation. As a result, contrastive learning based on the uni-proxy cannot effectively guide the model to reduce the variation. To address this issue, we maintain a proxy pair for each cluster updated by the least similar boundary sample pair since they concretely reveal the intra-class variation of the cluster. Through contrastive loss, the proxy updated based on one boundary sample generates strong pulls to another one and its surrounding samples due to the low similarity, and two proxies collaboratively form bidirectional strong pulls to effectively reduce intra-class variance. To mitigate the impact of boundary samples being noisy, we further propose local-global consistency guided label refinement, which utilizes local fine-grained cues to select reliable samples with high overlap in the local and global feature neighborhoods. Comprehensive experiments on Market-1501 and MSMT17 demonstrate that the proposed method surpasses state-of-the-art approaches.
Context-FPN and Memory Contrastive Learning for Partially Supervised Instance Segmentation

Partially supervised instance segmentation aims to segment objects on both limited seen categories and novel unseen categories (without annotated masks), thereby eliminating expensive demands of mask annotation for new categories. Existing work mainly utilize the pipeline model of detection first and then segmentation, and explores how to provide more discriminative regions of interest for the class-agnostic mask head, but these methods do not perform well when faced with complex scenes. In this work, we propose a novel method, named CCMask, that combines Context Feature Pyramid Network (Context-FPN) and Memory Contrastive Learning Head (MCL Head) to achieve effective class-agnostic mask segmentation. Specifically, we introduce a Context-FPN to obtain context-rich feature map via context extraction module, which will benefit the subsequent task heads. In the MCL Head, we employ foreground/background query memory queue to store queries from recent training batches, this helps the MCL Head learns the general concepts of foreground and background. These strategies collectively contribute to improve the discrimination between foreground and background. Exhaustive experiments on COCO dataset demonstrate that our method achieves state-of-the-art results.
MemDNet: Memorizing More Exogenous Information to Dehaze Natural Hazy Image

Model trained on synthetic datasets are difficult to obtain good results in the real world due to domain shift. Previous works have been devoted to mining the potential of existing natural datasets to obtain better results, but the number of images in the dataset is ultimately limited. In this paper, we propose MemDNet, a two-branch dehazing network that incorporates an atmospheric scattering model. Our approach utilizes a residual network and a U-shaped network to estimate the corresponding parameters. To alleviate the issue of insufficient natural hazy image pairs, we propose a generalized memory branch that incorporates information from exogenous images. This allows the network to memorize the fine details of the exogenous images, thereby enhancing its ability to recover image colors. Extensive experimental results demonstrate that our proposed memory branch is general and effective, and our method outperforms the state-of-the-art methods on real datasets.
TU-Former: A Hybrid U-Shaped Transformer Network for SAR Image Denoising

In order to obtain a better synthetic aperture radar (SAR) image noise suppression effect, we combine Convolutional Neural Network (CNN) and Transformer network to construct a U-shaped hybrid Transformer (TU-former) for SAR image denoising. The encoder of the TU-former network consists of convolution, residual and self-attentive mechanisms. To avoid large-scale training of the model, TU-former first uses convolutional structure and residual structure for shallow feature extraction of images. Subsequently, TU-former performs long-term dependencies by using the self-attention mechanism of the Transformer block to further extract the deep features of the image. Finally, TU-former sums the output of the decoder with the input noise image to obtain the final denoised image. Compared with the state-of-the-art SAR image denoising algorithm, the proposed algorithm not only improves in each objective index but also shows great advantages in the visual effect after denoising.