Reviewerly/exHarmony huggingface dataset:
Look for Tiny Rosettes from Drone Imagery

In this paper, we present our recent research work on applying the Generative adversarial networks(GAN)-based super-resolution method to upscale low-resolution airborne RGB images for precise weed detection. Often better detection accuracy requires high-resolution(HR) imagery for remote weed detection using drones. However, HR imagery means narrow ground coverage per survey flight. So this paper is to find the balance between the image resolution and the remote detection accuracy for better cost-efficiency. Our proposed super-resolution model is well trained by selected training samples of low-resolution(LR) and HR image pairs. Our model shows better performance in improving the quality of high-resolution remote sensing images and achieving accurate detection. In addition, we reviewed several super-resolution techniques, including interpolation, neighbor embedding, sparse coding, random forest, and CNN-based methods. Hence, this paper presents a cost-effective use of low-resolution airborne color images paired with a super-resolution approach to detect weeds in complex landscapes.
Long-tailed Detection Based on Multi-Expert Aggregation

Long-tailed data distribution is a common phenomenon observed in nature. However, this distribution poses challenges for neural network training, as it performs well on samples with head categories, but not so well on tail categories - precisely what requires attention in certain scenes. Although existing methods involve re-sampling data or designing loss functions, they often fail to adapt well to changing data distributions. In this article, we propose a long-tailed detection approach based on multi-expert aggregation (MEA). This method employs multi-expert networks to learn about head, medium, and tail categories from a fixed long-tailed distribution. Different data are modelled using different loss designs, and the results of multi-expert networks are aggregated to achieve good accuracy for both head and tail categories. The effectiveness of this method has been verified on LVIS.
Improving Zero-Shot Semantic Segmentation using Dynamic Kernels

Zero-shot Semantic Segmentation (ZS3) is a challenging task that segments objects belonging to classes that are completely unseen during training. An established and intuitive approach is to formulate ZS3 as a combination of two subtasks where, at first, mask proposals are generated and then each pixel in those regions is assigned a class label. Most of the existing works struggle to generate masks with high generalization capability, which results in significant underperformance in unseen classes. In this connection, we propose the use of ‘Dynamic Kernels’ to help a ZS3 model better ‘understand’ the objects in the training phase by taking advantage of their inherent inductive biases to generate better mask proposals. They act as specialized agents that are updated based on their corresponding contents from the seen classes and then utilize that knowledge to understand unseen objects. The proposed pipeline also leverages the Contrastive Language-Image Pre-Training (CLIP) architecture to perform segment classification which further improves the generalization performance by exploiting its cross-modal training. Dynamic kernels go hand-in-hand with CLIP since it is able to process the granularity of CLIP from image level to pixel level resulting in performance improvement for both the seen and unseen classes. Our method, ‘Zero-Shot dynamic Kernel Network’ (ZSK-Net), outperforms the previous works by achieving +6.4h IoU on the Pascal VOC dataset. It also achieves state-of-the-art result on the COCO-Stuff dataset by +0.9h IoU on a single prompt setting.
Diving into Continual Ultra-fine-grained Visual Categorization

Recent advance in ultra-fine-grained visual categorization (ultra-FGVC) has significantly boosted the capability of deep neural networks for ultra-FGVC tasks. However, building models for continually learning to recognize increasing ultra-fine-grained categories is still under-explored. This limits the application of ultra-FGVC techniques in real-world production. To this, we take the first attempt for continual ultra-FGVC. By evaluating existing continual learning methods on the constructed continual ultra-FGVC benchmark, we observe that the main bottleneck lies in the limited model plasticity for incrementally adapting to new tasks. This can be caused by excessive anti-forgetting constraints as the difficult ultra-FGVC task requires substantial update of parameters, and over-fitting on early tasks given that the ultra-fine-grained categories are with very few training samples. To tackle these problems, we propose a joint self-supervised learning and prompting model. The prompt-based continual learning framework offers proper anti-forgetting operation by fixed pretrained vision transformer and adaptive prompt selection. By jointly optimizing the learnable prompts with an adversarial self-supervised loss, the over-fitting on each continual learning task is mitigated. Extensive experiments demonstrate that the proposed method outperforms existing continual learning methods on the challenging continual ultra-FGVC problem.
TeleGhost: Asymmetric Telepresence System Using AR and VR Avatars in a Shared Real Space

This study proposes a simplified telepresence system based on extended reality for increasing the number of remote-able tasks. The system provides an immersive virtual environment that duplicates remote real spaces to enable communication among system users in different geographic locations through their digital avatars. The motion of the avatars corresponds with that of the mobile devices manipulated by the system users, and the avatars' position and orientation in the real space represent the viewpoint and viewing angle of the users. The three experiments showed that the proposed system creates a high-fidelity immersive environment in a short period of time and positively supports users' remote collaboration in the virtualized real space. The system successfully demonstrated its potential to contribute to the expansion of remotely enabled operations.
Towards Fully Decoupled End-to-End Person Search

End-to-end person search aims to jointly detect and re-identify a target person in raw scene images with a unified model. The detection task unifies all persons while the re-id task discriminates different identities, resulting in conflict optimal objectives. Existing works proposed to decouple end-to-end person search to alleviate such conflict. Yet these methods are still sub-optimal on one or two of the sub-tasks due to their partially decoupled models, which limits the overall person search performance. In this paper, we propose to fully decouple person search towards optimal person search. A task-incremental person search network is proposed to incrementally construct an end-to-end model for the detection and re-id sub-task, which decouples the model architecture for the two sub-tasks. The proposed task-incremental network allows task-incremental training for the two conflicting tasks. This enables independent learning for different objectives thus fully decoupled the model for personsearch. Comprehensive experimental evaluations demonstrate the effectiveness of the proposed fully decoupled models for end-to-end person search.
High Dimensional Exact $K$ Nearest Neighbor Search Using Lower Bound Technique and Parallel Computing

For the past decade, the K Nearest Neighbor (K-NN) search in high dimensional space has been explored extensively. Considerable theoretical and practical algorithms to accelerate approximate K-NN search have been presented. The approximate methods can improve searching efficiency and achieve satisfactory performance. Nevertheless, they are inherently approximation approaches and are not guaranteed to yield exact solutions. To obtain the K-NN results over high dimensional datasets efficiently while guaranteeing the same results as the linear search is a challenging task, attracting a large number of scholars. To this end, in this paper, we focus on improving the exact K-NN search efficiency over high dimensional datasets, and present a framework named LBPC to solve this problem. The lower bound based method and parallel computing are combined in LBPC to accelerate the exact K-NN search. In LBPC, the whole K-NN search task is divided into some sub-tasks and these sub-tasks can be conducted concurrently using the lower bound based method. The LBPC scheme allows users to utilize any lower bound to accelerate the exact K-NN query. In this paper, we use the segment mean to construct the lower bound and provide the theoretical analysis to show its computational efficiency and lower bound property. Various experiments are conducted to analyze the efficiency of LBPC, and the experimental results validate its effectiveness.
Revisiting Image Reconstruction for Semi-supervised Semantic Segmentation

Autoencoding, which aims to reconstruct the input images through a bottleneck latent representation, is one of the classic feature representation learning strategies. While it has proven effective as an auxiliary task for semi-supervised learning, its popularity has waned with the advent of more sophisticated methods in recent years. In this paper, we revisit the idea of using image reconstruction as the auxiliary task and incorporate it with a modern semi-supervised semantic segmentation framework. Surprisingly, we discover that such an old idea in semi-supervised learning can produce results competitive with state-of-the-art semantic segmentation algorithms. By visualizing the intermediate layer activations of the image reconstruction module, we show that the feature map channels exhibit a strong correlation with semantic concepts. This observation explains why joint training with the reconstruction task proves beneficial for the segmentation task. Motivated by our observation, we further proposed a modification to the image reconstruction task, aiming to further disentangle the object clue from the background patterns. From experiment evaluation on various datasets, we show that using reconstruction as auxiliary loss can lead to consistent improvements in various datasets and methods. The proposed method can further lead to significant improvement in object-centric segmentation tasks.
REIN: Reusing ImageNet to Improve Open-set Object Detection

There has been a notable advancement in the task of object detection over the last few years. However, recent studies reveal that modern object detectors suffer a considerable performance drop in an open-set setting and tend to incorrectly detect unknown objects and mislabel them as belonging to known categories with a high degree of confidence. We observe that the ‘catch-all background’ approach with limited training data from the detection dataset is insufficient to equip a detector with the ability to reject all unknown objects as background. Based on this observation, we propose to augment the detection dataset by reusing a subset of ImageNet that does not contain any object categories predefined by the detection task. The augmented training on a held-in background dataset allows the detector to reject unknown object instances as background instead of mislabelling them. Our experiments demonstrate that the proposed simple yet effective REusing ImageNet (REIN) technique improves the performance of object detectors in an open-set setting. Given the promising results, we hope that our method can encourage our community to pay more attention to open-set object detection and to rethink the data augmentation in detection.
Text-image guided Diffusion Model for generating Deepfake celebrity interactions

Deepfake images are fast becoming a serious concern due to their realism. Diffusion models have recently demonstrated highly realistic visual content generation, which makes them an excellent potential tool for Deepfake generation. To curb their exploitation for Deepfakes, it is imperative to first explore the extent to which diffusion models can be used to generate realistic content that is controllable with convenient prompts. This paper devises and explores a novel method in that regard. Our technique alters the popular stable diffusion model to generate a controllable high-quality Deepfake image with text and image prompts. In addition, the original stable model lacks severely in generating quality images that contain multiple persons. The modified diffusion model is able to address this problem, it add input anchor image’s latent at the beginning of inferencing rather than Gaussian random latent as input. Hence, we focus on generating forged content for celebrity interactions, which may be used to spread rumors. We also apply Dreambooth to enhance the realism of our fake images. Dreambooth trains the pairing of center words and specific features to produce more refined and personalized output images. Our results show that with the devised scheme, it is possible to create fake visual content with alarming realism, such that the content can serve as believable evidence of meetings between powerful political figures.
Enhancing Image Representation and Compression: An Innovative Nz-Nqer Framework With Block Truncation Quantum Coding

Efficiently representing and compressing large-scale images in quantum circuits remains a challenge due to increasing circuit complexity. This paper proposes an innovative approach called NZ-NEQR (Non-Zero Novel Quantum Representation) to tackle this issue. Eliminating zero connections from the existing NEQR approach significantly reduces the complexity of state label preparation. The proposed NZ-NEQR approach achieves remarkable improvements in both image representation and compression, leading to a substantial reduction in required bits per pixel. To further enhance compression, Block Truncation Coding (BTC) is adopted as a compression scheme, which effectively truncates images into two labels suitable for quantum compression and reconstruction. Notably, the proposed method requires fewer bits per pixel compared to existing NEQR approaches. Additionally, NZ-NEQR combined with BTC employs only 5 qubits for representing a $1024 \times 1024$ image, and compression using a $4 \times 4$ quantum BTC block represents a significant advancement. Computational results demonstrate the superior effectiveness of the proposed NZ-NEQR approach combined with BTC for quantum image representation and compression, surpassing the performance of NEQR and JPEG approaches.
A Feature Enhanced Scale-adaptive Convolutional Network for Ship Detection in Maritime Surveillance<sup>*</sup>

Accurate ship detection is of great importance in intelligent maritime surveillance and numerous methods have been proposed over the years. However, due to the interference caused by water dynamics and the different sizes of various ships, existing methods are often prone to produce a large number of false alarms or missing detection. Therefore, in this paper, we propose a feature enhanced scale-adaptive convolutional network for effective ship detection in complex maritime scenarios. The proposed model is built on a one-stage deep convolutional network based detector and consists of feature enhancement modules, scale-adaptive modules, and an improved detection head. First, a feature enhancement module is proposed and incorporated into multiple convolutional layers. The statistical information calculated from both channel and spatial domains is fused to alleviate irregular noise and increase the weight of ship feature representation. In addition, the receptive field of deep layer is enlarged via a scale-adaptive module devised, so that the robustness of the proposed model over varying ship scales could be enhanced. Finally, an improved detection head is further applied on different feature maps. A centerness branch is introduced following the combination of classification and regression branches to suppress low-quality bounding boxes for more accurate ship detection. Experimental results on two real-world maritime surveillance datasets demonstrate that our proposed model is able to improve ship detection accuracy with less computational cost against a number of state-of-the-art ones.
Domain Adaptation for Edge Detectors Using Lightweight Networks

Edge Detectors are one of the most fundamental tools used in Computer Vision applications. Using deep learning, state-of-the-art (SOTA) models are able to produce sharp, fine edges, mirroring human-level performance. However, most of these SOTA models are trained solely on color images. We experimentally determined that this skewed data severely hinders performance on images from other domain spaces. In this paper, we propose a way to adapt SOTA models to single-channel inputs from other domains with minimal overhead, using small-scale feed-forward networks. Our model, which serves as an additional layer to existing SOTA models, greatly boosts their performance on out-of-domain data, allowing the edge detector to generalize to new domain spaces without undergoing extensive, resource-or data-heavy retraining. Using a single off-the-shelf GPU and a small 30-image dataset, we were able to train this low-complexity (less than 13k parameters) model in half an hour to boost the F-score of the produced edgemaps for chest x-rays by over 0.2.
Plug n’ Play Channel Shuffle Module for Enhancing Tiny Vision Transformers

Vision Transformers (ViTs) have demonstrated remarkable performance in various computer vision tasks. However, the high computational complexity hinders ViTs’ applicability on devices with limited memory and computing resources. Although certain investigations have delved into the fusion of convolutional layers with self-attention mechanisms to enhance the efficiency of ViTs, there remains a knowledge gap in constructing tiny yet effective ViTs solely based on the self-attention mechanism. Furthermore, the straightforward strategy of reducing the feature channels in a large but outperforming ViT often results in significant performance degradation despite improved efficiency. To address these challenges, we propose a novel channel shuffle module to improve tiny-size ViTs, showing the potential of pure self-attention models in environments with constrained computing resources. Inspired by the channel shuffle design in ShuffleNetV2 [1], our module expands the feature channels of a tiny ViT and partitions the channels into two groups: the Attended and Idle groups. Self-attention computations are exclusively employed on the designated Attended group, followed by a channel shuffle operation that facilitates information exchange between the two groups. By incorporating our module into a tiny ViT, we can achieve superior performance while maintaining a comparable computational complexity to the vanilla model. Specifically, our proposed channel shuffle module consistently improves the top-1 accuracy on the ImageNet-IK dataset for various tiny ViT models by up to 2.8%, with the changes in model complexity being less than 0.03 GMACs.
AI-Based Mobile Paper Grading: Trends and Challenges

Many universities, schools, and other educational institutions embraced distance learning due to the Covid-19 Pandemic in previous years. Because some instructors are unfamiliar with it, creating exams for students using online platforms is not always simple. Therefore, it takes a lot of time and effort to create online assessments and manually correct them. Additionally, grading many tests can burden instructors, especially if they are unfamiliar with online technology. Exams can be made as multiple-choice questions, which can help instructors overcome human error in judging exam papers, as a temporary fix for this rather than creating lengthy questions. This, however, is not a long-term fix because instructors still have to spend time grading the multiple-choice questions. Artificial intelligence can therefore be quite useful in this case. This paper offers an overview of notable implementations that autonomously grade and compute exam paper outcomes using vision-based and Artificial Intelligence-based mobile approaches. Additionally, different techniques offer a variety of methods, each with its benefits and limitations. In this paper, each strategy's methodologies, as well as its results performance, are discussed. Also, it discusses which approach is more appropriate for an automated Grading system. Based on the results reviewed from different sources, it was found that Artificial Intelligence-based approaches and vision-based approaches are accurate in detecting the objectives of this project. However, the AI-based approach has more efficiency due to the combination of various models, resulting in higher accuracy. Finally, both approaches' advantages and limitations are described throughout the report.
Effects of Different Levels of Self-Representation on Spatial Awareness, Self-Presence and Spatial Presence During Virtual Locomotion

Recently, there has been growing interest in investigating the effects of self-representation on user experience and perception in virtual environments. However, few studies investigated the effects of levels of body representation (full-body, lower-body and viewpoint) on locomotion experience in terms of spatial awareness, self-presence and spatial presence during virtual locomotion. Understanding such effects is essential for building new virtual locomotion systems with better locomotion experience. In the present study, we first built a walking-in-place (WIP) virtual locomotion system that can represent users using avatars at three levels (full-body, lower-body and viewpoint) and is capable of rendering walking animations during in-place walking of a user. We then conducted a virtual locomotion experiment using three levels of representation to investigate the effects of body representation on spatial awareness, self-presence and spatial presence during virtual locomotion. Experimental results showed that the full-body representation provided better virtual locomotion experience in these three factors compared to that of the lower-body representation and the viewpoint representation. The lower-body representation also provided better experience than the viewpoint representation. These results suggest that self-representation of users in virtual environments using a full-body avatar is critical for providing better locomotion experience. Using full-body avatars for self-representation of users should be considered when building new virtual locomotion systems and applications.
Ensuring Privacy and Security of IoT Networks Utilizing Blockchain and Federated Learning

The Internet of Things (IoT) has become a game-changing technology, bridging the gap between the real and virtual worlds and allowing for smooth data transfer and communication between linked objects. The other two potential technologies are blockchain (BC) and artificial intelligence (AI), whose application areas are incredibly diverse and which may perform best when combined. Since some traditional machine learning (ML) techniques have limitations, this article proposed using distributed machine learning techniques such as federated learning and blockchain to build a more reliable and secure IoT network that will be better protected and less susceptible to outside intrusions. As an alternative to centralized cloud storage, we also recommended using decentralized data storage techniques like the InterPlanetary File System (IPFS) and Hyperledger Fabric (HLF). Additionally, as a proof of concept, we deployed our model using Ethereum Smart Contracts (SC). Using the well-known cybersecurity dataset known as Edge-IIoTset, we utilized both centralized and federated machine learning models to evaluate the efficiency of the suggested approach. The experimental results and successful deployment of Smart Contracts demonstrate that employing Blockchain and distributed storage systems is preferable for safeguarding IoT networks.