Reviewerly/exHarmony huggingface dataset:
LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation

In the text-to-image generation field, recent remarkable progress in Stable Diffusion makes it possible to generate rich kinds of novel photorealistic images. However, current models still face misalignment issues (e.g., problematic spatial relation understanding and numeration failure) in complex natural scenes, which impedes the high-faithfulness text-to-image generation. Although recent efforts have been made to improve controllability by giving fine-grained guidance (e.g., sketch and scribbles), this issue has not been fundamentally tackled since users have to provide such guidance information manually. In this work, we strive to synthesize high-fidelity images that are semantically aligned with a given textual prompt without any guidance. Toward this end, we propose a coarse-to-fine paradigm to achieve layout planning and image generation. Concretely, we first generate the coarse-grained layout conditioned on a given textual prompt via in-context learning based on Large Language Models. Afterward, we propose a fine-grained object-interaction diffusion method to synthesize high-faithfulness images conditioned on the prompt and the automatically generated layout. Extensive experiments demonstrate that our proposed method outperforms the state-of-the-art models in terms of layout and image generation. Our code and settings are available at https://layoutllm-t2i.github.io/.
A Capture to Registration Framework for Realistic Image Super-Resolution in the Industry Environment

The acquisition and processing of visual data in industrial environments are of paramount importance. High-resolution (HR) images offer superior clarity and richer textural detail compared to low-resolution (LR) images. On the one hand, owing to the incorporation of richer information, HR images demonstrate substantially enhanced performance compared to LR images in downstream applications, such as anomaly detection. On the other hand, they provide valuable insights to designers and quality inspectors who require a detailed understanding of the images. Currently, the majority of research on super-resolution focuses on natural scenes such as cities and fields, however, the development of datasets for industrial scenes is still in its infancy. To address the image distortion in building realistic LR-HR image pairs in the industry environment, we design a capture to registration framework. It consists of the standard imaging system, physical calibration of the imaging system, as well as the rigid to elastic registration of the LR-HR image pairs. Thus, we build the first realistic industrial sence super-resolution dataset (IndSR), comprises of 50 sets of calibrated images with three scale factors and five typical defects. To benchmark IndSR, we employ quantitative, qualitative, and task-oriented studies to evaluate the representative super-resolution and anomaly detection methods. Besides, we systematically investigate and discuss the performances and results of the existing SISR methods to advance research in the field of super-resolution in industry environment. The IndSR dataset can be available from https://byw4ng.github.io/IndSR/.
AcFormer: An Aligned and Compact Transformer for Multimodal Sentiment Analysis

Multimodal Sentiment Analysis (MSA) is a popular research topic aimed at utilizing multimodal signals for understanding human emotions. The primary approach to solving this task is to develop complex fusion techniques. However, the heterogeneity and unaligned nature between modalities pose significant challenges to fusion. Additionally, existing methods lack consideration for the efficiency of modal fusion. To tackle these issues, we propose AcFormer, which contains two core ingredients: i) contrastive learning within and across modalities to explicitly align different modality streams before fusion; and ii) pivot attention for multimodal interaction/fusion. The former encourages positive triplets of image-audio-text to have similar representations in contrast to negative ones. The latter introduces attention pivots that can serve as cross-modal information bridges and limit cross-modal attention to a certain number of fusion pivot tokens. We evaluate AcFormer on multiple MSA tasks, including multimodal emotion recognition, humor detection, and sarcasm detection. Empirical evidence shows that AcFormer achieves the optimal performance with minimal computation cost compared to previous state-of-the-art methods. Our code is publicly available at https://github.com/dingchaoyue/AcFormer.
An NLP-based Malicious Destruction Recognition on Web-based Online Articles using Machine Learning

In today's digital era, online articles have become the primary source of knowledge acquisition, replacing traditional library research. However, the ease of online editing by web users, especially on platforms like Wikipedia, poses a significant challenge of detecting and mitigating malicious destruction in web-based articles. Existing studies have employed web-mining techniques and cross-language learning with the Gradient Tree machine learning algorithm to identify vandalism on specific web contents, achieving up to 80% accuracy. To address this pressing issue, our research proposes a text-based online web articles vandalism detector using a Decision Tree algorithm. Building upon previous studies, we aim to enhance the accuracy performance of the proposed Vandalism Detector System on Wikipedia pages by implementing both basic and advanced machine learning pre-processing techniques with a Logistic Regression learning model. To create a reliable dataset, we cleaned the Wiki Data and merged it with the current Wikidata Vandalism Corpus, comprising four CSV files (edits.csv, annotators.csv, annotations.csv, and and gold-annotators.csv). The resulting cleaned dataset, termed “wikivand.csv,” served as the primary source dictionary, prepared using Python libraries and the Label Binarizer technique. Our proposed model was then trained to predict Yes or No labels, enabling it to identify whether a web article contains vandalism content.
Finding Efficient Pruned Network via Refined Gradients for Pruned Weights

With the growth of deep neural networks (DNN), the number of DNN parameters has drastically increased. This makes DNN models hard to be deployed on resource-limited embedded systems. To alleviate this problem, dynamic pruning methods have emerged, which try to find diverse sparsity patterns during training by utilizing Straight-Through-Estimator (STE) to approximate gradients of pruned weights. STE can help the pruned weights revive in the process of finding dynamic sparsity patterns. However, using these coarse gradients causes training instability and performance degradation owing to the unreliable gradient signal of the STE approximation. In this work, to tackle this issue, we introduce refined gradients to update the pruned weights by forming dual forwarding paths from two sets (pruned and unpruned) of weights. We propose a novel Dynamic Collective Intelligence Learning (DCIL) which makes use of the learning synergy between the collective intelligence of both weight sets. We verify the usefulness of the refined gradients by showing enhancements in the training stability and the model performance on the CIFAR and ImageNet datasets. DCIL outperforms various previously proposed pruning schemes including other dynamic pruning methods with enhanced stability during training. The code is provided in Github.
Fine-Tuning Pre-Trained Language Model for Urgency Classification on Food Safety Feedback

Singapore Food Agency (SFA) receives hundreds of food safety feedback reports regarding Singapore's food safety every week, which can be time-consuming and costly to manage them. Prompt response to urgent food safety feedback is crucial in cases of food poisoning outbreaks. Automating the task of feedback urgency classification can help SFA officers to prioritise feedback efficiently and effectively, so that they can respond quickly to urgent cases. In this paper, we propose an approach to fine-tune a pre-trained language model based on BERT, which is a sequence classification task, for feedback urgency classification. In addition, to speed up the labeling of task-specific feedback data, we also propose a process that utilizes the zero-shot text classification and decision tree methods for data labeling with minimal human supervision. We have conducted experiments to evaluate the proposed fine-tuned BERT model and compared with the DistilBERT and XLNet models for the feedback urgency classification task. The performance results show that the proposed fine-tuned BERT model has achieved promising performance and outperformed the fine-tuned DistilBERT and XLNet models by 7% and 5%, respectively in macro-averaged F1-score.
Rethinking Neighborhood Consistency Learning on Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) involves predicting unlabeled data in a target domain by using labeled data from the source domain. However, recent advances in pseudo-labeling (PL) methods have been hampered by noisy pseudo-labels that diminish the local discriminativeness of the target structure. Although neighborhood-based PL can help preserve the local structure, it also risks assigning the whole local neighborhood to the wrong semantic category. To address this issue, we propose a novel framework called neighborhood consistency learning (NCL) that operates at both the semantic and instance levels and features a new consistency objective function. Specifically, our objective function aims to promote semantic consistency in the target neighborhood by computing the correlation matrix between the target samples and their neighborhood aggregation over a batch and matching the correlation matrix to an identity matrix. Importantly, our approach allows the target neighborhood to receive gradients from several potential positive categories instead of just one certain category. Our extensive experiments on UDA benchmarks demonstrate the effectiveness of NCL over other state-of-the-art PL-based methods.
A Dense Small Object Detection Algorithm Based on a Global Normalization Attention Mechanism

To address the challenges of detecting a large number of objects and a high proportion of small objects in aerial drone imagery, we proposed an aerial dense small object detection algorithm called Global Normalization Attention Mechanism You Only Look Once (GNYL) based on the Global Normalization Attention Mechanism. In the backbone network of GNYL, we embedded a GNAM (Global Normalization Attention Mechanism) that explores channel attention features and spatial attention features from input features in a concatenated manner. It utilizes batch normalization’s scale factors to suppress irrelevant channels or pixels. Furthermore, the spatial attention sub-module introduces a three-dimensional arrangement with a multi-layer perceptron to reduce information loss and amplify global interaction representation. Finally, the computed attention weights are weighted to form the global normalized attention weights, which increases the utilization of effective information in input feature channels and spatial dimensions. We have optimized the backbone network, feature enhancement network, and detection heads to improve detection accuracy while ensuring a lightweight detection network. Specifically, we have added a small object detection layer to enhance the localization accuracy for the abundant small objects in aerial imagery. The algorithm’s performance was evaluated using the publicly available VisDrone2019 dataset. Compared to the baseline network YOLOv8l, GNYL achieved a 7.2% improvement in mAP0.5 and a 5.0% improvement in mAP0.95. Compared to CDNet, GNYL showed a 14.5% improvement in mAP0.5 and a 9.1% improvement in mAP0.95. These experimental results demonstrate the strong practicality of the GNYL object detection network for detecting dense small objects in the aerial imagery captured by unmanned aerial vehicles.
Exploring Hyperspectral Histopathology Image Segmentation from a Deformable Perspective

Hyperspectral images (HSIs) offer great potential for computational pathology. However, limited by the spectral redundancy and the lack of spectral prior in popular 2D networks, previous HSI based techniques do not perform well. To address these problems, we propose to segment HSIs from a deformable perspective, which processes different spectral bands independently and fuses spatiospectral features of interest via deformable attention mechanisms. In addition, we propose Deformable Self-Supervised Spectral Regression (DF-S3R), which introduces two self-supervised pre-text tasks based on the low rank prior of HSIs enabling the network learning with spectrum-related features. During pre-training, DF-S3R learns both spectral structures and spatial morphology, and the jointly pre-trained architectures help alleviate the transfer risk to downstream fine-tuning. Compared to previous works, experiments show that our deformable architecture and pre-training method perform much better than other competitive methods on pathological semantic segmentation tasks, and the visualizations indicate that our method can trace the critical spectral characteristics from subtle spectral disparities. Code will be released at https://github.com/Ayakax/DFS3R.
Enhanced Image Deblurring: An Efficient Frequency Exploitation and Preservation Network

Most of these frequency-based deblurring methods mainly have two major limitations: (1) insufficient exploitation of frequency information, (2) inadequate preservation of frequency information. In this paper, we propose a novel Efficient Frequency Exploitation and Preservation Network (EFEP) to address these limitations. Firstly, we propose a novel Frequency-Balanced Exploitation Encoder (FBE-Encoder) to sufficiently exploit frequency information. We insert a novel Frequency-Balanced Navigator (FBN) module in the encoder, which establishes a dynamic balance that adaptively explores and integrates the correlations between frequency features and other features presented in the network. And it also can highlight the most important regions in frequency features. Secondly, considering the limitation that frequency information is inevitably lost in deep network architectures, we present an Enhanced Selective Frequency Decoder (ESF-Decoder) that not only effectively reduces spatial information redundancy, but also fully explores the different importance of various frequency information to ensure the supplement of valid spatial information and weaken the invalid information. Thirdly, each encoder/decoder block of the EFEP consists of multiple Contrastive Residual Blocks (CRBs), which are designed to explicitly compute and incorporate feature distinctions. Powered by the above designs, our EFEP outperforms state-of-the-art models on both quantitative and qualitative evaluations.
Empirical Comparison of Energy Efficiency Between 3-Tier and High-Speed 3-Tier Network Topologies

The high levels of energy use for cloud computing affect carbon emissions and climate change. This research thoroughly compares the energy utilisation of conventional and high-speed three-tier data centres (DCs). The green cloud simulator (GCS) was used to compare network configurations and energy-saving algorithms for each cloud's energy consumption in 10 trials. The power saver schedule algorithm used the least server energy, while the green and RR schedule algorithms used more. Despite the testing of several energy conservation approaches, core switch and aggregate switch energy usage remained unchanged. Although the high-speed three-tier DC architecture used more energy than the standard design, this is a key difference between the two network topologies. The authors discuss the power saver schedule algorithm's benefits for server energy utilisation. This study shows how three-tier DC architecture can save money by reducing core switch and aggregation switch energy use. These evaluations should include the make span coefficient, and future research should focus on DC transform energy-saving techniques.
A Hierarchical Deep Video Understanding Method with Shot-Based Instance Search and Large Language Model

Deep video understanding (DVU) is often considered a challenge due to the aim of interpreting a video with storyline, which is designed to solve two levels of problems: predicting the human interaction in scene-level and identifying the relationship between two entities in movie-level. Based on our understanding of the movie characteristics and analysis of DVU tasks, in this paper, we propose a four-stage method to solve the task, which includes video structuring, shot based instance search, interaction & relation prediction and shot-scene summary & Question Answering (QA) with ChatGPT. In these four stages, shot based instance search allows accurate identification and tracking of characters at an appropriate video granularity. Using ChatGPT in QA, on the one hand, can narrow the answer space, on the other hand, with the help of the powerful text understanding ability, ChatGPT can help us answer the questions by giving background knowledge. We rank first in movie-level group 2 and scene-level group 1, second in movie-level group 1 and scene-level group 2 in ACM MM 2023 Grand Challenge.
Utilization of SIKOJA Application (Jambi City Information System) in Supporting the Implementation of Smart Governance in Jambi City

The Jambi City Regional Government provides the SIKOJA Application (Jambi City Information System) to make it easier for the public to obtain integrated information. This study aims to identify the government's perspective on the development of the SIKOJA Application and identify the use of the SIKOJA Application from the community's point of view. This research was conducted using qualitative methods, and secondary data were obtained from the study of planning documents, statistical data, and supporting documents. Primary data were obtained through direct observation and in-depth interviews. The results show that no development constraints were found by the indicators of constraints in the development of integrated information systems. However, it was found that external obstacles in the form of difficulties in integrating several applications, and it was found that the utilization of the SIKOJA Application was still not optimal among the community because there were still few people who knew and used the SIKOJA Application.
A chat about actinic keratosis: Examining capabilities and user experience of ChatGPT as a digital health technology in dermato‐oncology

Abstract Background The potential applications of artificial intelligence (AI) in dermatology are evolving rapidly. Chatbots are an emerging trend in healthcare that rely on large language models (LLMs) to generate answers to prompts from users. However, the factuality and user experience (UX) of such chatbots remain to be evaluated in the context of dermato‐oncology. Objectives To examine the potential of Chat Generative Pretrained Transformer (ChatGPT) as a reliable source of information in the context of actinic keratosis (AK) and to evaluate clinicians' attitudes and UX with regard to the chatbot. Methods A set of 38 clinical questions were compiled and entered as natural language queries in separate, individual conversation threads in ChatGPT (OpenAI, default GPT 3.5). Questions pertain to patient education, diagnosis, and treatment. ChatGPT's responses were presented to a panel of 7 dermatologists for rating of factual accuracy, currency of information, and completeness of the response. Attitudes towards ChatGTP were explored qualitatively and quantitatively using a validated user experience questionnaire (UEQ). Results ChatGPT answered 12 questions (31.6%) with an accurate, current, and complete response. ChatGPT performed best for questions on patient education, including pathogenesis of AK and potential risk factors, but struggled with diagnosis and treatment. Major deficits were seen in grading AK, providing up‐to‐date treatment guidance, and asserting incorrect information with unwarranted confidence. Further, responses were considered verbose with an average word count of 198 (SD 55) and overly alarming of the risk of malignant transformation. Based on UEQ responses, the expert panel considered ChatGPT an attractive and efficient tool, scoring highest for speed of information retrieval, but deemed the chatbot inaccurate and verbose, scoring lowest for clarity . Conclusions While dermatologists rated ChatGPT high in UX, the underlying LLMs that enable such chatbots require further development to guarantee accuracy and concision required in a clinical setting.
Cybersecurity Assessment Framework: A Systematic Review<sup>*</sup>

In today's interconnected world, cybersecurity has become a crucial concern for governments, corporations, and individuals. Establishing robust cybersecurity frameworks that effectively address the unique challenges faced by countries is imperative. This systematic literature review explores the origins, functions, assessment methods, remaining issues, and global adoption of cybersecurity frameworks. The review identified 42 relevant articles that provide valuable insights into these areas. The origins of cybersecurity frameworks can be traced back to the 1970s, evolving in response to the increasing prevalence of cybercrime and advancements in technology. Common functions of cybersecurity frameworks include identification, protection, detection, response, and recovery. Assessment methods focus on evaluating the efficiency and maturity of frameworks. The review also highlights the remaining issues, such as the rapidly evolving threat landscape, lack of standardization, complexity, and scalability. Furthermore, it identifies key cybersecurity frameworks employed globally, including the NIST Framework, ISO/IEC 27001, COBIT, CIS Control, and SANS Critical Security Controls. These frameworks address the technology, governance, cultural, and country-specific needs of organizations. The findings contribute to a comprehensive understanding of cybersecurity frameworks and can inform policymakers, practitioners, and researchers in developing effective cybersecurity strategies.
Think before You Leap: Content-Aware Low-Cost Edge-Assisted Video Semantic Segmentation

Offloading computing to edge servers is a promising solution to support growing video understanding applications at resource-constrained IoT devices. Recent efforts have been made to enhance the scalability of such systems by reducing inference costs on edge servers. However, existing research is not directly applicable to pixel-level vision tasks such as video semantic segmentation (VSS), partly due to the fluctuating VSS accuracy and segment bitrate caused by the dynamic video content. In response, we present Penance, a new edge inference cost reduction framework. By exploiting softmax outputs of VSS models and the prediction mechanism of H.264/AVC codecs, Penance optimizes model selection and compression settings to minimize the inference cost while meeting the required accuracy within the available bandwidth constraints. We implement Penance in a commercial IoT device with only CPUs. Experimental results show that Penance consumes a negligible 6.8% more computation resources than the optimal strategy while satisfying accuracy and bandwidth constraints with a low failure rate.