bluuebunny/crossref_metadata_2025 huggingface dataset:
Development of Finite State Machine Agent in Idle Breeder Game
Authors: Riyadi Agung , Syah Sultan Ilyas Arsalillah Yuswan
Published: August 2023

"Idle Breeder" merupakan sebuah game idle top down yang memberikan pengalaman berternak dengan suasana santai bagi para pemainnya. Setiap karakter hewan didesain dengan perilaku yang menyerupai hewan aslinya. Namun, penerapan game sebelumnya menghadapi berbagai masalah, terutama dalam manajemen kode program yang menyulitkan developer secara scalable dan mengakibatkan beberapa komponen tidak dapat digunakan kembali (reusable). Oleh karena itu, penelitian ini bertujuan untuk mengatasi masalah tersebut dengan mengimplementasikan Finite State Machine (FSM) sebagai solusi. Metode pengembangan yang digunakan adalah Multimedia Development Life Cycle (MDLC), sesuai dengan pengembangan fitur ini. Temuan dari penelitian ini akan memberikan contoh implementasi FSM pada game agent, membantu para pengembang game mobile memahami kapan dan bagaimana mengimplementasikan FSM secara tepat. Hasil pengujian menunjukkan beberapa kelebihan FSM seperti penggunaan script yang modular dan reusable. Pendekatan ini mengurangi redundansi dalam kode dan mempercepat proses implementasi. Selain itu, FSM juga menawarkan skalabilitas tinggi, memungkinkan pengembangan game dalam jangka waktu yang panjang dan mempermudah maintenance. Namun, dikarenakan menggunakan FSM, performa menjadi lebih berat. Oleh karena itu, penggunaan FSM akan lebih efektif ketika ingin memiliki skalabilitas tinggi dan sistem yang kompleks."Idle Breeder" is a top down idle game that provides players with a relaxing farming experience. Each animal character is designed with behaviors that resemble the real animal. However, previous game implementations faced various problems, especially in program code management which made it difficult for developers to be scalable and resulted in some components not being reusable. Therefore, this research aims to overcome these problems by implementing Finite State Machine (FSM) as a solution. The development method used is Multimedia Development Life Cycle (MDLC), in accordance with the development of this feature. The findings of this research will provide examples of FSM implementation in game agents, helping mobile game developers understand when and how to implement FSM appropriately. The test results show several advantages of FSM such as the use of modular and reusable scripts. This approach reduces redundancy in code and speeds up the implementation process. In addition, FSM also offers high scalability, allowing for long-term game development and easier maintenance. However, due to the use of FSM, the performance becomes slower. Therefore, using FSM is more effective when high scalability and complex systems are desired.
Application of K-Means Algorithm on Clustering Recipients of Non-Cash Food Assistance (NCFA)
Authors: Saputra Said Nanda , Haerani Elin , Jasril Jasril , Oktavia Lola , Syafria Fadhilah
Published: August 2023

Persoalan Kemiskinan pada berbagai daerah Indonesia menjadi fokus perhatian. Program BPNT (Bantuan Pangan Non Tunai) bermaksud memangkas biaya pangan dan membagikan gizi yang sepadan terhadap KPM (Keluarga Penerima Manfaat). Penelitian ini menerapkan algoritma K-Means untuk menganalisis pola karakteristik penerima BPNT di Pekanbaru. Data yang digunakan berasal dari penelitian sebelumnya oleh Firza Syahputra dan dari Dinas Sosial Kota Pekanbaru tahun 2020-2021 dengan 732 data dan 41 parameter. Penerapan K-Means dilakukan melalui Google Colab. Melalui data mining dan metode clustering, ditemukan dua klaster dengan 666 data dalam klaster 1 dan 16 data dalam klaster 2. Evaluasi menggunakan Silhouette Score menunjukkan hasil yang baik, dengan nilai 0.9169796594018274. Penelitian ini berpotensi membantu pemerintah dalam mengambil keputusan yang efektif selama penyebaran bantuan pangan non tunai kepada rakyat yang membutuhkan. Dengan demikian, algoritma K-Means Clustering dapat mengidentifikasi pola karakteristik penerima BPNT dan membedakan kelompok yang layak dan tidak layak menerima bantuan.Poverty issues in various parts of Indonesia are the focus of attention. The NCFA (Non-Cash Food Assistance) program's purpose are to lower food consumption and give Beneficiary Families (BF) a healthy diet. The k-means technique use in this study to assess the distinctive patterns of NCFA grantees in Pekanbaru. The data used comes from previous research by Firza Syahputra and from Social Affairs Office Pekanbaru in 2020-2021 with 732 data and 41 parameters. The application of k-means is done through Google Colab. Through data mining and clustering methods, two clusters were found with 666 data in cluster 1 and 16 data in cluster 2. Evaluation using Silhouette Score showed good results, with a value of 0.9169796594018274. This research has the potential to assist the government in making effective decisions in distributing non-cash food help people in need. For the result, the k-means Clustering technique is able to recognize the traits of NCFA recipients and identify groups that are and are not eligible for aid.
Information Center Chatbot in Higher Education Using Dialogflow
Authors: Isnafirlah Dais , Kamayani Mia
Published: August 2023

Fakultas Teknologi Industri dan Informatika UHAMKA, memiliki website resmi yang mengacu pada sistem informasi universitas yang berfungsi untuk pelayanan yang efektif dan menggunakan teknologi terkini. Faktanya, masih memiliki beberapa permasalahan yang terjadi yaitu dosen dan mahasiswa masih sering bertanya via japri ke pihak sekretariat, akademik maupun pimpinan prodi dan fakultas jika memerlukan informasi, serta masih sering melewatkan informasi yang ada di broadcast Grup WhatsApp. Oleh karena itu, tujuan dari penelitian ini untuk merencanakan dan menerapkan teknologi chatbot yang memberikan mahasiswa akses informasi yang cepat dan mudah tentang akademik Fakultas Teknologi Industri dan Informatika UHAMKA secara mudah dan cepat. Chatbot ini diimplementasikan menggunakan tools Dialogflow dengan metode Prototype dan Telegram API. Setelah melakukan uji coba blackbox, Bot Telegram yang dibuat telah terbukti mampu memberikan respons yang memuaskan terhadap pertanyaan yang diajukan, sehingga dapat memberikan bantuan kepada user dalam memperoleh informasi akademik yang dibutuhkan. Selain itu, pengguna tidak perlu menunggu terlalu lama untuk menerima balasan dari chatbot, sehingga membuat chatbot yang digunakan dapat beroperasi dengan cepat dan tepat.The UHAMKA Faculty of Industrial and Informatics Technology has an official website that refers to a university information system that functions for effective service and uses the latest technology. In fact, there are still several problems that occur, namely lecturers and students still often ask via email to the secretariat, academics or study program and faculty leaders if they need information, and they still often miss information that is   WhatsApp Group. Therefore, the purpose of this research is to plan and implement chatbot technology that gives students quick and easy access to information about the academics of the Faculty of Industrial Technology and Informatics UHAMKA easily and quickly. This chatbot is implemented using Dialogflow tools with the Prototype method and the Telegram API. After conducting a blackbox trial, the Telegram Bot that was created has been proven to be able to provide a satisfactory response to the questions asked, so that it can provide assistance to users in obtaining the required academic information. In addition, users don't have to wait too long to receive a reply from the chatbot, so that the chatbot used can operate quickly and precisely.
Decision Support System for Determining Potable Well Water Quality in Pangirkiran Village Using the MOORA Method
Authors: Samsudin Samsudin , Abullah Jumjumi , Syahputra Dedyka , Koto Muhammad Hendrik
Published: August 2023

Air sumur adalah mata air buatan yang digunakan oleh penduduk pedesaan. Air sumur memiliki banyak manfaat bagi masyarakat desa, misalnya untuk memasak, mandi, membersihkan/mencuci pakaian, dan juga untuk air minum. Mata air sumur juga harus memiliki nilai kesehatan yang tidak boleh membuat air tersebut tidak layak untuk diminum. Kebutuhan air bersih setiap tahunnya akan terus berkurang sementara kebutuhan air bersih semakin berkurang, hal ini disebabkan oleh banyaknya pembangunan infrastruktur yang dapat mengganggu potensi alam dan kelestarian air mineral. Analisis data penelitian ini bertujuan untuk menghasilkan rekomendasi penentuan kualitas air sumur minum di Desa Pangirkiran. Kriteria dan bobot yang digunakan adalah C1 = pH air (15%), C2 = jenis warna air (25%), C3 = bau air (30%), C4 = zat yang terkandung dalam air (15%), dan C5 = suhu air (15%), dari 25 alternatif yang digunakan diperoleh hasil rekomendasi penelitian yaitu dengan hasil rangking tertinggi dengan nilai hasil optimasi (34.63539523976). Penelitian ini bertujuan untuk menganalisis kualitas sumur gali dan sumur bor yang digunakan oleh penduduk pedesaan di daerah Pangirkiran dan untuk mengetahui karakteristik air sumur dengan menggunakan metode MOORA (Multi-Objective Optimization in the Basis of Ratio Analysis) di daerah desa Pangirkiran.Well water is an artificial spring used by rural residents. Well water has many uses for village communities, for example for cooking, bathing, cleaning/washing clothes, and also for drinking water. Well spring water must also have a health value that must not make the water unfit for drinking. The need for clean water every year will continue to decrease while the need for clean water is decreasing, this is due to the many infrastructure developments that can disrupt the natural potential and the preservation of mineral water. This research data analysis aims to produce recommendations for determining the quality of drinking well water in Pangirkiran Village. The criteria and weights used are C1 = pH of water (15%), C2 = type of water color (25%), C3 = smell of water (30%), C4 = substance contained in water (15%), and C5 = temperature water (15%), of the 25 alternatives used, the results of the research recommendations were obtained with the highest ranking results with optimization results (34.63539523976). This study aims to analyze the quality of dug and drilled wells used by rural residents in the Pangirkiran area and to determine the characteristics of well water using the MOORA (Multi-Objective Optimization in the Basis of Ratio Analysis) method in the Pangirkiran village area.
Design of an Integrated Medical Record Information System at Puskesmas Kota Banjar
Authors: Shofa Rahmi Nur , Rachman Andi Nur , Ramdani Cecep Muhamad Sidik , Dewi Euis Nur Fitriani
Published: August 2023

Perkembangan pemanfaatan ilmu pengetahuan dan teknologi memberikan dampak yang dapat digunakan diberbagai bidang salah satunya Puskesmas. Salah satu pengembangan yang dilakukan puskesmas yaitu mengembangkan untuk meningkatkan kualitas manajemen pelayanan kesehatan dengan menerapkan sistem informasi puskesmas berbasis teknologi informasi. Pencatatan apabila dilakukan dengan cara ditulis pada buku pasien merupakan cara lama yang digunakan. Maka perlu adanya suatu sistem yang membantu untuk memudahkan pencatatan. Metode pengembangan pada sistem informasi yang akan di ajukan ini yaitu dengan metode Waterfall. Menghasilkan Sistem Informasi Rekam Medis yang terintegrasi untuk mempermudah pihak Puskesmas di daerah Kota Banjar dalam mengolah data jika terdapat pasien yang sebelumnya belum pernah mengunjungi puskesmas tersebut maka akan terdapat Rekam Medis. Sistem Informasi mempermudah dalam mengelola data Rekam Medis sehingga tidak perlu mencari catatan yang begitu banyak dikarenakan data nya sudah tersimpan dalam database. Sistem Informasi Rekam Medis terintegrasi maka data pasien akan terlihat di semua Puskesmas yang ada di Kota Banjar.   Pengujian dalam penelitian dilakukan dengan tahapan metode System Usability Scale (SUS) mendapat nilai akhir yaitu 73,1 dan Hasil pengujian menghasilkan nilai rekomendasi dalam kategori Acceptable.The development of the use of science and technology has an impact that can be used in various fields, one of which is the Puskesmas. One of the developments carried out by the puskesmas is developing to improve the quality of health service management by implementing an information technology-based puskesmas information system. Recording when it is done by writing it in the patient book is the old way used. So, it is necessary to have a system that helps to facilitate recording. The development method for the information system that will be proposed is the Waterfall method. Producing an integrated Medical Record Information System to make it easier for the Puskesmas in the Banjar City area to process data. If there are patients who have never visited the puskesmas before, there will be Medical Records. The Information System makes it easy to manage Medical Record data so there is no need to search for so many records because the data is already stored in the database. The Medical Record Information System is integrated so that patient data will be visible in all Community Health Centers in Banjar City. Testing in the study was carried out using the System Usability Scale (SUS) method, obtaining a final score of 73.1 and the test results yielded a recommendation value in the Acceptable category.
Política Nacional de Resíduos Sólidos (PNRS), Obsolescência Programada e Psicológica: impactos sobre pessoas e o meio ambiente
Authors: Da Silva Carvalho Carlos Antônio , De Miranda Maria Geralda , Santos Avelar Katia Eliane
Published: August 2023

Este artigo discorre sobre os principais aspectos da Política Nacional de Resíduos Sólidos (PNRS) e do Plano Nacional de Resíduos Sólidos (Planares), os efeitos da obsolescência programada e psicológica sobre o cidadão e o meio ambiente, e caminhos para o desenvolvimento sustentável. Seguiu-se a metodologia de Prodanov e Freitas (2013), tratando-se de pesquisa básica, exploratória, qualitativa e bibliográfica. O estudo concluiu que a PNRS e o Planares são importantes legislações para o controle da poluição e preservação do meio ambiente, força as empresas reverem seus métodos de produção e cria a obrigação de adoção de medidas de proteção e recuperação do ecossistema. A obsolescência é inerente aos produtos, mas a preocupação recai sobre a redução sistemática do tempo de vida útil, trazendo prejuízos ao consumidor, o consumismo, o aumento da extração de recursos naturais para produção e o aumento de resíduos e rejeitos, comprometendo a sustentabilidade do planeta.
Safe Multi-agent Learning via Trapping Regions
Authors: Czechowski Aleksander , Oliehoek Frans A.
Published: August 2023

One of the main challenges of multi-agent learning lies in establishing convergence of the algorithms, as, in general, a collection of individual, self-serving agents is not guaranteed to converge with their joint policy, when learning concurrently. This is in stark contrast to most single-agent environments, and sets a prohibitive barrier for deployment in practical applications, as it induces uncertainty in long term behavior of the system. In this work, we apply the concept of trapping regions, known from qualitative theory of dynamical systems, to create safety sets in the joint strategy space for decentralized learning. We propose a binary partitioning algorithm for verification that candidate sets form trapping regions in systems with known learning dynamics, and a heuristic sampling algorithm for scenarios where learning dynamics are not known. We demonstrate the applications to a regularized version of Dirac Generative Adversarial Network, a four-intersection traffic control scenario run in a state of the art open-source microscopic traffic simulator SUMO, and a mathematical model of economic competition.
Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment
Authors: Jin Peng , Li Hao , Cheng Zesen , Huang Jinfa , Wang Zhennan , Yuan Li , Liu Chang , Chen Jie
Published: August 2023

Text-video retrieval is a challenging cross-modal task, which aims to align visual entities with natural language descriptions. Current methods either fail to leverage the local details or are computationally expensive. What's worse, they fail to leverage the heterogeneous concepts in data. In this paper, we propose the Disentangled Conceptualization and Set-to-set Alignment (DiCoSA) to simulate the conceptualizing and reasoning process of human beings. For disentangled conceptualization, we divide the coarse feature into multiple latent factors related to semantic concepts. For set-to-set alignment, where a set of visual concepts correspond to a set of textual concepts, we propose an adaptive pooling method to aggregate semantic concepts to address the partial matching. In particular, since we encode concepts independently in only a few dimensions, DiCoSA is superior at efficiency and granularity, ensuring fine-grained interactions using a similar computational complexity as coarse-grained alignment. Extensive experiments on five datasets, including MSR-VTT, LSMDC, MSVD, ActivityNet, and DiDeMo, demonstrate that our method outperforms the existing state-of-the-art methods.
Physics-Guided Human Motion Capture with Pose Probability Modeling
Authors: Ju Jingyi , Huang Buzhen , Zhu Chen , Li Zhihao , Wang Yangang
Published: August 2023

Incorporating physics in human motion capture to avoid artifacts like floating, foot sliding, and ground penetration is a promising direction. Existing solutions always adopt kinematic results as reference motions, and the physics is treated as a post-processing module. However, due to the depth ambiguity, monocular motion capture inevitably suffers from noises, and the noisy reference often leads to failure for physics-based tracking. To address the obstacles, our key-idea is to employ physics as denoising guidance in the reverse diffusion process to reconstruct physically plausible human motion from a modeled pose probability distribution. Specifically, we first train a latent gaussian model that encodes the uncertainty of 2D-to-3D lifting to facilitate reverse diffusion. Then, a physics module is constructed to track the motion sampled from the distribution. The discrepancies between the tracked motion and image observation are used to provide explicit guidance for the reverse diffusion model to refine the motion. With several iterations, the physics-based tracking and kinematic denoising promote each other to generate a physically plausible human motion. Experimental results show that our method outperforms previous physics-based methods in both joint accuracy and success rate. More information can be found at https://github.com/Me-Ditto/Physics-Guided-Mocap.
Clustered-patch Element Connection for Few-shot Learning
Authors: Lai Jinxiang , Yang Siqian , Zhou Junhong , Wu Wenlong , Chen Xiaochen , Liu Jun , Gao Bin-Bin , Wang Chengjie
Published: August 2023

Weak feature representation problem has influenced the performance of few-shot classification task for a long time. To alleviate this problem, recent researchers build connections between support and query instances through embedding patch features to generate discriminative representations. However, we observe that there exists semantic mismatches (foreground/ background) among these local patches, because the location and size of the target object are not fixed. What is worse, these mismatches result in unreliable similarity confidences, and complex dense connection exacerbates the problem. According to this, we propose a novel Clustered-patch Element Connection (CEC) layer to correct the mismatch problem. The CEC layer leverages Patch Cluster and Element Connection operations to collect and establish reliable connections with high similarity patch features, respectively. Moreover, we propose a CECNet, including CEC layer based attention module and distance metric. The former is utilized to generate a more discriminative representation benefiting from the global clustered-patch features, and the latter is introduced to reliably measure the similarity between pair-features. Extensive experiments demonstrate that our CECNet outperforms the state-of-the-art methods on classification benchmark. Furthermore, our CEC approach can be extended into few-shot segmentation and detection tasks, which achieves competitive performances.
Deep Unfolding Convolutional Dictionary Model for Multi-Contrast MRI Super-resolution and Reconstruction
Authors: Lei Pengcheng , Fang Faming , Zhang Guixu , Xu Ming
Published: August 2023

Magnetic resonance imaging (MRI) tasks often involve multiple contrasts. Recently, numerous deep learning-based multi-contrast MRI super-resolution (SR) and reconstruction methods have been proposed to explore the complementary information from the multi-contrast images. However, these methods either construct parameter-sharing networks or manually design fusion rules, failing to accurately model the correlations between multi-contrast images and lacking certain interpretations. In this paper, we propose a multi-contrast convolutional dictionary (MC-CDic) model under the guidance of the optimization algorithm with a well-designed data fidelity term. Specifically, we bulid an observation model for the multi-contrast MR images to explicitly model the multi-contrast images as common features and unique features. In this way, only the useful information in the reference image can be transferred to the target image, while the inconsistent information will be ignored. We employ the proximal gradient algorithm to optimize the model and unroll the iterative steps into a deep CDic model. Especially, the proximal operators are replaced by learnable ResNet. In addition, multi-scale dictionaries are introduced to further improve the model performance. We test our MC-CDic model on multi-contrast MRI SR and reconstruction tasks. Experimental results demonstrate the superior performance of the proposed MC-CDic model against existing SOTA methods. Code is available at https://github.com/lpcccc-cv/MC-CDic.
TG-VQA: Ternary Game of Video Question Answering
Authors: Li Hao , Jin Peng , Cheng Zesen , Zhang Songyang , Chen Kai , Wang Zhennan , Liu Chang , Chen Jie
Published: August 2023

Video question answering aims at answering a question about the video content by reasoning the alignment semantics within them. However, since relying heavily on human instructions, i.e., annotations or priors, current contrastive learning-based VideoQA methods remains challenging to perform fine-grained visual-linguistic alignments. In this work, we innovatively resort to game theory, which can simulate complicated relationships among multiple players with specific interaction strategies, e.g., video, question, and answer as ternary players, to achieve fine-grained alignment for VideoQA task. Specifically, we carefully design a VideoQA-specific interaction strategy to tailor the characteristics of VideoQA, which can mathematically generate the fine-grained visual-linguistic alignment label without label-intensive efforts. Our TG-VQA outperforms existing state-of-the-art by a large margin (more than 5%) on long-term and short-term VideoQA datasets, verifying its effectiveness and generalization ability. Thanks to the guidance of game-theoretic interaction, our model impressively convergences well on limited data (10^4 videos), surpassing most of those pre-trained on large-scale data (10^7 videos).
Controlling Neural Style Transfer with Deep Reinforcement Learning
Authors: Feng Chengming , Hu Jing , Wang Xin , Hu Shu , Zhu Bin , Wu Xi , Zhu Hongtu , Lyu Siwei
Published: August 2023

Controlling the degree of stylization in the Neural Style Transfer (NST) is a little tricky since it usually needs hand-engineering on hyper-parameters. In this paper, we propose the first deep Reinforcement Learning (RL) based architecture that splits one-step style transfer into a step-wise process for the NST task. Our RL-based method tends to preserve more details and structures of the content image in early steps, and synthesize more style patterns in later steps. It is a user-easily-controlled style-transfer method. Additionally, as our RL-based model performs the stylization progressively, it is lightweight and has lower computational complexity than existing one-step Deep Learning (DL) based models. Experimental results demonstrate the effectiveness and robustness of our method.
Locate, Refine and Restore: A Progressive Enhancement Network for Camouflaged Object Detection
Authors: Li Xiaofei , Yang Jiaxin , Li Shuohao , Lei Jun , Zhang Jun , Chen Dong
Published: August 2023

Camouflaged Object Detection (COD) aims to segment objects that blend in with their surroundings. Most existing methods mainly tackle this issue by a single-stage framework, which tends to degrade performance in the face of small objects, low-contrast objects and objects with diverse appearances. In this paper, we propose a novel Progressive Enhancement Network (PENet) for COD by imitating the human visual detection system, which follows a three-stage detection process: locate objects, refine textures and restore boundary. Specifically, our PENet contains three key modules, i.e., the object location module (OLM), the group attention module (GAM) and the context feature restoration module (CFRM). The OLM is designed to position the object globally, the GAM is developed to refine both high-level semantic and low-level texture feature representation, and the CFRM is leveraged to effectively aggregate multi-level features for progressively restoring the clear boundary. Extensive results demonstrate that our PENet significantly outperforms 32 state-of-the-art methods on four widely used benchmark datasets
SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation
Authors: Li Xuewei , Wu Tao , Qi Zhongang , Wang Gaoang , Shan Ying , Li Xi
Published: August 2023

As an important and challenging problem in computer vision, PAnoramic Semantic Segmentation (PASS) gives complete scene perception based on an ultra-wide angle of view. Usually, prevalent PASS methods with 2D panoramic image input focus on solving image distortions but lack consideration of the 3D properties of original 360 degree data. Therefore, their performance will drop a lot when inputting panoramic images with the 3D disturbance. To be more robust to 3D disturbance, we propose our Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation (SGAT4PASS), considering 3D spherical geometry knowledge. Specifically, a spherical geometry-aware framework is proposed for PASS. It includes three modules, i.e., spherical geometry-aware image projection, spherical deformable patch embedding, and a panorama-aware loss, which takes input images with 3D disturbance into account, adds a spherical geometry-aware constraint on the existing deformable patch embedding, and indicates the pixel density of original 360 degree data, respectively. Experimental results on Stanford2D3D Panoramic datasets show that SGAT4PASS significantly improves performance and robustness, with approximately a 2% increase in mIoU, and when small 3D disturbances occur in the data, the stability of our performance is improved by an order of magnitude. Our code and supplementary material are available at https://github.com/TencentARC/SGAT4PASS.
Complete Instances Mining for Weakly Supervised Instance Segmentation
Authors: Li Zecheng , Zeng Zening , Liang Yuqi , Yu Jin-Gang
Published: August 2023

Weakly supervised instance segmentation (WSIS) using only image-level labels is a challenging task due to the difficulty of aligning coarse annotations with the finer task. However, with the advancement of deep neural networks (DNNs), WSIS has garnered significant attention. Following a proposal-based paradigm, we encounter a redundant segmentation problem resulting from a single instance being represented by multiple proposals. For example, we feed a picture of a dog and proposals into the network and expect to output only one proposal containing a dog, but the network outputs multiple proposals. To address this problem, we propose a novel approach for WSIS that focuses on the online refinement of complete instances through the use of MaskIoU heads to predict the integrity scores of proposals and a Complete Instances Mining (CIM) strategy to explicitly model the redundant segmentation problem and generate refined pseudo labels. Our approach allows the network to become aware of multiple instances and complete instances, and we further improve its robustness through the incorporation of an Anti-noise strategy. Empirical evaluations on the PASCAL VOC 2012 and MS COCO datasets demonstrate that our method achieves state-of-the-art performance with a notable margin. Our implementation will be made available at https://github.com/ZechengLi19/CIM.
Analyzing and Combating Attribute Bias for Face Restoration
Authors: Li Zelin , Zeng Dan , Yan Xiao , Shen Qiaomu , Tang Bo
Published: August 2023

Face restoration (FR) recovers high resolution (HR) faces from low resolution (LR) faces and is challenging due to its ill-posed nature. With years of development, existing methods can produce quality HR faces with realistic details. However, we observe that key facial attributes (e.g., age and gender) of the restored faces could be dramatically different from the LR faces and call this phenomenon attribute bias, which is fatal when using FR for applications such as surveillance and security. Thus, we argue that FR should consider not only image quality as in existing works but also attribute bias. To this end, we thoroughly analyze attribute bias with extensive experiments and find that two major causes are the lack of attribute information in LR faces and bias in the training data. Moreover, we propose the DebiasFR framework to produce HR faces with high image quality and accurate facial attributes. The key design is to explicitly model the facial attributes, which also allows to adjust facial attributes for the output HR faces. Experiment results show that DebiasFR has comparable image quality but significantly smaller attribute bias when compared with state-of-the-art FR methods.
U-Match: Two-view Correspondence Learning with Hierarchy-aware Local Context Aggregation
Authors: Li Zizhuo , Zhang Shihua , Ma Jiayi
Published: August 2023

Local context capturing has become the core factor for achieving leading performance in two-view correspondence learning. Recent advances have devised various local context extractors whereas typically adopting explicit neighborhood relation modeling that is restricted and inflexible. To address this issue, we introduce U-Match, an attentional graph neural network that has the flexibility to enable implicit local context awareness at multiple levels. Specifically, a hierarchy-aware graph representation (HAGR) module is designed and fleshed out by local context pooling and unpooling operations. The former encodes local context by adaptively sampling a set of nodes to form a coarse-grained graph, while the latter decodes local context by recovering the coarsened graph back to its original size. Moreover, an orthogonal fusion module is proposed for the collaborative use of HAGR module, which integrates complementary local and global information into compact feature representations without redundancy. Extensive experiments on different visual tasks prove that our method significantly surpasses the state-of-the-arts. In particular, U-Match attains an AUC at 5 degree threshold of 60.53% on the challenging YFCC100M dataset without RANSAC, outperforming the strongest prior model by 8.61 absolute percentage points. Our code is publicly available at https://github.com/ZizhuoLi/U-Match.
Adaptive Sparse ViT: Towards Learnable Adaptive Token Pruning by Fully Exploiting Self-Attention
Authors: Liu Xiangcheng , Wu Tianyi , Guo Guodong
Published: August 2023

Vision transformer has emerged as a new paradigm in computer vision, showing excellent performance while accompanied by expensive computational cost. Image token pruning is one of the main approaches for ViT compression, due to the facts that the complexity is quadratic with respect to the token number, and many tokens containing only background regions do not truly contribute to the final prediction. Existing works either rely on additional modules to score the importance of individual tokens, or implement a fixed ratio pruning strategy for different input instances. In this work, we propose an adaptive sparse token pruning framework with a minimal cost. Specifically, we firstly propose an inexpensive attention head importance weighted class attention scoring mechanism. Then, learnable parameters are inserted as thresholds to distinguish informative tokens from unimportant ones. By comparing token attention scores and thresholds, we can discard useless tokens hierarchically and thus accelerate inference. The learnable thresholds are optimized in budget-aware training to balance accuracy and complexity, performing the corresponding pruning configurations for different input instances. Extensive experiments demonstrate the effectiveness of our approach. Our method improves the throughput of DeiT-S by 50% and brings only 0.2% drop in top-1 accuracy, which achieves a better trade-off between accuracy and latency than the previous methods.
Learning in Multi-Memory Games Triggers Complex Dynamics Diverging from Nash Equilibrium
Authors: Fujimoto Yuma , Ariu Kaito , Abe Kenshi
Published: August 2023

Repeated games consider a situation where multiple agents are motivated by their independent rewards throughout learning. In general, the dynamics of their learning become complex. Especially when their rewards compete with each other like zero-sum games, the dynamics often do not converge to their optimum, i.e., the Nash equilibrium. To tackle such complexity, many studies have understood various learning algorithms as dynamical systems and discovered qualitative insights among the algorithms. However, such studies have yet to handle multi-memory games (where agents can memorize actions they played in the past and choose their actions based on their memories), even though memorization plays a pivotal role in artificial intelligence and interpersonal relationship. This study extends two major learning algorithms in games, i.e., replicator dynamics and gradient ascent, into multi-memory games. Then, we prove their dynamics are identical. Furthermore, theoretically and experimentally, we clarify that the learning dynamics diverge from the Nash equilibrium in multi-memory zero-sum games and reach heteroclinic cycles (sojourn longer around the boundary of the strategy space), providing a fundamental advance in learning in games.
SLViT: Scale-Wise Language-Guided Vision Transformer for Referring Image Segmentation
Authors: Ouyang Shuyi , Wang Hongyi , Xie Shiao , Niu Ziwei , Tong Ruofeng , Chen Yen-Wei , Lin Lanfen
Published: August 2023

Referring image segmentation aims to segment an object out of an image via a specific language expression. The main concept is establishing global visual-linguistic relationships to locate the object and identify boundaries using details of the image. Recently, various Transformer-based techniques have been proposed to efficiently leverage long-range cross-modal dependencies, enhancing performance for referring segmentation. However, existing methods consider visual feature extraction and cross-modal fusion separately, resulting in insufficient visual-linguistic alignment in semantic space. In addition, they employ sequential structures and hence lack multi-scale information interaction. To address these limitations, we propose a Scale-Wise Language-Guided Vision Transformer (SLViT) with two appealing designs: (1) Language-Guided Multi-Scale Fusion Attention, a novel attention mechanism module for extracting rich local visual information and modeling global visual-linguistic relationships in an integrated manner. (2) An Uncertain Region Cross-Scale Enhancement module that can identify regions of high uncertainty using linguistic features and refine them via aggregated multi-scale features. We have evaluated our method on three benchmark datasets. The experimental results demonstrate that SLViT surpasses state-of-the-art methods with lower computational cost. The code is publicly available at: https://github.com/NaturalKnight/SLViT.
Detecting Adversarial Faces Using Only Real Face Self-Perturbations
Authors: Wang Qian , Xian Yongqin , Ling Hefei , Zhang Jinyuan , Lin Xiaorui , Li Ping , Chen Jiazhong , Yu Ning
Published: August 2023

Adversarial attacks aim to disturb the functionality of a target system by adding specific noise to the input samples, bringing potential threats to security and robustness when applied to facial recognition systems. Although existing defense techniques achieve high accuracy in detecting some specific adversarial faces (adv-faces), new attack methods especially GAN-based attacks with completely different noise patterns circumvent them and reach a higher attack success rate. Even worse, existing techniques require attack data before implementing the defense, making it impractical to defend newly emerging attacks that are unseen to defenders. In this paper, we investigate the intrinsic generality of adv-faces and propose to generate pseudo adv-faces by perturbing real faces with three heuristically designed noise patterns. We are the first to train an adv-face detector using only real faces and their self-perturbations, agnostic to victim facial recognition systems, and agnostic to unseen attacks. By regarding adv-faces as out-of-distribution data, we then naturally introduce a novel cascaded system for adv-face detection, which consists of training data self-perturbations, decision boundary regularization, and a max-pooling-based binary classifier focusing on abnormal local color aberrations. Experiments conducted on LFW and CelebA-HQ datasets with eight gradient-based and two GAN-based attacks validate that our method generalizes to a variety of unseen adversarial attacks.
From Generation to Suppression: Towards Effective Irregular Glow Removal for Nighttime Visibility Enhancement
Authors: Wu Wanyu , Wang Wei , Wang Zheng , Jiang Kui , Xu Xin
Published: August 2023

Most existing Low-Light Image Enhancement (LLIE) methods are primarily designed to improve brightness in dark regions, which suffer from severe degradation in nighttime images. However, these methods have limited exploration in another major visibility damage, the glow effects in real night scenes. Glow effects are inevitable in the presence of artificial light sources and cause further diffused blurring when directly enhanced. To settle this issue, we innovatively consider the glow suppression task as learning physical glow generation via multiple scattering estimation according to the Atmospheric Point Spread Function (APSF). In response to the challenges posed by uneven glow intensity and varying source shapes, an APSF-based Nighttime Imaging Model with Near-field Light Sources (NIM-NLS) is specifically derived to design a scalable Light-aware Blind Deconvolution Network (LBDN). The glow-suppressed result is then brightened via a Retinex-based Enhancement Module (REM). Remarkably, the proposed glow suppression method is based on zero-shot learning and does not rely on any paired or unpaired training data. Empirical evaluations demonstrate the effectiveness of the proposed method in both glow suppression and low-light enhancement tasks.
Video Diffusion Models with Local-Global Context Guidance
Authors: Yang Siyuan , Zhang Lu , Liu Yu , Jiang Zhizhuo , He You
Published: August 2023

Diffusion models have emerged as a powerful paradigm in video synthesis tasks including prediction, generation, and interpolation. Due to the limitation of the computational budget, existing methods usually implement conditional diffusion models with an autoregressive inference pipeline, in which the future fragment is predicted based on the distribution of adjacent past frames. However, only the conditions from a few previous frames can't capture the global temporal coherence, leading to inconsistent or even outrageous results in long-term video prediction. In this paper, we propose a Local-Global Context guided Video Diffusion model (LGC-VD) to capture multi-perception conditions for producing high-quality videos in both conditional/unconditional settings. In LGC-VD, the UNet is implemented with stacked residual blocks with self-attention units, avoiding the undesirable computational cost in 3D Conv. We construct a local-global context guidance strategy to capture the multi-perceptual embedding of the past fragment to boost the consistency of future prediction. Furthermore, we propose a two-stage training strategy to alleviate the effect of noisy frames for more stable predictions. Our experiments demonstrate that the proposed method achieves favorable performance on video prediction, interpolation, and unconditional video generation. We release code at https://github.com/exisas/LGC-VD.
Actor-Multi-Scale Context Bidirectional Higher Order Interactive Relation Network for Spatial-Temporal Action Localization
Authors: Yu Jun , Zheng Yingshuai , Ruan Shulan , Liu Qi , Cheng Zhiyuan , Wu Jinze
Published: August 2023

The key to video action detection lies in the understanding of interaction between persons and background objects in a video. Current methods usually employ object detectors to extract objects directly or use grid features to represent objects in the environment, which underestimate the great potential of multi-scale context information (e.g., objects and scenes of different sizes). How to exactly represent the multi-scale context and make full utilization of it still remains an unresolved challenge for spatial-temporal action localization. In this paper, we propose a novel Actor-Multi-Scale Context Bidirectional Higher Order Interactive Relation Network (AMCRNet) that extracts multi-scale context through multiple pooling layers with different sizes. Specifically, we develop an Interactive Relation Extraction module to model the higher-order relation between the target person and the context (e.g., other persons and objects). Along this line, we further propose a History Feature Bank and Interaction method to achieve better performance by modeling such relation across continuing video clips. Extensive experimental results on AVA2.2 and UCF101-24 demonstrate the superiority and rationality of our proposed AMCRNet.
Black-box Prompt Tuning for Vision-Language Model as a Service
Authors: Yu Lang , Chen Qin , Lin Jiaju , He Liang
Published: August 2023

In the scenario of Model-as-a-Service (MaaS), pre-trained models are usually released as inference APIs. Users are allowed to query those models with manually crafted prompts. Without accessing the network structure and gradient information, it's tricky to perform continuous prompt tuning on MaaS, especially for vision-language models (VLMs) considering cross-modal interaction. In this paper, we propose a black-box prompt tuning framework for VLMs to learn task-relevant prompts without back-propagation. In particular, the vision and language prompts are jointly optimized in the intrinsic parameter subspace with various evolution strategies. Different prompt variants are also explored to enhance the cross-model interaction. Experimental results show that our proposed black-box prompt tuning framework outperforms both hand-crafted prompt engineering and gradient-based prompt learning methods, which serves as evidence of its capability to train task-relevant prompts in a derivative-free manner.
Spatially Covariant Lesion Segmentation
Authors: Zhang Hang , Wang Rongguang , Zhang Jinwei , Liu Dongdong , Li Chao , Li Jiahao
Published: August 2023

Compared to natural images, medical images usually show stronger visual patterns and therefore this adds flexibility and elasticity to resource-limited clinical applications by injecting proper priors into neural networks.

In this paper, we propose spatially covariant pixel-aligned classifier (SCP) to improve the computational efficiency and meantime maintain or increase accuracy for lesion segmentation.

SCP relaxes the spatial invariance constraint imposed by convolutional operations and optimizes an underlying implicit function that maps image coordinates to network weights, the parameters of which are obtained along with the backbone network training and later used for generating network weights to capture spatially covariant contextual information.

We demonstrate the effectiveness and efficiency of the proposed SCP using two lesion segmentation tasks from different imaging modalities: white matter hyperintensity segmentation in magnetic resonance imaging and liver tumor segmentation in contrast-enhanced abdominal computerized tomography.

The network using SCP has achieved 23.8, 64.9 and 74.7 reduction in GPU memory usage, FLOPs, and network size with similar or better accuracy for lesion segmentation.
Learning Object Consistency and Interaction in Image Generation from Scene Graphs
Authors: Zhang Yangkang , Meng Chenye , Li Zejian , Chen Pei , Yang Guang , Yang Changyuan , Sun Lingyun
Published: August 2023

This paper is concerned with synthesizing images conditioned on a scene graph (SG), a set of object nodes and their edges of interactive relations. We divide existing works into image-oriented and code-oriented methods. In our analysis, the image-oriented methods do not consider object interaction in spatial hidden feature. On the other hand, in empirical study, the code-oriented methods lose object consistency as their generated images miss certain objects in the input scene graph. To alleviate these two issues, we propose Learning Object Consistency and Interaction (LOCI). To preserve object consistency, we design a consistency module with a weighted augmentation strategy for objects easy to be ignored and a matching loss between scene graphs and image codes. To learn object interaction, we design an interaction module consisting of three kinds of message propagation between the input scene graph and the learned image code. Experiments on COCO-stuff and Visual Genome datasets show our proposed method alleviates the ignorance of objects and outperforms the state-of-the-art on visual fidelity of generated images and objects.
MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment
Authors: Zhang Zicheng , Sun Wei , Min Xiongkuo , Wang Qiyuan , He Jun , Zhou Quan , Zhai Guangtao
Published: August 2023

The visual quality of point clouds has been greatly emphasized since the ever-increasing 3D vision applications are expected to provide cost-effective and high-quality experiences for users. Looking back on the development of point cloud quality assessment (PCQA), the visual quality is usually evaluated by utilizing single-modal information, i.e., either extracted from the 2D projections or 3D point cloud. The 2D projections contain rich texture and semantic information but are highly dependent on viewpoints, while the 3D point clouds are more sensitive to geometry distortions and invariant to viewpoints. Therefore, to leverage the advantages of both point cloud and projected image modalities, we propose a novel no-reference Multi-Modal Point Cloud Quality Assessment (MM-PCQA) metric. In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling. Then we render the point clouds into 2D image projections for texture feature extraction. To achieve the goals, the sub-models and projected images are encoded with point-based and image-based neural networks. Finally, symmetric cross-modal attention is employed to fuse multi-modal quality-aware information. Experimental results show that our approach outperforms all compared state-of-the-art methods and is far ahead of previous no-reference PCQA methods, which highlights the effectiveness of the proposed method. The code is available at https://github.com/zzc-1998/MM-PCQA.