bluuebunny/crossref_metadata_2025 huggingface dataset:
Building a Gender-Responsive Higher Education Institution through Collaborative Governance
Authors: Fithriyyah Mustiqowati Ummul , Sari Ilmi Puspita , Monalisa Monalisa
Published: July 2024

This research aims to determine the extent of the implementation of Gender Responsive Higher Education (PTRG) at the State Islamic University Sultan Syarif Kasim Riau (UIN Suska Riau) using the collaborative governance model by Ansel and Gash. Collaborative governance is a part that supports the implementation of gender-responsive indicators in higher education. It examines the commitment and trust built by stakeholders in addressing gender disparities. The methodology used in this research is descriptive qualitative, with data obtained from observations in various faculties of UIN Suska Riau and interviews with 23 informants consisting of 19 academic members, 1 private sector representative, 2 Non-Government Organization (NGO) members, and 1 community member. The findings show that the implementation of Gender Responsive Higher Education at UIN Suska Riau is carried out by various actors in line with collaborative governance actors, including the government, private sector, and community. In this research, the campus community acts as the government actor on behalf of the university institution, PT. Orbit Ventura Indonesia as the private sector actor, Rumah KitaB and AMAN as NGOs, and the community has been optimal. This is evident from most of the gender-responsive higher education indicators is well achieved. Stakeholder collaboration leads to actions that strengthen planning in the implementation of gender responsiveness.Abstrak: Penelitian ini bertujuan untuk mengetahui sejauh mana pelaksanaan Perguruan Tinggi Responsive Gender (PTRG) di Universitas Islam Negeri Sultan Syarif Kasim Riau (UIN Suska Riau) dengan menggunakan teori tata kelola kolaboratif model Ansel dan Gash. Tata kelola kolaboratif menjadi bagian yang mendukung pelaksanaan indikator responsif gender Perguruan Tinggi. Melihat komitmen dan kepercayaan yang dibangun oleh para pemangku kepentingan dalam menyelesaikan ketimpangan gender. Metodologi yang digunakan dalam penelitian ini adalah kualitatif deskriptif dengan perolehan data dari observasi ke berbagai fakultas UIN Suska Riau dan wawancara dengan 23 informan yang terdiri dari sivitas akademika sebanyak 19 orang, sektor swasta 1 orang, anggota Lembaga Swadaya Masyarakat (LSM) 2 orang dan masyarakat 1 orang. Temuan penelitian menunjukkan bahwasanya pelaksanaan Perguruan Tinggi Responsive Gender di UIN Suska Riau dilakukan oleh berbagai aktor sesuai dengan aktor tata kelola kolaboratif yang terdiri dari pemerintah, sektor swasta dan masyarakat maka aktor dalam penelitian ini warga kampus sebagai aktor pemerintah atas nama instansi perguruan tinggi, PT. Orbit Ventura Indonesia sebagai aktor swasta, Rumah KitaB dan AMAN sebagai LSM serta masyarakat dalam pelaksanaannya sudah optimal. Hal ini terlihat dari sebagaian besar indikator Perguruan Tinggi Responsive Gender yang tercapai dengan baik. Kolaborasi pemangku kepentingan menimbulkan tindakan yang mengarah pada penguatan perencanaan dalam pelaksanaan responsif gender.
Economic Assessment of Damage to Specially Protected Natural Areas from Negative Water Impacts: Methodological and Applied Aspects
Authors: Ayusheeva Svetlana N. ,   Baikal Institute of Nature Management, Siberian Branch of the Russian Academy of Sciences, Andreeva Alyona M. , Maksanova Liudmila B.-Zh. , Sangadieva Iraida G. ,   Baikal Institute of Nature Management, Siberian Branch of the Russian Academy of Sciences,   Baikal Institute of Nature Management, Siberian Branch of the Russian Academy of Sciences,   Buryat State Agriculture Academy by V.R. Filippov
Published: July 2024

The article continues a series of publications on the economic assessment of the consequences of negative water impacts caused by the rising water levels of Lake Baikal. The study focuses on certain methodological aspects of assessing damage to the infrastructure of coastal specially protected natural areas (SPNAs). Thus, within the Republic of Buryatia, SPNAs of federal significance (Transbaical National Park, Baikal and Barguzinsky Reserves, Frolikha and Kabansky Wildlife Refuges) and regional significance (Upper Angara, Ad-jacent to Lake Baikal, and Enkheluk Wildlife Refuges, and the recreational area “Baikal Coast”) are affected by negative impacts due to changes in Lake Baikal’s water levels. The methodological framework for assessing damage relies on the fundamental principles of the “Methodology for Assessing Probabilistic Damage” (VIEMS methodology). Considering the limited application of the methodology concerning SPNAs, the article justifies methodological provisions and proposes an algorithm for assessing damage to SPNA infrastructure, taking into account their functional purpose.
Conceptual Foundations of Studying Counter-Discourse in Social Media
Authors: Krivosheev Aleksandr A. ,   St. Petersburg State University
Published: July 2024

This article examines the main concepts and theories used in the study of political discourse in social media. It defines the key features of the internet and social networks and their impact on changing communication for-mats. The article addresses the issue of democratization and simultaneous complexity in information pro-cessing for a wide range of users. During the review, the theories of the “global village”, discursive space, in-formation ecology, virtual spaces and media literacy are highlighted. Various concepts assess differently the role and influence of social media on discourse reproduction: from negative evaluations – users become mere consumers of content – to positive ones – users are capable of analyzing and fairly freely constructing political discourses. It is emphasized that social networks offer significant opportunities for countering dominant dis-courses and forming counter-discourses. Conclusion dwells upon the fact that new communication tools pro-vide new avenues for user interaction in creating and discussing political texts.
From the desk of the Editor-in-Chief: Introduction to the Black Theatre Review, Nurturing
Authors: Green Omiyẹmi (Artisia)
Published: July 2024

In this issue, Vol. 3.1., we invite readers to explore the collective body of Black and African Diasporic theatre history, dramatic literature, performance and production, multimedia media, artistic leadership, pedagogy, and praxis for elements of nurturing that center Black female and femme experiences. For this issue, nurturing is defined as the acts, rituals, and strategies of care that enhance one’s inner power and enable them to flow toward joy and vitality. Black women scholars, artists, and administrators often teach, conduct research and creative scholarship, govern, build, and nurture within institutional systems in the face of lack and their peril. Collectively, the entries featured in this issue attempt to move Black women—protagonists, playwrights, contemporary icons—from the margins to the center. Black Feminist and Afro-indigenous acts, rituals, and strategies of care are explicit and remind readers that love can be found everywhere. In all circumstances, a real revolution occurs within.
Modeling Artificial Intelligence in Real-Time Collaborative Piano Playing Systems
Authors: Liu Huiming
Published: July 2024

Due to the challenges associated with mastering fundamental piano playing techniques, the inefficiency of self-guided learning, and the prohibitive cost of one-on-one instruction, many novices abandon their musical pursuits prematurely. Our research addresses these issues by enhancing music feature extraction methods through artificial intelligence modeling and developing a piano-playing ability evaluation system. This system leverages an attention mechanism and an LSTM neural network model to assess a player’s abilities based on rhythm, thematic prominence, and musical expression within various levels of piano scores. By analyzing sample tracks from the Thompson Simple Piano Tutorial, our system demonstrates robust performance, achieving an overall F-Measure above 0.9 with an average value of 0.9641. These results indicate that the evaluation system offers precise assessments and can significantly aid piano instruction, providing learners with reliable feedback on their progress.
The Dual Nature of the Inﬂuence of Artiﬁcial Intelligence in the Implementation of Human Rights: Human-Minded and International Legal Dimensions
Authors: Abashidze A. Kh. , Popovic M.
Published: July 2024

INTRODUCTION. The subject of this study is achievements (results) of technology, or more precisely, products of artiﬁcial intelligence (intelligent machine; intelligent computer program, etc.) and their impact on the implementation of human rights through the prism of the human mind and in the context of the international legal, including the human rights dimension.

MATERIALS AND METHODS. The scientiﬁc research is based on the work of both Russian and foreign specialists in the ﬁeld of law, international law, international human rights law, as well as specialists in the regulation and use of artiﬁcial intelligence products. Documents and materials of international organizations, primarily the United Nations (UN), as well as national legal acts of the Russian Federation have been analyzed. In the preparation of the study, general scientiﬁc, comparative legal and speciﬁcally legal methods were used.

RESEARCH RESULTS. Within the framework of the conducted research, the authors delved into the concept of the term “artiﬁcial intelligence” itself, and come to the conclusion that despite its widespread use by specialists in various ﬁelds, there is no single concept of this term at the moment. The authors analyzed the national legal framework of the Russian Federation directly or indirectly regulating the use of AI products, as well as international legal achievements in the regulation of this area (primarily at the universal (UN) and integration (European Union (EU)) levels), with a special emphasis on the implementation and observance of human rights and freedoms.

DISCUSSION AND CONCLUSIONS. The authors came to the conclusion that the nature of the impact of artiﬁcial intelligence on the realization of human rights is dual. Already at the initial stage of deﬁning the legal regime for the development and use of AI products weak mechanisms of control and responsibility in this sphere are laid down. The question arises as to which legal regime is preferable for states whose corporations are leading in the development of AI products. Despite existing international legal regulatory mechanisms, this protection is still insuﬃcient.
Statehood and its International Legal Signiﬁcance
Authors: Romaikin I. A.
Published: July 2024

INTRODUCTION. The modern system of international relations features an incredible complexity of its processes and events. Thus, in diﬀerent countries there is a transformation of the existing state-legal forms as well as crises of government institutions. New types of states are appearingon the political map of the world while the established forms of international cooperationare changing. Nevertheless, states are still the main participants of international communication.

MATHERIALS AND METHODS. Therefore, to perform an in-depth analysis of the modern international system of international relations as a whole and the position of individual states in it with quantitative indicators, one needs to create research tools that allow to make a qualitative assessment. We consider the category of “statehood” to be one of the main categories able to most fully assess the qualitative position of certain states in the world system and analyze various external manifestations of their essential and most signiﬁcant characteristics. This article is devoted to a theoretical analysis of the category of “statehood” as a concept through which the unchanging essence of the state as a political unity and as a participant of international communication as well as the description of the forms of its external representation in international relations.

RESEARCH RESULTS. Statehood appears as a comprehensive characteristic of the state organization of society and the degree of development of its intangible culture, which is externally distributed in various methods and forms of the political and legal structure of this society and the basis for the historical continuity of its development. Statehood is not identical to the concept of the state, as it is a qualitative characteristic of a state-organized society. Statehood is a reﬂection of the internal organizing essence of the state. The carrier of statehood is the people.

DISCUSSION AND CONCLUSIONS. The main conclusions of the conducted research are as follows. First, the main forms of external realization of statehood in international law are the authority of the state and its national state tradition. The authority of the state is connected with the concept of international legality and is based on the constant and conscious adherence of the state to the general principles and principles of international law. National state tradition characterizes the state as a participant in international legal relations, possessing its own established state culture. Secondly, the modern system of international relations appears as a system of relations of authority of states, behind which there are all their diverse and multifaceted statuses and characteristics.
Building communities and professionalism through student-organised conferences
Authors: Banderlipe I McRhon , Ingram Valerie
Published: July 2024

Doctoral training operates under the backdrop of a neocolonial relationship in higher education governing universities and their students, as reflected in academic regulations and supervision. When professional and academic standards governing doctoral students dominate academia, this article mediates the affordances institutions can provide and the capabilities students can deliver by organising their learning activities such as doctoral conferences. In the first section, this article frames the contextual and professional imperatives around doctoral students’ organisation of their conference. Secondly, the article outlines a range of interdependent activities that have allowed students to facilitate and develop their networks and capabilities in organising doctoral conferences. Preliminary findings from this case study were drawn to understand how student-organised conferences might catalyse the development of communities and solidarities needed to thrive in their studies. Finally, the authors provide reflections on the impact of the neoliberal and multidisciplinary nature of the research that doctoral students encounter during their PhD and the possibilities of a transformative learning experience.
Impacto da COVID-19 na educação de crianças e adolescentes público-alvo do atendimento educacional especializado e em seus familiares e cuidadores
Authors: Colichi Rosana , Jimenez Figueroa Andres Eduardo , Lima Silvana Andrea Molina
Published: July 2024

The study analyzed the impacts of the pandemic on children and adolescents, the target audience for specialized educational services (AEE) and their families and caregivers. Descriptive cross-sectional study, with data collected from 97 guardians in a city in the interior of São Paulo, Brazil. The researchers used a form they developed, and the data were analyzed statistically, considering p >= 0.05. Results point to differences in the profile and changes occurring in teaching between public and private students concerning support from health and education professionals and technological infrastructure. Regarding pedagogical issues, most participants reported problems related to low adherence to distance learning and the institution’s conduct, materials and resources offered. Challenges for parents and caregivers were revealed in the form of emotional exhaustion and changes in routine, with associations to the type of school, degree of difficulty, number of people living in the house, degree of disability of the child/adolescent, and difficulty in accepting the role of educator. These results suggest that families and caregivers were impacted by the pandemic, affecting the care tasks and education of these children and adolescents.
Certified Policy Verification and Synthesis for MDPs under Distributional Reach-Avoidance Properties
Authors: Akshay S. , Chatterjee Krishnendu , Meggendorfer Tobias , Žikelić Đorđe
Published: July 2024

Markov Decision Processes (MDPs) are a classical model for decision making in the presence of uncertainty. Often they are viewed as state transformers with planning objectives defined with respect to paths over MDP states. An increasingly popular alternative is to view them as distribution transformers, giving rise to a sequence of probability distributions over MDP states. For instance, reachability and safety properties in modeling robot swarms or chemical reaction networks are naturally defined in terms of probability distributions over states. Verifying such distributional properties is known to be hard and often beyond the reach of classical state-based verification techniques.

In this work, we consider the problems of certified policy (i.e. controller) verification and synthesis in MDPs under distributional reach-avoidance specifications. By certified we mean that, along with a policy, we also aim to synthesize a (checkable) certificate ensuring that the MDP indeed satisfies the property. Thus, given the target set of distributions and an unsafe set of distributions over MDP states, our goal is to either synthesize a certificate for a given policy or synthesize a policy along with a certificate, proving that the target distribution can be reached while avoiding unsafe distributions. To solve this problem, we introduce the novel notion of distributional reach-avoid certificates and present automated procedures for (1) synthesizing a certificate for a given policy, and (2) synthesizing a policy together with the certificate, both providing formal guarantees on certificate correctness. Our experimental evaluation demonstrates the ability of our method to solve several non-trivial examples, including a multi-agent robot-swarm model, to synthesize certified policies and to certify existing policies.
CompetEvo: Towards Morphological Evolution from Competition
Authors: Huang Kangyao , Guo Di , Zhang Xinyu , Ji Xiangyang , Liu Huaping
Published: July 2024

Training an agent to adapt to specific tasks through co-optimization of morphology and control has widely attracted attention. However, whether there exists an optimal configuration and tactics for agents in a multiagent competition scenario is still an issue that is challenging to definitively conclude. In this context, we propose competitive evolution (CompetEvo), which co-evolves agents' designs and tactics in confrontation. We build arenas consisting of three animals and their evolved derivatives, placing agents with different morphologies in direct competition with each other. The results reveal that our method enables agents to evolve a more suitable design and strategy for fighting compared to fixed-morph agents, allowing them to obtain advantages in combat scenarios. Moreover, we demonstrate the amazing and impressive behaviors that emerge when confrontations are conducted under asymmetrical morphs.
Dual Enhancement in ODI Super-Resolution: Adapting Convolution and Upsampling to Projection Distortion
Authors: Ji Xiang , Xu Changqiao , Zhong Lujie , Yang Shujie , Xiao Han , Muntean Gabriel-Miro
Published: July 2024

Omnidirectional images (ODIs) demand considerably higher resolution to ensure high quality across all viewports. Traditional convolutional neural networks (CNN)-based single-image super-resolution (SISR) networks, however, are not effective for spherical ODIs. This is due to the uneven pixel density distribution and varying texture complexity in different regions that arise when projecting from a sphere to a plane. Additionally, the computational and memory costs associated with large-sized ODIs present a challenge for real-world application. To address these issues, we propose an efficient distortion-adaptive super-resolution network (ODA-SRN). Specifically, ODA-SRN employs a series of specially designed Distortion Attention Block Groups (DABG) as its backbone. Our Distortion Attention Blocks (DABs) utilize multi-segment parameterized convolution to generate dynamic filters, which compensate for distortion and texture fading during feature extraction. Moreover, we introduce an upsampling scheme that accounts for the dependence of pixel position and distortion degree to achieve pixel-level distortion offset. A comprehensive set of results demonstrates that our ODA-SRN significantly improves the super-resolution performance for ODIs, both quantitatively and qualitatively, when compared to other state-of-the-art methods.
Do You Remember the Future? Weak-to-Strong Generalization in 3D Object Detection
Authors: Gambashidze Alexander , Dadukin Aleksandr , Golyadkin Maxim , Razzhivina Maria , Makarov Ilya
Published: July 2024

This paper demonstrates a novel method for LiDAR-based 3D object detection, addressing major field challenges: sparsity and occlusion. Our approach leverages temporal point cloud sequences to generate frames that provide comprehensive views of objects from multiple angles. To address the challenge of generating these frames in real-time, we employ Knowledge Distillation within a Teacher-Student framework, allowing the Student model to emulate the Teacher’s advanced perception. We pioneered the application of weak-to-strong generalization in computer vision by training our Teacher model on enriched, object-complete data. In this demo, we showcase the exceptional quality of labels produced by the X-Ray Teacher on object-complete frames, showing our method distilling its knowledge to enhance object 3D detection models.
NEGOTIATOR: A Comprehensive Framework for Human-Agent Negotiation Integrating Preferences, Interaction, and Emotion
Authors: Keskin Mehmet Onur , Buzcu Berk , Koçyiğit Berkecan , Çakan Umut , Doğru Anıl , Aydoğan Reyhan
Published: July 2024

The paper introduces a comprehensive human-agent negotiation framework designed to facilitate the development and evaluation of research studies on human-agent negotiation without building each component from scratch. Leveraging the interoperability and reusability of its components, this framework offers various functionalities, including speech-to-text conversion, emotion recognition, a repository of negotiation strategies, and an interaction manager capable of managing gestures designed for Nao, Pepper, and QT, and coordinating message exchanges in a turn-taking fashion. This framework aims to lower the entry barrier for researchers in human-agent negotiation by providing a versatile platform that supports a wide range of research directions, including affective computing, natural language processing, decision-making, and non-verbal communication.
Interactive Visual Learning for Stable Diffusion
Authors: Lee Seongmin , Hoover Benjamin , Strobelt Hendrik , Wang Zijie J. , Peng ShengYun , Wright Austin , Li Kevin , Park Haekyu , Yang Haoyang , Chau Duen Horng
Published: July 2024

Diffusion-based generative models’ impressive ability to create convincing images has garnered global attention. However, their complex internal structures and operations often pose challenges for non-experts to grasp. We introduce Diffusion Explainer, the first interactive visualization tool designed to elucidate how Stable Diffusion transforms text prompts into images. It tightly integrates a visual overview of Stable Diffusion’s complex components with detailed explanations of their underlying operations. This integration enables users to fluidly transition between multiple levels of abstraction through animations and interactive elements. Offering real-time hands-on experience, Diffusion Explainer allows users to adjust Stable Diffusion’s hyperparameters and prompts without the need for installation or specialized hardware. Accessible via users’ web browsers, Diffusion Explainer is making significant strides in democratizing AI education, fostering broader public access. More than 7,200 users spanning 113 countries have used our open-sourced tool at https://poloclub.github.io/diffusion-explainer/. A video demo is available at https://youtu.be/MbkIADZjPnA.
DFRP: A Dual-Track Feedback Recommendation System for Educational Resources
Authors: Meng ChaoJun , Pan Changfan , Li Zilong , Zhou Cong , Cao Xinran , Zhu Jia
Published: July 2024

The educational disparities among different regions are remarkably significant. The educational resource platform can effectively bridge the educational capability gap between regions. Most of the existing recommendation algorithms only consider interaction history, while we argue that the dependencies between knowledge points and education-related features are crucial for education resource recommendations. To address this, we propose DFRP, an educational resource recommendation platform based on knowledge graphs(KGs) and educational scale feedback. DFRP employs a recommendation algorithm based on teaching pathways and educational dimensions to achieve accurate recommendations and active feedback on educational resources. We also provide a detailed description of the system framework and present a demonstration scenario that uses educational scales for active feedback and KGs to show knowledge point dependencies.
ComVas: Contextual Moral Values Alignment System
Authors: Padhi Inkit , Dognin Pierre , Rios Jesus , Luss Ronny , Achintalwar Swapnaja , Riemer Matthew , Liu Miao , Sattigeri Prasanna , Nagireddy Manish , Varshney Kush R. , Bouneffouf Djallel
Published: July 2024

In contemporary society, the integration of artificial intelligence (AI) systems into various aspects of daily life raises significant ethical concerns. One critical aspect is to ensure that AI systems align with the moral values of the endusers. To that end, we introduce the Contextual Moral Value Alignment System, ComVas. Unlike traditional AI systems which have moral values predefined, ComVas empowers users to dynamically select and customize the desired moral values thereby guiding the system’s decision-making process. Through a user-friendly interface, individuals can specify their preferred morals, allowing the system to steer the model’s responses and actions accordingly. ComVas utilizes advanced natural language processing techniques to engage with the users in a meaningful dialogue, understanding their preferences, and reasoning about moral dilemmas in diverse contexts. This demo article showcases the functionality of ComVas, illustrating its potential to foster ethical decision-making in AI systems while respecting individual autonomy and promoting user-centric design principles.
AADMIP: Adversarial Attacks and Defenses Modeling in Industrial Processes
Authors: Pozdnyakov Vitaliy , Kovalenko Aleksandr , Makarov Ilya , Drobyshevskiy Mikhail , Lukyanov Kirill
Published: July 2024

The development of the smart manufacturing trend includes the integration of Artificial Intelligence technologies into industrial processes. One example of such implementation is deep learning models that diagnose the current state of a technological process. Recent studies have demonstrated that small data perturbations, named adversarial attacks, can significantly affect the correct predictions of such models. This fact is critical in industrial systems, where AI-based decisions can be made to manage physical equipment. In this work, we present a system which can help to evaluate the robustness of technological process diagnosis models to adversarial attacks, as well as consider protection options. We briefly review the system's modules and also consider some useful applications. Our demo video is available at: http://tinyurl.com/3by9zcj5
Probabilistic Contrastive Learning for Domain Adaptation
Authors: Li Junjie , Zhang Yixin , Wang Zilei , Hou Saihui , Tu Keyu , Zhang Man
Published: July 2024

Contrastive learning has shown impressive success in enhancing feature discriminability for various visual tasks in a self-supervised manner, but the standard contrastive paradigm (features+l2 normalization) has limited benefits when applied in domain adaptation. We find that this is mainly because the class weights (weights of the final fully connected layer) are ignored in the domain adaptation optimization process, which makes it difficult for features to cluster around the corresponding class weights. To solve this problem, we propose the simple but powerful Probabilistic Contrastive Learning (PCL), which moves beyond the standard paradigm by removing l2 normalization and replacing the features with probabilities. PCL can guide the probability distribution towards a one-hot configuration, thus minimizing the discrepancy between features and class weights. We conduct extensive experiments to validate the effectiveness of PCL and observe consistent performance gains on five tasks, i.e., Unsupervised/Semi-Supervised Domain Adaptation (UDA/SSDA), Semi-Supervised Learning (SSL), UDA Detection and Semantic Segmentation. Notably, for UDA Semantic Segmentation on SYNTHIA, PCL surpasses the sophisticated CPSL-D by 2% in terms of mean IoU with a much lower training cost (PCL: 1*3090, 5 days v.s. CPSL-D: 4*V100, 11 days). Code is available at https://github.com/ljjcoder/Probabilistic-Contrastive-Learning.
Advancing Medical Image Segmentation via Self-supervised Instance-adaptive Prototype Learning
Authors: Liang Guoyan , Zhou Qin , Chen Jingyuan , Wang Zhe , Yao Chang
Published: July 2024

Medical Image Segmentation (MIS) plays a crucial role in medical therapy planning and robot navigation. Prototype learning methods in MIS focus on generating segmentation masks through pixel-to-prototype comparison. However, current approaches often overlook sample diversity by using a fixed prototype per semantic class and neglect intra-class variation within each input. In this paper, we propose to generate instance-adaptive prototypes for MIS, which integrates a common prototype proposal (CPP) capturing common visual patterns and an instance-specific prototype proposal (IPP) tailored to each input. To further account for the intra-class variation, we propose to guide the IPP generation by re-weighting the intermediate feature map according to their confidence scores. These confidence scores are hierarchically generated using a transformer decoder. Additionally we introduce a novel self-supervised filtering strategy to prioritize the foreground pixels during the training of the transformer decoder. Extensive experiments demonstrate favorable performance of our method.
C3L: Content Correlated Vision-Language Instruction Tuning Data Generation via Contrastive Learning
Authors: Ma Ji , Suo Wei , Wang Peng , Zhang Yanning
Published: July 2024

Vision-Language Instruction Tuning (VLIT) is a critical training phase for Large Vision-Language Models (LVLMs). With the improving capabilities of open-source LVLMs, researchers have increasingly turned to generate VLIT data by using open-source LVLMs and achieved significant progress. However, such data generation approaches are bottlenecked by the following challenges: 1) Since multi-modal models tend to be influenced by prior language knowledge, directly using LVLMs to generate VLIT data would inevitably lead to low content relevance between generated data and images. 2) To improve the ability of the models to generate VLIT data, previous methods have incorporated an additional training phase to boost the generative capacity. This process hurts the generalization of the models to unseen inputs (i.e., “exposure bias” problem). In this paper, we propose a new Content Correlated VLIT data generation via Contrastive Learning (C3L). Specifically, we design a new content relevance module which enhances the content relevance between VLIT data and images by computing Image Instruction Correspondence Scores S(I2C). Moreover, a contrastive learning module is introduced to further boost the VLIT data generation capability of the LVLMs. A large number of automatic measures on four benchmarks show the effectiveness of our method.
To Promote Full Cooperation in Social Dilemmas, Agents Need to Unlearn Loyalty
Authors: Leung Chin-wing , Lenaerts Tom , Turrini Paolo
Published: July 2024

If given the choice, what strategy should agents use to switch partners in strategic social interactions? While many analyses have been performed on specific switching heuristics, showing how and when these lead to more cooperation, no insights have been provided into which rule will actually be learnt by agents when given the freedom to do so.

Starting from a baseline model that has demonstrated the potential of rewiring for cooperation, we provide answers to this question over the full spectrum of social dilemmas. Multi-agent Q-learning with Boltzmann exploration is used to learn when to sever or maintain an association. In both the Prisoner's Dilemma and the Stag Hunt games we observe that the Out-for-Tat rewiring rule, breaking ties with other agents choosing socially undesirable actions, becomes dominant, confirming at the same time that cooperation flourishes when rewiring is fast enough relative to imitation. Nonetheless, in the transitory region before full cooperation, a Stay strategy, keeping a connection at all costs, remains present, which shows that loyalty needs to be overcome for full cooperation to emerge.

In conclusion, individuals learn cooperation-promoting rewiring rules but need to overcome a kind of loyalty to achieve full cooperation in the full spectrum of social dilemmas.
ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformers
Authors: Mei Luoyu , Wang Shuai , Cheng Yun , Liu Ruofeng , Yin Zhimeng , Jiang Wenchao , Wang Shuai , Gong Wei
Published: July 2024

Semantic recognition is pivotal in virtual reality (VR) applications, enabling immersive and interactive experiences. A promising approach is utilizing millimeter-wave (mmWave) signals to generate point clouds. However, the high computational and memory demands of current mmWave point cloud models hinder their efficiency and reliability. To address this limitation, our paper introduces ESP-PCT, a novel Enhanced Semantic Performance Point Cloud Transformer with a two-stage semantic recognition framework tailored for VR applications. ESP-PCT takes advantage of the accuracy of sensory point cloud data and optimizes the semantic recognition process, where the localization and focus stages are trained jointly in an end-to-end manner. We evaluate ESP-PCT on various VR semantic recognition conditions, demonstrating substantial enhancements in recognition efficiency. Notably, ESP-PCT achieves a remarkable accuracy of 93.2% while reducing the computational requirements (FLOPs) by 76.9% and memory usage by 78.2% compared to the existing Point Transformer model simultaneously. These underscore ESP-PCT's potential in VR semantic recognition by achieving high accuracy and reducing redundancy. The code and data of this project are available at \url{https://github.com/lymei-SEU/ESP-PCT}.
Motion-Aware Heatmap Regression for Human Pose Estimation in Videos
Authors: Song Inpyo , Lee Jongmin , Ryu Moonwook , Lee Jangwon
Published: July 2024

We present an approach to solving 2D human pose estimation in videos. The problem of human pose estimation in videos differs from estimating human poses in static images since videos contain a lot of motion related information. Thus, we investigate how to utilize by the information of the human body movements across in a sequence of video frames for estimating human poses in videos. To do this, we introduce a novel heatmap regression method what we call motion-aware heatmap regression. Our approach computes motion vectors in joint keypoints from adjacent frames. We then design a new style of heatmap that we call Motion-Aware Heatmaps to reflect the motion uncertainty of each joint point. Unlike traditional heatmaps, our motion-aware heatmaps not only consider the current joint locations but also account how joints move over time. Furthermore, we introduce a simple yet effective framework designed to incorporate motion information into heatmap regression. We evaluate our motion-aware heatmap regression on PoseTrack(2018, 21) and Sub-JHMDB datasets. Our results validate that the proposed motion-aware heatmaps significantly improve the precision of human pose estimation in videos, particularly in challenging scenarios such as videos like sports game footage with substantial human motions. (Code and related materials are available at https://github.com/Songinpyo/MTPose.)
Learning a Spiking Neural Network for Efficient Image Deraining
Authors: Song Tianyu , Jin Guiyue , Li Pengpeng , Jiang Kui , Chen Xiang , Jin Jiyu
Published: July 2024

Recently, spiking neural networks (SNNs) have demonstrated substantial potential in computer vision tasks. In this paper, we present an Efficient Spiking Deraining Network, called ESDNet. Our work is motivated by the observation that rain pixel values will lead to a more pronounced intensity of spike signals in SNNs. However, directly applying deep SNNs to image deraining task still remains a significant challenge. This is attributed to the information loss and training difficulties that arise from discrete binary activation and complex spatiotemporal dynamics. To this end, we develop a spiking residual block to convert the input into spike signals, then adaptively optimize the membrane potential by introducing attention weights to adjust spike responses in a data-driven manner, alleviating information loss caused by discrete binary activation. By this way, our ESDNet can effectively detect and analyze the characteristics of rain streaks by learning their fluctuations. This also enables better guidance for the deraining process and facilitates high-quality image reconstruction. Instead of relying on the ANN-SNN conversion strategy, we introduce a gradient proxy strategy to directly train the model for overcoming the challenge of training. Experimental results show that our approach gains comparable performance against ANN-based methods while reducing energy consumption by 54%. The code source is available at https://github.com/MingTian99/ESDNet.
STAR: Spatio-Temporal State Compression for Multi-Agent Tasks with Rich Observations
Authors: Li Chao , Hu Yujing , Yang Shangdong , Lv Tangjie , Fan Changjie , Li Wenbin , Zhang Chongjie , Gao Yang
Published: July 2024

This paper focuses on the problem of learning compressed state representations for multi-agent tasks. Under the assumption of rich observation, we pinpoint that the state representations should be compressed both spatially and temporally to enable efficient prioritization of task-relevant features, while existing works typically fail. To overcome this limitation, we propose a novel method named Spatio-Temporal stAte compRession (STAR) that explicitly defines both spatial and temporal compression operations on the learned state representations to encode per-agent task-relevant features. Specifically, we first formalize this problem by introducing Task Informed Partially Observable Stochastic Game (TI-POSG). Then, we identify the spatial representation compression in it as encoding the latent states from the joint observations of all agents, and achieve this by learning representations that approximate the latent states based on the information theoretical principle. After that, we further extract the task-relevant features of each agent from these representations by aligning them based on their reward similarities, which is regarded as the temporal representation compression. Structurally, we implement these two compression by learning a set of agent-specific decoding functions and incorporate them into a critic shared by agents for scalable learning. We evaluate our method by developing decentralized policies on 12 maps of the StarCraft Multi-Agent Challenge benchmark, and the superior performance demonstrates its effectiveness.
RoboFusion: Towards Robust Multi-Modal 3D Object Detection via SAM
Authors: Song Ziying , Zhang Guoxing , Liu Lin , Yang Lei , Xu Shaoqing , Jia Caiyan , Jia Feiyang , Wang Li
Published: July 2024

Multi-modal 3D object detectors are dedicated to exploring secure and reliable perception systems for autonomous driving (AD). Although achieving state-of-the-art (SOTA) performance on clean benchmark datasets, they tend to overlook the complexity and harsh conditions of real-world environments. With the emergence of visual foundation models (VFMs), opportunities and challenges are presented for improving the robustness and generalization of multi-modal 3D object detection in AD. Therefore, we propose RoboFusion, a robust framework that leverages VFMs like SAM to tackle out-of-distribution (OOD) noise scenarios. We first adapt the original SAM for AD scenarios named SAM-AD. To align SAM or SAM-AD with multi-modal methods, we then introduce AD-FPN for upsampling the image features extracted by SAM. We employ wavelet decomposition to denoise the depth-guided images for further noise reduction and weather interference. At last, we employ self-attention mechanisms to adaptively reweight the fused features, enhancing informative features while suppressing excess noise. In summary, RoboFusion significantly reduces noise by leveraging the generalization and robustness of VFMs, thereby enhancing the resilience of multi-modal 3D object detection. Consequently, RoboFusion achieves SOTA performance in noisy scenarios, as demonstrated by the KITTI-C and nuScenes-C benchmarks. Code is available at https://github.com/adept-thu/RoboFusion.
A Consistency and Integration Model with Adaptive Thresholds for Weakly Supervised Object Localization
Authors: Su Hao , Yang Meng
Published: July 2024

Weakly Supervised Object Localization (WSOL) is a challenging task, which aims to learn object localization with less costly image-level labels. Existing convolution neural network (CNN) based methods tend to focus on discriminative regions of objects, while transformer-based methods overemphasize deep global features powerful for classification and lack the capability to perceive object details, leading to prediction results far from the object boundary. In this paper, we propose a novel Consistency and Integration Model with Adaptive Thresholds (CIAT) that exploits the spatial-semantic consistency between shallow and deep features to activate more object regions and detects the object regions adaptively in different images. First, we introduce a simple plug-and-play consistency and integration module of shallow-deep features (CISD), which utilizes shallow features efficiently to enhance the entire object perception. Then, we design an online adaptive threshold (OAT) based on Bayesian decision theory, which computes a reasonable segmentation threshold adaptive for the localization map of each image, making the predicted bounding box closer to the ground truth. Extensive experiments on two widely used CUB-200-2011 and ILSVRC datasets verify the effectiveness of our methods.
Who Looks like Me: Semantic Routed Image Harmonization
Authors: Sun Jinsheng , Yao Chao , Wang Xiaokun , Guo Yu , Zhang Yalan , Ban Xiaojuan
Published: July 2024

Image harmonization, aiming to seamlessly blend extraneous foreground objects with background images, is a promising and challenging task.Ensuring a synthetic image appears realistic requires maintaining consistency in visual characteristics, such as texture and style, across global and semantic regions.In this paper, We approach image harmonization as a semantic routed style transfer problem, and propose an imageharmonization model by routing semantic similarity explicitly to enhance the consistency of appearance characteristics.To refine calculate the similarity between the composed foreground and background instance, we propose an InstanceSimilarity Evaluation Module(ISEM).To harness analogous semantic information effectively, we further introduceStyle Transfer Block(STB) to establish fine-grained foreground-background semantic correlation.Our method has achieved excellent experimental results on existing datasets and our model outperforms the state-of-the-art by a margin of 0.45 dB on iHarmony4 dataset.
Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion
Authors: Sun Yiming , Cao Bing , Zhu Pengfei , Hu Qinghua
Published: July 2024

Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.
Multi-Granularity Graph-Convolution-Based Method for Weakly Supervised Person Search
Authors: Tai Haichun , Cheng De , Li Jie , Wang Nannan , Gao Xinbo
Published: July 2024

One-step Weakly Supervised Person Search (WSPS) jointly performs pedestrian detection and person Re-IDentification (ReID) only with bounding box annotations, which makes the traditional person ReID problem more suitable and efficient for real-world applications. However, this task is very challenging due to the following reasons: 1) large feature gap between person ReID and general object detection tasks when learning shared representations; 2) difficult pseudo identity estimation for each person image with unrefined raw detection and dramatic scale changes. To address above issues, we propose a multi-granularity graph convolution framework to jointly optimize the aligned task features, as well as to assist the pseudo label estimation. Specifically, the multi-granularity feature alignment module (MFA) in the designed two-branch framework, employs cluster-level bi-directional interaction of various granularity information to narrow down the large feature gap. Further, upon the MFA module, we introduce the multi-granularity graph-convolution-based pseudo-label estimation module, to enhance feature representations for distinguishing diverse identities. Extensive experimental results demonstrate the effectiveness of the proposed method, and show superior performances to state-of-the art methods by a large margin on CUHK-SYSU and PRW datasets.