bluuebunny/crossref_metadata_2025 huggingface dataset:
Extraordinary Burial of the Great Migration Period from Karban-I Necropolis (Northern Altai)
Authors:   Altai State University, Seregin Nikolai , Demin Mikhail ,   Altai State Pedagogical University, Matrenin Sergey ,   Altai State University
Published: June 2023

The article presents the results of the materials study from kurgan 11 of the Karban-I site, excavated in 1989 by the expedition of the Barnaul State Pedagogical Institute. This archaeological complex is located on the left bank of the Katun river, 1.7 km northwest of the Kuyus village of Chemal district of the Altai Republic. The key characteristics of the fixed structures (mound with an oval-shaped crepe-laying; a shallow grave pit, a burial chamber in the form of a stone box) and the method of inhumation (single position of the corpse on the back; orientation of the deceased with his head to the western sector of the horizon; the absence of an accompanying burial of a horse) indicate that that this object belongs to the Karban tradition of ritual practice of the population of Altai of the 2nd century BC – 5th century AD. Analysis of the discovered inventory (horn onlays for a bow, combat knife, typesetting belt, bone arrowheads, awl, pendants and braids made of non-ferrous metal, beads) and its comparison with materials from synchronous complexes in adjacent territories became the basis for determining the chronology of the burial within the Early Xianbei period (2nd – early 3nd centuries AD). It was established that the set of objects included items that were typical for both female and male “standard” rite of the Altai nomads of this period. It is concluded that the buried individual, most likely a male, was a representative of the prosperous stratum of the ordinary population. Judging by the availability of means of long-range and close combat including numerous equipment, he was part of a group of professional warriors. At the same time, the deceased during his lifetime occupied a rather high position in a small group of pastoralists who left the Karban-I necropolis.
Following the Research. Part I
Authors:   State Hermitage Museum, Zasetskaya Irina
Published: June 2023

This article is the first part of a critical review of the works from Vol. 18, no. 2 of the Lower Volga Archaeological Bulletin, dedicated to the Scythian-Sarmatian period and the coverage of the categories of material culture of this era. The issue was published in 2019 and was dedicated to the anniversary of Prof, Irina P. Zasetskaya.
Reconstruction of Original Content of the Kurgan Funeral Vessels Based on Microbial and Enzymatic Parameters
Authors:   Institute of Physicochemical and Biological Problems in Soil Science of the Russian Academy of Sciences, Kashirskaya Natalia , Khomutova Tatiana ,   Institute of Physicochemical and Biological Problems in Soil Science of the Russian Academy of Sciences, Dushchanova Kamilla ,   Institute of Physicochemical and Biological Problems in Soil Science of the Russian Academy of Sciences, Fornasier Flavio ,   CREA, Research Centre for Viticulture and Enology, branch of Gorizia, Kovalev Denis ,   Center of Applied Archeology
Published: June 2023

The original content of ritual vessels from the burials of the two kurgan cemeteries was reconstructed using the multisubstrate testing system of microbial respiration and enzymatic activity of the soil from the pots. For this purpose, a laboratory model experiment was conducted and the decomposition of protein, lipid and polysaccharide organic materials was studied. Basing on the results of the model experiment, most indicative enzymes produced by soil microbial community under decomposition of each type of organic materials were found. They were nonanoate esterase, alkaline phosphatase, acid phosphatase, and leucine-aminopeptidase. The results of the assessment of enzymatic activity made it possible to reconstruct the original contents of burial vessels from two burial mounds “Beysuzhek-35” (Bronze Age) and “Spokoynyy” (Bronze Age and Early Iron Age). We found that most of the pots contained plant food. Animal proteins and fats were in three out of nine pots. One pot was empty or had water in it. In the male burials of the Yamnaya culture, ritual food in pots was more nutritious and included animal fats and proteins, while in the female burial in pots there was a plant starch and protein food. Multisubstrate testing of the respiratory responses of the soil microbial community and determination of the activities of the enzymes nonanoate esterase, alkaline phosphatase, acid phosphatase, and leucine-aminopeptidase are promising approaches to study the type of ritual food in the pots from ancient burials.
Rhodian Amphorae of the 3rd – 2nd Centuries BC from the Krasnodar Museum Collection
Authors:   Saratov State University, Monakhov Sergey , Kuznetsova Elena ,   Saratov State University
Published: June 2023

This paper presents stamped Rhodian containers from the excavations of ancient monuments of the Kuban river region, stored in the Krasnodar State Historical and Archaeological Museum-Reserve named after E.D. Felitsyn. The greater part of the amphorae was found as a result of excavations of Maeotian burials mainly, which contained other imports: black-glazed or red-glazed ceramics, relief bowls, etc. The first part of the publication focuses on characterizing the complexes. It is noted that in some instances we encounter inconsistencies in the dating of different inventory items originating from the same burial. In the second part of the article, single amphorae, which origin cannot reliably be identified, are analyzed. The stamps imprinted on them are of special significance. There are stamps containing new previously unknown combinations of eponyms and fabricants names on three of the amphorae. In two cases, the commonly accepted period of activity of the fabricants Διονύσιος and ΙΜΑ(-) should be prolonged for 10–15 years. The situation with the fabricant Ζωίλος is different. Traditionally, his name was associated with eponyms of the III period (198–161 BC), however, in our case his stamp is on the amphora in combination with the stamp of the eponym dated to the Vb period (125–121 BC) – Τεισαμένος. It is thought that here the point at issue is a homonym. An indirect proof of this is the different typological affiliation of the fabricants’ stamps. Among the Rhodian stamps, there are rectangular unemblemed imprints with the name Ζωίλος and round imprints with the same name around the rose. In the final part, examples of new combinations of stamps of eponyms and fabricants, whose activities do not have chronological gaps, are given as well as vessels with stamps of previously unknown stamps are considered. The amphora stamped by fabricant Μένων II, who worked in the time of the eponyms of periods II and III, is of special interest; the eponymous stamp is reconstructed as may be supposed. In this case, the vessel itself is of interest, representing a later, previously unknown variety of amphorae of the “koroni” variant.
Poltsevskaya Cultures of Primorye in the Context of Ethnocultural Indicators
Authors:   Institute of History, Archaeology and Ethnology of the Peoples of the Far East, Far Eastern Branch of the Russian Academy of Sciences, Dyakova Olga
Published: June 2023

The spreading area of monuments of the Poltsevskaya culture of the Far East is extensive. In Russia, they are located across the territories of the Amur region and of Primorye, in China they spread throughout Manchuria. The time of the functioning of culture falls on a difficult historical period of transition from antiquity to the Middle Ages, including the era of the great migration of peoples. The degree of study of the Poltsevskaya culture varies across territories. In the Amur region, the dynamics of the development of the Poltsevskaya culture was revealed: Zheltyy Yar (7th–6th century BC); Poltsevskaya (6th–2nd–1st centuries BC); Kukelevsky (1st–4th centuries AD), the contact of the Poltsevskaya culture with medieval Tungus-Manchus people (carriers of the Mohe culture) was traced, two locally-chronological groups of monuments Blagoslaveninskaya and Naifeldskaya with Poltsevo-Mohe traditions were identified (4th–9th centuries AD). In China, three of its varieties were distinguished. In Primorye, the study of culture is controversial, which is manifested in the variety of cultural names including those of Suyfunskaya, Olginskaya, Poltsevskaya, Smolninskaya and Nikolaevskaya. All the cultural communities claim to be independent. However, the identified cultural indicators on the single-layer monuments of Primorye: Monakino 4, Wrangel 3, Mikhailovskoye settlement as evidence for a common Poltsevskaya culture or identity developing in time and space. The Poltsevskaya culture traditions are preserved in the material culture of the Far Eastern Paleoasiates (Nivkhs people) up to the present.
A Segment of the Axiological Space of the Russian Media Supertext in the Period of the Special Military Operation in the Ukraine: Blockade and Isolation
Authors:   Ural Federal University, Kupina Natalia A.
Published: June 2023

The article presents a linguo­axiological analysis of a supertext segment of traditional and online Russian media during the period of the special operation in Ukraine (February 24 — December 24, 2022). The aim of the study is to identify the mental specifics of the axiological space of our anxious time, taking into account the accepted distinction between axiologemes, counteraxiologemes, as well as the contextually determined axiological marking of the language units. Among suggested features are: a linguoaxiological interpretation of the markedness of the lexeme blockade, as well as the corresponding attributive and genitive combinations; interpretation of the counteraxiologemes to isolate, isolation; interpretation of groups of verbs that nominate technologies of prohibition and restrictions aimed at excluding Russian civilization from the global environment. The policy of blockade and isolation is stated to contribute to the formation of axiological resistance based on upholding the core values of Russian linguistic culture. The article reveals a fragment of the axiological lexicon, its systematic description being a future task of Russian linguistic axiology.
Media Intelligence as an Essential Characteristic of a Holistic Media Personality
Authors:   Surgut State Pedagogical University, Gavrilov Viktor V.
Published: June 2023

In this article, based on G. Garner’s research on “multiple intelligence”, the author suggests, along with “digital” and “linguistic” (which are directly related to journalism), to identify another type of intelligence — “media intelligence”. The relevance and necessity of research in this direction is explained by the fact that in the era of digitalization of journalism, there is a new social demand for specialists who are able to take into account the needs of the audience when broadcasting information, prepare materials in any sign systems, own digital tools for collecting, processing and relaying data, see the threats of digitalization (fake information, manipulation of consciousness, information noise, etc.) and level them. The author briefly describes various views on the problem of forming a “new generation journalist”. It is concluded that special criteria are needed to assess the effectiveness of the author’s activities. In this regard, it should be noted that drawing up a “media portrait” of a person according to specific and verifiable criteria can be the optimal solution to this problem, an answer to a social request. The author lists the criteria by which, in his opinion, the assessment of the level of media intelligence of a journalist can be carried out, and therefore, the work on the formation of a holistic media personality in the era of digital media will be adjusted in the future.
Geopoetics of Siberia in the Story of N. A. Lukhmanova “Belokrinitsky Archpriest Athanasius”
Authors:   Tyumen State University, Pershina Alyona I.
Published: June 2023

The article examines the features of the geopoetics of schismatic Siberia in the last third of the nineteenth century, presented in the story of Nadezhda Aleksandrovna Lukhmanova “Belokrinitsy Archpriest Athanasius” (1896). The author investigates the interrelation of factual (history of Belokrinitsky priesthood, life of dissenters in Siberia) and mythological principles (unknown Belovodie, hailstones of Kitezh) in the story. For the first time the materials provincial press are considered as a historical source (“Tobolsk diocesan Gazette” 1880–1890.), Which also allows to trace formation of the creative idea of the writer. The mythopoetics of the Siberian city as a city of salvation, the Promised Land, the Underground realm of dissenters, the territory of initiation is revealed. The topos of a merchant’s — schismatic — mansion is represented through the prism of geopoetics of “cultural space”, “secret / reserve place”, etc. The article also considers the “women’s question”, which in this story is connected in the story to the axiology of the old­believers’ family foundations of Siberia. For Lukhmanova, Siberia is the source of creative subjectivity, a reflection based on her “experience of living here”.
Transformation of the teacher’s image in Soviet and Russian Cinematography: Intracultural Dialogue
Authors:   Ural Federal University, Kirillova Natalia B. , Lebedeva Valeria Yu. ,   Ural Federal University
Published: June 2023

The object of the research in the article is a comparative analysis of the visual language of three­dimensional types of stop­motion and computer 3D animation, which actively influences the creation of images of screen culture. The authors note that the visual effects of volumetric stop­motion animation are a kind of proto­form of 3D computer animation, which has been actively used in the creation of special effects in cinema in recent decades. Relying on the methods of cultural­historical and semiotic analysis, the authors consider the evolution of the visual language of animation, which has gone its way from the traditions of hand­drawn and puppet animation to multifunctional computer animation techniques that contribute to the formation of multidimensional images of modern screen culture.
A Machine with Short-Term, Episodic, and Semantic Memory Systems
Authors: Kim Taewoon , Cochez Michael , Francois-Lavet Vincent , Neerincx Mark , Vossen Piek
Published: June 2023

Inspired by the cognitive science theory of the explicit human memory systems, we have modeled an agent with short-term, episodic, and semantic memory systems, each of which is modeled with a knowledge graph. To evaluate this system and analyze the behavior of this agent, we designed and released our own reinforcement learning agent environment, “the Room”, where an agent has to learn how to encode, store, and retrieve memories to maximize its return by answering questions. We show that our deep Q-learning based agent successfully learns whether a short-term memory should be forgotten, or rather be stored in the episodic or semantic memory systems. Our experiments indicate that an agent with human-like memory systems can outperform an agent without this memory structure in the environment.
Cross-Modal Label Contrastive Learning for Unsupervised Audio-Visual Event Localization
Authors: Bao Peijun , Yang Wenhan , Ng Boon Poh , Er Meng Hwa , Kot Alex C.
Published: June 2023

This paper for the first time explores audio-visual event localization in an unsupervised manner. Previous methods tackle this problem in a supervised setting and require segment-level or video-level event category ground-truth to train the model. However, building large-scale multi-modality datasets with category annotations is human-intensive and thus not scalable to real-world applications. To this end, we propose cross-modal label contrastive learning to exploit multi-modal information among unlabeled audio and visual streams as self-supervision signals. At the feature representation level, multi-modal representations are collaboratively learned from audio and visual components by using self-supervised representation learning. At the label level, we propose a novel self-supervised pretext task i.e. label contrasting to self-annotate videos with pseudo-labels for localization model training. Note that irrelevant background would hinder the acquisition of high-quality pseudo-labels and thus lead to an inferior localization model. To address this issue, we then propose an expectation-maximization algorithm that optimizes the pseudo-label acquisition and localization model in a coarse-to-fine manner. Extensive experiments demonstrate that our unsupervised approach performs reasonably well compared to the state-of-the-art supervised methods.
Explicit Invariant Feature Induced Cross-Domain Crowd Counting
Authors: Cai Yiqing , Chen Lianggangxu , Guan Haoyue , Lin Shaohui , Lu Changhong , Wang Changbo , He Gaoqi
Published: June 2023

Cross-domain crowd counting has shown progressively improved performance. However, most methods fail to explicitly consider the transferability of different features between source and target domains. In this paper, we propose an innovative explicit Invariant Feature induced Cross-domain Knowledge Transformation framework to address the inconsistent domain-invariant features of different domains. The main idea is to explicitly extract domain-invariant features from both source and target domains, which builds a bridge to transfer more rich knowledge between two domains. The framework consists of three parts, global feature decoupling (GFD), relation exploration and alignment (REA), and graph-guided knowledge enhancement (GKE). In the GFD module, domain-invariant features are efficiently decoupled from domain-specific ones in two domains, which allows the model to distinguish crowds features from backgrounds in the complex scenes. In the REA module both inter-domain relation graph (Inter-RG) and intra-domain relation graph (Intra-RG) are built. Specifically, Inter-RG aggregates multi-scale domain-invariant features between two domains and further aligns local-level invariant features. Intra-RG preserves taskrelated specific information to assist the domain alignment. Furthermore, GKE strategy models the confidence of pseudolabels to further enhance the adaptability of the target domain. Various experiments show our method achieves state-of-theart performance on the standard benchmarks. Code is available at https://github.com/caiyiqing/IF-CKT.
Bidirectional Optical Flow NeRF: High Accuracy and High Quality under Fewer Views
Authors: Chen Shuo , Yan Binbin , Sang Xinzhu , Chen Duo , Wang Peng , Guo Xiao , Zhong Chongli , Wan Huaming
Published: June 2023

Neural Radiance Fields (NeRF) can implicitly represent 3D-consistent RGB images and geometric by optimizing an underlying continuous volumetric scene function using a sparse set of input views, which has greatly benefited view synthesis tasks. However, NeRF fails to estimate correct geometry when given fewer views, resulting in failure to synthesize novel views. Existing works rely on introducing depth images or adding depth estimation networks to resolve the problem of poor synthetic view in NeRF with fewer views. However, due to the lack of spatial consistency of the single-depth image and the poor performance of depth estimation with fewer views, the existing methods still have challenges in addressing this problem. So this paper proposes Bidirectional Optical Flow NeRF(BOF-NeRF), which addresses this problem by mining optical flow information between 2D images. Our key insight is that utilizing 2D optical flow images to design a loss can effectively guide NeRF to learn the correct geometry and synthesize the right novel view. We also propose a view-enhanced fusion method based on geometry and color consistency to solve the problem of novel view details loss in NeRF. We conduct extensive experiments on the NeRF-LLFF and DTU MVS benchmarks for novel view synthesis tasks with fewer images in different complex real scenes. We further demonstrate the robustness of BOF-NeRF under different baseline distances on the Middlebury dataset. In all cases, BOF-NeRF outperforms current state-of-the-art baselines for novel view synthesis and scene geometry estimation.
Weakly Supervised 3D Multi-Person Pose Estimation for Large-Scale Scenes Based on Monocular Camera and Single LiDAR
Authors: Cong Peishan , Xu Yiteng , Ren Yiming , Zhang Juze , Xu Lan , Wang Jingya , Yu Jingyi , Ma Yuexin
Published: June 2023

Depth estimation is usually ill-posed and ambiguous for monocular camera-based 3D multi-person pose estimation. Since LiDAR can capture accurate depth information in long-range scenes, it can benefit both the global localization of individuals and the 3D pose estimation by providing rich geometry features. Motivated by this, we propose a monocular camera and single LiDAR-based method for 3D multi-person pose estimation in large-scale scenes, which is easy to deploy and insensitive to light. Specifically, we design an effective fusion strategy to take advantage of multi-modal input data, including images and point cloud, and make full use of temporal information to guide the network to learn natural and coherent human motions. Without relying on any 3D pose annotations, our method exploits the inherent geometry constraints of point cloud for self-supervision and utilizes 2D keypoints on images for weak supervision. Extensive experiments on public datasets and our newly collected dataset demonstrate the superiority and generalization capability of our proposed method. Project homepage is at \url{https://github.com/4DVLab/FusionPose.git}.
Defending Backdoor Attacks on Vision Transformer via Patch Processing
Authors: Doan Khoa D. , Lao Yingjie , Yang Peng , Li Ping
Published: June 2023

Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. Along with the improvement in performance, security and robustness of ViTs are also of great importance to study. In contrast to many recent works that exploit the robustness of ViTs against adversarial examples, this paper investigates a representative causative attack, i.e., backdoor. We first examine the vulnerability of ViTs against various backdoor attacks and find that ViTs are also quite vulnerable to existing attacks. However, we observe that the clean-data accuracy and backdoor attack success rate of ViTs respond distinctively to patch transformations before the positional encoding. Then, based on this finding, we propose an effective method for ViTs to defend both patch-based and blending-based trigger backdoor attacks via patch processing.
The performances are evaluated on several benchmark datasets, including CIFAR10, GTSRB, and TinyImageNet, which show the proposedds defense is very successful in mitigating backdoor attacks for ViTs. To the best of our knowledge, this paper presents the first defensive strategy that utilizes a unique characteristic of ViTs against backdoor attacks.
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis
Authors: Fan Wan-Cyuan , Chen Yen-Chun , Chen DongDong , Cheng Yu , Yuan Lu , Wang Yu-Chiang Frank
Published: June 2023

Diffusion models (DMs) have shown great potential for high-quality image synthesis. However, when it comes to producing images with complex scenes, how to properly describe both image global structures and object details remains a challenging task. In this paper, we present Frido, a Feature Pyramid Diffusion model performing a multi-scale coarse-to-fine denoising process for image synthesis. Our model decomposes an input image into scale-dependent vector quantized features, followed by a coarse-to-fine gating for producing image output. During the above multi-scale representation learning stage, additional input conditions like text, scene graph, or image layout can be further exploited. Thus, Frido can be also applied for conditional or cross-modality image synthesis. We conduct extensive experiments over various unconditioned and conditional image generation tasks, ranging from text-to-image synthesis, layout-to-image, scene-graph-to-image, to label-to-image. More specifically, we achieved state-of-the-art FID scores on five benchmarks, namely layout-to-image on COCO and OpenImages, scene-graph-to-image on COCO and Visual Genome, and label-to-image on COCO.
Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment
Authors: Feng Wei , Ju Lie , Wang Lin , Song Kaimin , Zhao Xin , Ge Zongyuan
Published: June 2023

Generalizing a deep learning model to new domains is crucial for computer-aided medical diagnosis systems. Most existing unsupervised domain adaptation methods have made significant progress in reducing the domain distribution gap through adversarial training. However, these methods may still produce overconfident but erroneous results on unseen target images. This paper proposes a new unsupervised domain adaptation framework for cross-modality medical image segmentation. Specifically, We first introduce two data augmentation approaches to generate two sets of semantics-preserving augmented images. Based on the model's predictive consistency on these two sets of augmented images, we identify reliable and unreliable pixels. We then perform a selective entropy constraint: we minimize the entropy of reliable pixels to increase their confidence while maximizing the entropy of unreliable pixels to reduce their confidence. Based on the identified reliable and unreliable pixels, we further propose an adaptive semantic alignment module which performs class-level distribution adaptation by minimizing the distance between same class prototypes between domains, where unreliable pixels are removed to derive more accurate prototypes. We have conducted extensive experiments on the cross-modality cardiac structure segmentation task. The experimental results show that the proposed method significantly outperforms the state-of-the-art comparison algorithms. Our code and data are available at https://github.com/fengweie/SE_ASA.
SEFormer: Structure Embedding Transformer for 3D Object Detection
Authors: Feng Xiaoyu , Du Heming , Fan Hehe , Duan Yueqi , Liu Yongpan
Published: June 2023

Effectively preserving and encoding structure features from objects in irregular and sparse LiDAR points is a crucial challenge to 3D object detection on the point cloud. Recently, Transformer has demonstrated promising performance on many 2D and even 3D vision tasks. Compared with the fixed and rigid convolution kernels, the self-attention mechanism in Transformer can adaptively exclude the unrelated or noisy points and is thus suitable for preserving the local spatial structure in the irregular LiDAR point cloud. However, Transformer only performs a simple sum on the point features, based on the self-attention mechanism, and all the points share the same transformation for value. A such isotropic operation cannot capture the direction-distance-oriented local structure, which is essential for 3D object detection. In this work, we propose a Structure-Embedding transFormer (SEFormer), which can not only preserve the local structure as a traditional Transformer but also have the ability to encode the local structure. Compared to the self-attention mechanism in traditional Transformer, SEFormer learns different feature transformations for value points based on the relative directions and distances to the query point. Then we propose a SEFormer-based network for high-performance 3D object detection. Extensive experiments show that the proposed architecture can achieve SOTA results on the Waymo Open Dataset, one of the most significant 3D detection benchmarks for autonomous driving. Specifically, SEFormer achieves 79.02% mAP, which is 1.2% higher than existing works. https://github.com/tdzdog/SEFormer.
ShadowFormer: Global Context Helps Shadow Removal
Authors: Guo Lanqing , Huang Siyu , Liu Ding , Cheng Hao , Wen Bihan
Published: June 2023

Recent deep learning methods have achieved promising results in image shadow removal. However, most of the existing approaches focus on working locally within shadow and non-shadow regions, resulting in severe artifacts around the shadow boundaries as well as inconsistent illumination between shadow and non-shadow regions. It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. In this work, we first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer, to exploit non-shadow regions to help shadow region restoration. A multi-scale channel attention framework is employed to hierarchically capture the global information. Based on that, we propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions. We conduct extensive experiments on three popular public datasets, including ISTD, ISTD+, and SRD,
to evaluate the proposed method. Our method achieves state-of-the-art performance by using up to 150X fewer model parameters.
CALIP: Zero-Shot Enhancement of CLIP with Parameter-Free Attention
Authors: Guo Ziyu , Zhang Renrui , Qiu Longtian , Ma Xianzheng , Miao Xupeng , He Xuming , Cui Bin
Published: June 2023

Contrastive Language-Image Pre-training (CLIP) has been shown to learn visual representations with promising zero-shot performance. To further improve its downstream accuracy, existing works propose additional learnable modules upon CLIP and fine-tune them by few-shot training sets. However, the resulting extra training cost and data requirement severely hinder the efficiency for model deployment and knowledge transfer. In this paper, we introduce a free-lunch enhancement method, CALIP, to boost CLIP's zero-shot performance via a parameter-free attention module. Specifically, we guide visual and textual representations to interact with each other and explore cross-modal informative features via attention. As the pre-training has largely reduced the embedding distances between two modalities, we discard all learnable parameters in the attention and bidirectionally update the multi-modal features, enabling the whole process to be parameter-free and training-free. In this way, the images are blended with textual-aware signals and the text representations become visual-guided for better adaptive zero-shot alignment. We evaluate CALIP on various benchmarks of 14 datasets for both 2D image and 3D point cloud few-shot classification, showing consistent zero-shot performance improvement over CLIP. Based on that, we further insert a small number of linear layers in CALIP's attention module and verify our robustness under the few-shot settings, which also achieves leading performance compared to existing methods. Those extensive experiments demonstrate the superiority of our approach for efficient enhancement of CLIP. Code is available at https://github.com/ZiyuGuo99/CALIP.
Generating Transferable 3D Adversarial Point Cloud via Random Perturbation Factorization
Authors: He Bangyan , Liu Jian , Li Yiming , Liang Siyuan , Li Jingzhi , Jia Xiaojun , Cao Xiaochun
Published: June 2023

Recent studies have demonstrated that existing deep neural networks (DNNs) on 3D point clouds are vulnerable to adversarial examples, especially under the white-box settings where the adversaries have access to model parameters. However, adversarial 3D point clouds generated by existing white-box methods have limited transferability across different DNN architectures. They have only minor threats in real-world scenarios under the black-box settings where the adversaries can only query the deployed victim model. In this paper, we revisit the transferability of adversarial 3D point clouds. We observe that an adversarial perturbation can be randomly factorized into two sub-perturbations, which are also likely to be adversarial perturbations. It motivates us to consider the effects of the perturbation and its sub-perturbations simultaneously to increase the transferability for sub-perturbations also contain helpful information. In this paper, we propose a simple yet effective attack method to generate more transferable adversarial 3D point clouds. Specifically, rather than simply optimizing the loss of perturbation alone, we combine it with its random factorization. We conduct experiments on benchmark dataset, verifying our method's effectiveness in increasing transferability while preserving high efficiency.
Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer
Authors: He Sunan , Guo Taian , Dai Tao , Qiao Ruizhi , Shu Xiujun , Ren Bo , Xia Shu-Tao
Published: June 2023

Real-world recognition system often encounters the challenge of unseen labels. To identify such unseen labels, multi-label zero-shot learning (ML-ZSL) focuses on transferring knowledge by a pre-trained textual label embedding (e.g., GloVe). However, such methods only exploit single-modal knowledge from a language model, while ignoring the rich semantic information inherent in image-text pairs. Instead, recently developed open-vocabulary (OV) based methods succeed in exploiting such information of image-text pairs in object detection, and achieve impressive performance. Inspired by the success of OV-based methods, we propose a novel open-vocabulary framework, named multi-modal knowledge transfer (MKT), for multi-label classification. Specifically, our method exploits multi-modal knowledge of image-text pairs based on a vision and language pre-training (VLP) model. To facilitate transferring the image-text matching ability of VLP model, knowledge distillation is employed to guarantee the consistency of image and label embeddings, along with prompt tuning to further update the label embeddings. To further enable the recognition of multiple objects, a simple but effective two-stream module is developed to capture both local and global features. Extensive experimental results show that our method significantly outperforms state-of-the-art methods on public benchmark datasets.
NLIP: Noise-Robust Language-Image Pre-training
Authors: Huang Runhui , Long Yanxin , Han Jianhua , Xu Hang , Liang Xiwen , Xu Chunjing , Liang Xiaodan
Published: June 2023

Large-scale cross-modal pre-training paradigms have recently shown ubiquitous success on a wide range of downstream tasks, e.g., zero-shot classification, retrieval and image captioning. However, their successes highly rely on the scale and quality of web-crawled data that naturally contain much incomplete and noisy information (e.g., wrong or irrelevant contents). Existing works either design manual rules to clean data or generate pseudo-targets as auxiliary signals for reducing noise impact, which do not explicitly tackle both the incorrect and incomplete challenges at the same time. In this paper, to automatically mitigate the impact of noise by solely mining over existing data, we propose a principled Noise-robust Language-Image Pre-training framework (NLIP) to stabilize pre-training via two schemes: noise-harmonization and noise-completion. First, in noise-harmonization scheme, NLIP estimates the noise probability of each pair according to the memorization effect of cross-modal transformers, then adopts noise-adaptive regularization to harmonize the cross-modal alignments with varying degrees. Second, in noise-completion scheme, to enrich the missing object information of text, NLIP injects a concept-conditioned cross-modal decoder to obtain semantic-consistent synthetic captions to complete noisy ones, which uses the retrieved visual concepts (i.e., objects’ names) for the corresponding image to guide captioning generation. By collaboratively optimizing noise-harmonization and noise-completion schemes, our NLIP can alleviate the common noise effects during image-text pre-training in a more efficient way. Extensive experiments show the significant performance improvements of our NLIP using only 26M data over existing pre-trained models (e.g., CLIP, FILIP and BLIP) on 12 zero-shot classification datasets (e.g., +8.6% over CLIP on average accuracy), MSCOCO image captioning (e.g., +1.9 over BLIP trained with 129M data on CIDEr) and zero-shot image-text retrieval tasks.
Multi-Modality Deep Network for Extreme Learned Image Compression
Authors: Jiang Xuhao , Tan Weimin , Tan Tian , Yan Bo , Shen Liquan
Published: June 2023

Image-based single-modality compression learning approaches have demonstrated exceptionally powerful encoding and decoding capabilities in the past few years , but suffer from blur and severe semantics loss at extremely low bitrates. To address this issue, we propose a multimodal machine learning method for text-guided image compression, in which the semantic information of text is used as prior information to guide image compression for better compression performance. We fully study the role of text description in different components of the codec, and demonstrate its effectiveness. In addition, we adopt the image-text attention module and image-request complement module to better fuse image and text features, and propose an improved multimodal semantic-consistent loss to produce semantically complete reconstructions. Extensive experiments, including a user study, prove that our method can obtain visually pleasing results at extremely low bitrates, and achieves a comparable or even better performance than state-of-the-art methods, even though these methods are at 2x to 4x bitrates of ours.
Correlation Loss: Enforcing Correlation between Classification and Localization
Authors: Kahraman Fehmi , Oksuz Kemal , Kalkan Sinan , Akbas Emre
Published: June 2023

Object detectors are conventionally trained by a weighted sum of classification and localization losses. Recent studies (e.g., predicting IoU with an auxiliary head, Generalized Focal Loss, Rank & Sort Loss) have shown that forcing these two loss terms to interact with each other in non-conventional ways creates a useful inductive bias and improves performance. Inspired by these works, we focus on the correlation between classification and localization and make two main contributions: (i) We provide an analysis about the effects of correlation between classification and localization tasks in object detectors. We identify why correlation affects the performance of various NMS-based and NMS-free detectors, and we devise measures to evaluate the effect of correlation and use them to analyze common detectors. (ii) Motivated by our observations, e.g., that NMS-free detectors can also benefit from correlation, we propose Correlation Loss, a novel plug-in loss function that improves the performance of various object detectors by directly optimizing correlation coefficients: E.g., Correlation Loss on Sparse R-CNN, an NMS-free method, yields 1.6 AP gain on COCO and 1.8 AP gain on Cityscapes dataset. Our best model on Sparse R-CNN reaches 51.0 AP without test-time augmentation on COCO test-dev, reaching state-of-the-art. Code is available at: https://github.com/fehmikahraman/CorrLoss.
Weakly Supervised 3D Segmentation via Receptive-Driven Pseudo Label Consistency and Structural Consistency
Authors: Lan Yuxiang , Zhang Yachao , Qu Yanyun , Wang Cong , Li Chengyang , Cai Jia , Xie Yuan , Wu Zongze
Published: June 2023

As manual point-wise label is time and labor-intensive for fully supervised large-scale point cloud semantic segmentation, weakly supervised method is increasingly active. However, existing methods fail to generate high-quality pseudo labels effectively, leading to unsatisfactory results. In this paper, we propose a weakly supervised point cloud semantic segmentation framework via receptive-driven pseudo label consistency and structural consistency to mine potential knowledge. Specifically, we propose three consistency contrains: pseudo label consistency among different scales, semantic structure consistency between intra-class features and class-level relation structure consistency between pair-wise categories. Three consistency constraints are jointly used to effectively prepares and utilizes pseudo labels simultaneously for stable training. Finally, extensive experimental results on three challenging datasets demonstrate that our method significantly outperforms state-of-the-art weakly supervised methods and even achieves comparable performance to the fully supervised methods.
FSR: A General Frequency-Oriented Framework to Accelerate Image Super-resolution Networks
Authors: Li Jinmin , Dai Tao , Zhu Mingyan , Chen Bin , Wang Zhi , Xia Shu-Tao
Published: June 2023

Deep neural networks (DNNs) have witnessed remarkable achievement in image super-resolution (SR), and plenty of DNN-based SR models with elaborated network designs have recently been proposed. However, existing methods usually require substantial computations by operating in spatial domain. To address this issue, we propose a general frequency-oriented framework (FSR) to accelerate SR networks by considering data characteristics in frequency domain. Our FSR mainly contains dual feature aggregation module (DFAM) to extract informative features in both spatial and transform domains, followed by a four-path SR-Module with different capacities to super-resolve in the frequency domain. Specifically, DFAM further consists of a transform attention block (TABlock) and a spatial context block (SCBlock) to extract global spectral information and local spatial information, respectively, while SR-Module is a parallel network container that contains four to-be-accelerated branches. Furthermore, we propose an adaptive weight strategy for a trade-off between image details recovery and visual quality. Extensive experiments show that our FSR can save FLOPs by almost 40% while reducing inference time by 50% for other SR methods (e.g., FSRCNN, CARN, SRResNet and RCAN). Code is available at https://github.com/THU-Kingmin/FSR.
Networked Anti-coordination Games Meet Graphical Dynamical Systems: Equilibria and Convergence
Authors: Qiu Zirou , Chen Chen , Marathe Madhav V. , Ravi S. S. , Rosenkrantz Daniel J. , Stearns Richard E. , Vullikanti Anil
Published: June 2023

Evolutionary anti-coordination games on networks capture real-world strategic situations such as traffic routing and market competition. Two key problems concerning evolutionary games are the existence of a pure Nash equilibrium (NE) and the convergence time. In this work, we study these two problems for anti-coordination games under sequential and synchronous update schemes. For each update scheme, we examine two decision modes based on whether an agent considers its own previous action (self essential) or not (self non-essential) in choosing its next action. Using a relationship between games and dynamical systems, we show that for both update schemes, finding an NE can be done efficiently under the self non-essential mode but is computationally intractable under the self essential mode. We then identify special cases for which an NE can be obtained efficiently. For convergence time, we show that the dynamics converges in a polynomial number of steps under the synchronous scheme; for the sequential scheme, the convergence time is polynomial only under the self non-essential mode. Through experiments, we empirically examine the convergence time and the equilibria for both synthetic and real-world networks.
DM²: Decentralized Multi-Agent Reinforcement Learning via Distribution Matching
Authors: Wang Caroline , Durugkar Ishan , Liebman Elad , Stone Peter
Published: June 2023

Current approaches to multi-agent cooperation rely heavily on centralized mechanisms or explicit communication protocols to ensure convergence. This paper studies the problem of distributed multi-agent learning without resorting to centralized components or explicit communication. It examines the use of distribution matching to facilitate the coordination of independent agents. In the proposed scheme, each agent independently minimizes the distribution mismatch to the corresponding component of a target visitation distribution. The theoretical analysis shows that under certain conditions, each agent minimizing its individual distribution mismatch allows the convergence to the joint policy that generated the target distribution. Further, if the target distribution is from a joint policy that optimizes a cooperative task, the optimal policy for a combination of this task reward and the distribution matching reward is the same joint policy. This insight is used to formulate a practical algorithm (DM^2), in which each individual agent matches a target distribution derived from concurrently sampled trajectories from a joint expert policy. Experimental validation on the StarCraft domain shows that combining (1) a task reward, and (2) a distribution matching reward for expert demonstrations for the same task, allows agents to outperform a naive distributed baseline. Additional experiments probe the conditions under which expert demonstrations need to be sampled to obtain the learning benefits.
Correct for Whom? Subjectivity and the Evaluation of Personalized Image Aesthetics Assessment Models
Authors: Goree Samuel , Khoo Weslie , Crandall David J.
Published: June 2023

The problem of image aesthetic quality assessment is surprisingly difficult to define precisely. Most early work attempted to estimate the average aesthetic rating of a group of observers, while some recent work has shifted to an approach based on few-shot personalization. In this paper, we connect few-shot personalization, via Immanuel Kant's concept of disinterested judgment, to an argument from feminist aesthetics about the biased tendencies of objective standards for subjective pleasures. To empirically investigate this philosophical debate, we introduce PR-AADB, a relabeling of the existing AADB dataset with labels for pairs of images, and measure how well the existing groundtruth predicts our new pairwise labels. We find, consistent with the feminist critique, that both the existing groundtruth and few-shot personalized predictions represent some users' preferences significantly better than others, but that it is difficult to predict when and for whom the existing groundtruth will be correct. We thus advise against using benchmark datasets to evaluate models for personalized IAQA, and recommend caution when attempting to account for subjective difference using machine learning more generally.
Heuristic Search for Multi-Objective Probabilistic Planning
Authors: Chen Dillon Z. , Trevizan Felipe , Thiébaux Sylvie
Published: June 2023

Heuristic search is a powerful approach that has successfully been applied to a broad class of planning problems, including classical planning, multi-objective planning, and probabilistic planning modelled as a stochastic shortest path (SSP) problem. Here, we extend the reach of heuristic search to a more expressive class of problems, namely multi-objective stochastic shortest paths (MOSSPs), which require computing a coverage set of non-dominated policies. We design new heuristic search algorithms MOLAO* and MOLRTDP, which extend well-known SSP algorithms to the multi-objective case. We further construct a spectrum of domain-independent heuristic functions differing in their ability to take into account the stochastic and multi-objective features of the problem to guide the search. Our experiments demonstrate the benefits of these algorithms and the relative merits of the heuristics.