DOI: 10.5281/zenodo.18258267
title: PAIRS DATA AND CODE

description: PAIRS: Panofskian Art Interpretation and Retrieval System

Repository Contents

This repository contains the complete implementation, datasets, and experimental data for the paper "PAIRS: A Panofskian Art Interpretation and Retrieval System Based on Multi-Agent Collaboration".



File Structure




File
Size
Description




code.zip
-
Complete source code for PAIRS


annotation-data.zip
-
Pre-computed annotations for all artworks


semart-images-1.zip
-
SemArt dataset images (Part 1)


semart-images-2.zip
-
SemArt dataset images (Part 2)


semart-files.zip
-
SemArt dataset metadata and query files


SUPPLEMENTARY_MATERIALS.md
-
Supplementary materials for the paper






1. Code (code.zip)

Complete implementation of the PAIRS framework.

Structure

code/
├── art-retrieval-research/
│   └── data_annotation_system/    # MC-ArtAnnotate algorithm implementation
├── image_retrieval_platform/      # FastAPI-based retrieval platform
└── experiment/                    # Experiment scripts and baselines


Components



Data Annotation System: Multi-agent collaborative annotation framework with six specialized agents (Form Analysis, Subject Identification, Symbolism, Cultural Context, Technical Material, Coordinator).

Image Retrieval Platform: REST API for art image retrieval with LLM-based verification.

Experiment Code: Scripts for reproducing all experimental results, including baseline implementations (GPT-4o, Claude Sonnet 4.5, InternVL-2.5, BLIP-2, CLIP-Large).


See individual subdirectory README files for detailed usage instructions.



2. Annotation Data (annotation-data.zip)

Pre-computed multi-agent annotations for all artworks in the evaluation datasets.

Contents



Structured annotations following Panofsky's three-level iconographic framework

Global annotations (form, subject, symbolism, cultural context, technical analysis)

Regional annotations with spatial coordinates

Vector embeddings (BGE-large-en-v1.5, 1024 dimensions)




3. SemArt Dataset

The SemArt benchmark is split into three files due to size constraints.

Images



semart-images-1.zip: Artwork images (Part 1)

semart-images-2.zip: Artwork images (Part 2)


Metadata (semart-files.zip)



Image metadata (title, artist, date, type)

Official train/test splits

100 evaluation queries with ground-truth annotations


Dataset Scales

We construct three evaluation scales from SemArt:




Dataset
Images
Purpose




data-100
100
Rapid validation


data-1069
1,069
Primary benchmark (complete test set)


data-5000
5,000
Scalability testing




License

SemArt is released under CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial). See the original dataset page for details.

All experimental datasets are derived from the publicly available SemArt benchmark. The 100 evaluation queries are drawn from SemArt's official test partition, testing iconographic understanding, symbolic interpretation, and cultural context comprehension.

Image Sources: Artwork images in SemArt originate from museum digital collections including the Web Gallery of Art and Wikimedia Commons. Many source images are in the public domain due to artwork age (pre-1900).



4. PAIRS Output and Copyright

Annotation Outputs

PAIRS generates original textual annotations through LLM-based analysis. These annotations constitute derivative scholarly analysis rather than reproduction of copyrighted content. Annotations describe visual elements, interpret symbolic meanings, and provide cultural context—all original analytical text generated by our system.

No Image Reproduction

Our system does not reproduce, modify, or redistribute copyrighted artwork images. During retrieval, the system returns references to original images (URLs) rather than copies. Users access images through their original sources.

Model Outputs

LLM-generated content (annotations and verification reasoning) represents original analytical work. These outputs do not reproduce protected expression from training data but synthesize domain knowledge into new scholarly descriptions.



System Requirements



Python: 3.10+

GPU: CUDA-compatible GPU (recommended for embedding computation)

RAM:



Minimum 16GB for data-1069 dataset

Minimum 64GB for data-5000 dataset



API Keys: Anthropic, OpenAI, or compatible LLM services







Additional Documentation



SUPPLEMENTARY_MATERIALS.md - Detailed supplementary materials including dataset licensing, implementation notes, and best practices