Instructions to use adopd/LocateAnything-3B-dual-image-tagger-ADOPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use adopd/LocateAnything-3B-dual-image-tagger-ADOPD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="adopd/LocateAnything-3B-dual-image-tagger-ADOPD", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("adopd/LocateAnything-3B-dual-image-tagger-ADOPD", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use adopd/LocateAnything-3B-dual-image-tagger-ADOPD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "adopd/LocateAnything-3B-dual-image-tagger-ADOPD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adopd/LocateAnything-3B-dual-image-tagger-ADOPD",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/adopd/LocateAnything-3B-dual-image-tagger-ADOPD

SGLang

How to use adopd/LocateAnything-3B-dual-image-tagger-ADOPD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "adopd/LocateAnything-3B-dual-image-tagger-ADOPD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adopd/LocateAnything-3B-dual-image-tagger-ADOPD",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "adopd/LocateAnything-3B-dual-image-tagger-ADOPD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adopd/LocateAnything-3B-dual-image-tagger-ADOPD",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use adopd/LocateAnything-3B-dual-image-tagger-ADOPD with Docker Model Runner:
```
docker model run hf.co/adopd/LocateAnything-3B-dual-image-tagger-ADOPD
```

LocateAnything-3B Dual-Image Tagger ADOPD

Thinking with Anchors Project | ADOPD 2026 Paper: Thinking with Anchors: Grounded and Efficient Document Reasoning | ADOPD 2024 Paper | Dataset | Code

Use Restrictions

Non-commercial research use only. The ADOPD fine-tuned checkpoint weights in this repository are provided solely for non-commercial research. Commercial use of these checkpoint weights is not permitted. Users must also comply with every applicable upstream license and acceptable-use term; see USE_RESTRICTIONS.md.

Model Overview

Model developer: Thinking with Anchors project contributors
Base model: nvidia/LocateAnything-3B
Model type: dual-image vision-language region classifier
Task: document entity tagging (Doc2Tag)
Fine-tuning dataset: ADOPD region masks and semantic tags
Input: marked full-page image, region crop, and closed-set prompt
Output: one class in <ref>CLASS</ref> format

Description

This model classifies one supplied region in a document image. The companion pipeline creates two views from the source image and target geometry:

the full document page with the target region highlighted in red; and
a close-up crop of the same region.

Both images and the closed-set class list are sent to LocateAnything in one request. The model returns a single region class.

Training Data And Label Space

The checkpoint was fine-tuned on its 12-class Doc2Tag label space. The current ADOPD2026 release stores updated region labels under:

human_annotated_masks[].vlm_annotation.label

The current public taxonomy contains 12 target labels plus Other / Not Target and is not identical to the checkpoint's training label space. Use --taxonomy legacy for this checkpoint. Fine-tune on the public recipe before using the updated taxonomy as the output space.

Quick Start

git clone https://github.com/SichenZhu/ADOPD2026.git
cd ADOPD2026/release_code

python -m pip install -e model_zoo/common
python -m pip install -e model_zoo/locateanything_3b_tagger

hf download adopd/LocateAnything-3B-dual-image-tagger-ADOPD \
  --local-dir checkpoints/locany-tagger

adopd-locany-tagger-infer \
  --checkpoint checkpoints/locany-tagger \
  --taxonomy legacy \
  --image document.jpg \
  --bbox X0,Y0,X1,Y1 \
  --output prediction.json

The output contains the parsed tag and complete raw model response. Inference requires a CUDA-capable NVIDIA GPU. Eagle source and its compatibility overlay are only required for fine-tuning.

Fine-Tuning And Evaluation

Use adopd-locany-tagger-prepare to build a current-taxonomy recipe from ADOPD2026. Complete training and evaluation commands are in locateanything_3b_tagger.

Limitations

This model classifies a supplied region and does not discover regions. Its training label space differs from the updated public taxonomy. Marking and crop geometry can affect predictions.

License

The ADOPD fine-tuned checkpoint weights are subject to the non-commercial, research-only restriction above. The included NVIDIA License and the terms for the LocateAnything base model and bundled remote-code files also apply. Use is permitted only when all applicable terms are satisfied.

Citation

Please cite the ADOPD 2026 and ADOPD 2024 papers.

@misc{zhu2026thinkingwithanchors,
  title={Thinking with Anchors: Grounded and Efficient Document Reasoning},
  author={Sichen Zhu and Yuchen Zhu and Wenzhuo Xu and Jason Kuen and Wanrong Zhu and Jing Shi and Xuan Shen and Quanyi Wang and Yiwei Wang and Yujun Cai and Bing Shuai and Qin Zhang and Yongxin Chen and Shilong Liu and Molei Tao and Jiuxiang Gu},
  year={2026}
}

@inproceedings{gu2024adopd,
  title={{ADOPD}: A Large-Scale Document Page Decomposition Dataset},
  author={Jiuxiang Gu and Xiangxi Shi and Jason Kuen and Lu Qi and Ruiyi Zhang and Anqi Liu and Ani Nenkova and Tong Sun},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024},
  url={https://openreview.net/forum?id=x1ptaXpOYa}
}