Model Card for MITLL/LADI-v2-classifier-large

LADI-v2-classifier-large is based on microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft and fine-tuned on the MITLL/LADI-v2-dataset. LADI-v2-classifier is trained to identify labels of interest to disaster response managers from aerial images.

📘 NOTE 📘 This model is the main version of the large model which is trained on all splits of the LADI v2 dataset. It is intended for deployment and fine-tuning purposes. If you are interested in reproducing the results of our paper, see the 'reference' versions of the classifiers MITLL/LADI-v2-classifier-small-reference and MITLL/LADI-v2-classifier-large-reference models, which are trained only on the training split of the dataset.

Model Details

Model Description

The model architecture is based on swinv2 and fine-tuned on the LADI v2 dataset, which contains 10,000 post-disaster aerial images from 2015-2023 labeled by volunteers from the Civil Air Patrol. The images are labeled using multi-label classification for the following classes:

• bridges_any
• buildings_any
• buildings_affected_or_greater
• buildings_minor_or_greater
• debris_any
• flooding_any
• flooding_structures
• roads_any
• roads_damage
• trees_any
• trees_damage
• water_any

How to Get Started with the Model

LADI-v2-classifier-large is trained to identify features of interest to disaster response managers from aerial images. Use the code below to get started with the model.

The simplest way to perform inference is using the pipeline interface

from transformers import pipeline
image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg"

pipe = pipeline(model="MITLL/LADI-v2-classifier-large")
print(pipe(image_url))

[{'label': 'buildings_any', 'score': 0.9994631409645081},
 {'label': 'bridges_any', 'score': 0.9981274008750916},
 {'label': 'flooding_structures', 'score': 0.9974740147590637},
 {'label': 'roads_any', 'score': 0.9966784715652466},
 {'label': 'water_any', 'score': 0.9921613335609436},]

For finer-grained control, see below:

from transformers import AutoImageProcessor, AutoModelForImageClassification
import torch
import requests
from PIL import Image
from io import BytesIO

image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg"

img_data = requests.get(image_url).content
img = Image.open(BytesIO(img_data))

processor = AutoImageProcessor.from_pretrained("MITLL/LADI-v2-classifier-large")
model = AutoModelForImageClassification.from_pretrained("MITLL/LADI-v2-classifier-large")

inputs = processor(img, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predictions = torch.sigmoid(logits).detach().numpy()[0]
labels = [(model.config.id2label[idx], predictions[idx]) for idx in range(len(predictions))]
print(labels)

[('bridges_any', 0.9981274008750916),
 ('buildings_any', 0.9994631409645081),
 ('buildings_affected_or_greater', 0.9810144901275635),
 ('buildings_minor_or_greater', 0.010010059922933578),
 ('debris_any', 0.053571492433547974),
 ('flooding_any', 0.9028007388114929),
 ('flooding_structures', 0.9974740147590637),
 ('roads_any', 0.9966784715652466),
 ('roads_damage', 0.6155700087547302),
 ('trees_any', 0.9893258810043335),
 ('trees_damage', 0.49921801686286926),
 ('water_any', 0.9921613335609436)]

Citation

BibTeX:

Paper forthcoming - watch this space for details

---

• Developed by: Jeff Liu, Sam Scheele
• Funded by: Department of the Air Force under Air Force Contract No. FA8702-15-D-0001
• License: MIT
• Finetuned from model: microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft

---

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

This material is based upon work supported by the Department of the Air Force under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Department of the Air Force.

© 2024 Massachusetts Institute of Technology.

The software/firmware is provided to you on an As-Is basis

Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work.