|
--- |
|
license: mit |
|
library_name: transformers |
|
tags: |
|
- LADI |
|
- Aerial Imagery |
|
- Disaster Response |
|
- Emergency Management |
|
datasets: |
|
- MITLL/LADI-v2-dataset |
|
widget: |
|
- src: https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg |
|
- example_title: Example classification of flooded scene |
|
pipeline_tag: image-classification |
|
--- |
|
# Model Card for MITLL/LADI-v2-classifier-large |
|
LADI-v2-classifier-large is based on [microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft](https://huggingface.co/microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft) and fine-tuned on the [MITLL/LADI-v2-dataset](https://huggingface.co/datasets/MITLL/LADI-v2-dataset). LADI-v2-classifier is trained to identify labels of interest to disaster response managers from aerial images. |
|
|
|
π __NOTE__ π This model is the main version of the large model which is trained on all splits of the LADI v2 dataset. It is intended for deployment and fine-tuning purposes. If you are interested in reproducing the results of our paper, see the 'reference' versions of the classifiers [MITLL/LADI-v2-classifier-small-reference](https://huggingface.co/MITLL/LADI-v2-classifier-small-reference) and [MITLL/LADI-v2-classifier-large-reference](https://huggingface.co/MITLL/LADI-v2-classifier-large-reference) models, which are trained only on the training split of the dataset. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
The model architecture is based on swinv2 and fine-tuned on the LADI v2 dataset, which contains 10,000 post-disaster aerial images from 2015-2023 labeled by volunteers from the Civil Air Patrol. The images are labeled using multi-label classification for the following classes: |
|
|
|
- bridges_any |
|
- buildings_any |
|
- buildings_affected_or_greater |
|
- buildings_minor_or_greater |
|
- debris_any |
|
- flooding_any |
|
- flooding_structures |
|
- roads_any |
|
- roads_damage |
|
- trees_any |
|
- trees_damage |
|
- water_any |
|
|
|
## How to Get Started with the Model |
|
|
|
LADI-v2-classifier-large is trained to identify features of interest to disaster response managers from aerial images. Use the code below to get started with the model. |
|
|
|
The simplest way to perform inference is using the pipeline interface |
|
|
|
```python |
|
from transformers import pipeline |
|
image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg" |
|
|
|
pipe = pipeline(model="MITLL/LADI-v2-classifier-large") |
|
print(pipe(image_url)) |
|
``` |
|
|
|
``` |
|
[{'label': 'buildings_any', 'score': 0.9994631409645081}, |
|
{'label': 'bridges_any', 'score': 0.9981274008750916}, |
|
{'label': 'flooding_structures', 'score': 0.9974740147590637}, |
|
{'label': 'roads_any', 'score': 0.9966784715652466}, |
|
{'label': 'water_any', 'score': 0.9921613335609436},] |
|
``` |
|
|
|
For finer-grained control, see below: |
|
|
|
```python |
|
from transformers import AutoImageProcessor, AutoModelForImageClassification |
|
import torch |
|
import requests |
|
from PIL import Image |
|
from io import BytesIO |
|
|
|
image_url = "https://fema-cap-imagery.s3.amazonaws.com/Images/CAP_-_Flooding_Spring_2023/Source/IAWG_23-B-5061/A0005/D75_0793_DxO_PL6_P.jpg" |
|
|
|
img_data = requests.get(image_url).content |
|
img = Image.open(BytesIO(img_data)) |
|
|
|
processor = AutoImageProcessor.from_pretrained("MITLL/LADI-v2-classifier-large") |
|
model = AutoModelForImageClassification.from_pretrained("MITLL/LADI-v2-classifier-large") |
|
|
|
inputs = processor(img, return_tensors="pt") |
|
|
|
with torch.no_grad(): |
|
logits = model(**inputs).logits |
|
|
|
predictions = torch.sigmoid(logits).detach().numpy()[0] |
|
labels = [(model.config.id2label[idx], predictions[idx]) for idx in range(len(predictions))] |
|
print(labels) |
|
``` |
|
|
|
``` |
|
[('bridges_any', 0.9981274008750916), |
|
('buildings_any', 0.9994631409645081), |
|
('buildings_affected_or_greater', 0.9810144901275635), |
|
('buildings_minor_or_greater', 0.010010059922933578), |
|
('debris_any', 0.053571492433547974), |
|
('flooding_any', 0.9028007388114929), |
|
('flooding_structures', 0.9974740147590637), |
|
('roads_any', 0.9966784715652466), |
|
('roads_damage', 0.6155700087547302), |
|
('trees_any', 0.9893258810043335), |
|
('trees_damage', 0.49921801686286926), |
|
('water_any', 0.9921613335609436)] |
|
``` |
|
|
|
## Citation |
|
|
|
**BibTeX:** |
|
|
|
``` |
|
``` |
|
|
|
Paper forthcoming - watch this space for details |
|
|
|
--- |
|
|
|
- **Developed by:** Jeff Liu, Sam Scheele |
|
- **Funded by:** Department of the Air Force under Air Force Contract No. FA8702-15-D-0001 |
|
- **License:** MIT |
|
- **Finetuned from model:** [microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft](https://huggingface.co/microsoft/swinv2-large-patch4-window12to16-192to256-22kto1k-ft) |
|
--- |
|
|
|
DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. |
|
|
|
This material is based upon work supported by the Department of the Air Force under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Department of the Air Force. |
|
|
|
Β© 2024 Massachusetts Institute of Technology. |
|
|
|
The software/firmware is provided to you on an As-Is basis |
|
|
|
Delivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work. |