DOI: 10.7910/dvn/qxt5ql
title: Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response

description: The crisis image benchmark dataset consists data from several data sources such as CrisisMMD, data from AIDR and Damage Multimodal Dataset (DMD). The purpose of this work was to develop a consolidated dataset, create non-overlapping train/dev/test set and provide a benchmark results for the community.

We propose new datasets for disaster type detection, and informativeness classification, and damage severity assessment. Moreover, we relabel existing publicly available datasets for new tasks. We identify exact- and near-duplicates to form non-overlapping data splits, and finally consolidate them to create larger datasets. In our extensive experiments, we benchmark several state-of-the-art deep learning models and achieve promising results. We release our datasets and models publicly, aiming to provide proper baselines as well as to spur further research in the crisis informatics community.
&lt;br&gt;
&lt;a href="https://crisisnlp.qcri.org/crisis-image-datasets-asonam20"&gt;https://crisisnlp.qcri.org/crisis-image-datasets-asonam20&lt;/a&gt;
&lt;br&gt;
&lt;strong&gt;The labels in the dataset for different tasks are as follows:&lt;/strong&gt;&lt;br&gt;

&lt;strong&gt;Task 1: Disaster types&lt;/strong&gt;&lt;br&gt;
 &lt;ul&gt;
 &lt;li&gt;Earthquake&lt;/li&gt;
 &lt;li&gt;Fire&lt;/li&gt;
 &lt;li&gt;Flood&lt;/li&gt;
 &lt;li&gt;Hurricane&lt;/li&gt;
 &lt;li&gt;Landslide&lt;/li&gt;
 &lt;li&gt;Not disaster&lt;/li&gt;
 &lt;li&gt;Other disaster&lt;/li&gt;
 &lt;/ul&gt;

&lt;strong&gt;Task 2: Informativeness&lt;/strong&gt;&lt;br&gt;
 &lt;ul&gt;
 &lt;li&gt;Informative&lt;/li&gt;
 &lt;li&gt;Not informative&lt;/li&gt;
 &lt;/ul&gt;

&lt;strong&gt;Task 3: Humanitarian categories&lt;/strong&gt;&lt;br&gt;
 &lt;ul&gt;
 &lt;li&gt;Affected, injured, or dead people&lt;/li&gt;
 &lt;li&gt;Infrastructure and utility damage&lt;/li&gt;
 &lt;li&gt;Not humanitarian&lt;/li&gt;
 &lt;li&gt;Rescue volunteering or donation effort&lt;/li&gt;
 &lt;/ul&gt;

&lt;strong&gt;Task 4: Damage severity&lt;/strong&gt;&lt;br&gt;
 &lt;ul&gt;
 &lt;li&gt;Little or none&lt;/li&gt;
 &lt;li&gt;Mild&lt;/li&gt;
 &lt;li&gt;Severe&lt;/li&gt;
 &lt;/ul&gt;
&lt;br&gt;
&lt;br&gt;

 &lt;p&gt;&lt;strong&gt;Please cite the following papers, if you use any of these resources in your research.&lt;/strong&gt;&lt;/p&gt;
 &lt;ol&gt;
 &lt;li&gt;&lt;strong&gt;Firoj Alam&lt;/strong&gt;, Ferda Ofli, Muhammad Imran, Tanvirul Alam, Umair Qazi, &lt;a href="https://crisisnlp.qcri.org/crisis-image-datasets-asonam20#"&gt;&lt;strong&gt;Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response,&lt;/strong&gt;&lt;/a&gt; In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2020. [&lt;a href="https://crisisnlp.qcri.org/crisis_image_dataset_bib.html"&gt;Bibtex&lt;/a&gt;]&lt;/li&gt;
 &lt;li&gt;&lt;a href="http://sites.google.com/site/firojalam/"&gt;Firoj Alam&lt;/a&gt;, &lt;a href="https://sites.google.com/site/ferdaofli/"&gt;Ferda Ofli&lt;/a&gt;, and &lt;a href="http://mimran.me/"&gt;Muhammad Imran,&lt;/a&gt;&lt;a href="https://arxiv.org/pdf/1805.00713.pdf"&gt;&lt;strong&gt; CrisisMMD: Multimodal Twitter Datasets from Natural Disasters.&lt;/strong&gt;&lt;/a&gt; In Proceedings of the 12th International AAAI Conference on Web and Social Media (ICWSM), 2018, Stanford, California, USA. [&lt;a href="https://crisisnlp.qcri.org/bibs/crisismmd18_bib.html"&gt;Bibtex&lt;/a&gt;]&lt;/li&gt;
 &lt;li&gt;&lt;a href="https://www.researchgate.net/profile/Hussein_Mozannar"&gt;Hussein Mozannar&lt;/a&gt;, &lt;a href="https://www.researchgate.net/profile/Yara_Rizk"&gt;Yara Rizk&lt;/a&gt;, and &lt;a href="https://www.researchgate.net/profile/Mariette_Awad"&gt;Mariette Awad&lt;/a&gt;, &lt;strong&gt;Damage Identification in Social Media Posts using Multimodal Deep Learning&lt;/strong&gt;, In Proc. of ISCRAM, May 2018, pp. 529–543.&lt;/li&gt;
 &lt;/ol&gt;
description: Please download all files then untar them. Once you untar the files, then you should be able to find all files under: crisis_vision_benchmarks/. You can follow the readme to understand the structure of the files and directories.