###### Abstract
New York Airplane crashes at Hudson Bay ([[https://www.telegraph.co.uk/technology/twitter/4269765/New-York-plane-crash-Twitter-breaks-the-news-again.html](https://www.telegraph.co.uk/technology/twitter/4269765/New-York-plane-crash-Twitter-breaks-the-news-again.html)]([https://www.telegraph.co.uk/technology/twitter/4269765/New-York-plane-crash-Twitter-breaks-the-news-again.html](https://www.telegraph.co.uk/technology/twitter/4269765/New-York-plane-crash-Twitter-breaks-the-news-again.html)), accessed on 25 July 2021): An eyewitness tweeted about the crash, and the information became the headline of the \"Daily Telegraph\". 4. Boston Bombing ([[https://en.wikipedia.org/wiki/Boston_Marathon_bombing](https://en.wikipedia.org/wiki/Boston_Marathon_bombing)]([https://en.wikipedia.org/wiki/Boston_Marathon_bombing](https://en.wikipedia.org/wiki/Boston_Marathon_bombing)), accessed on 25 July 2021): The eyewitness tweets about the bombing incident were available well before any news channel covered the incident. 5. Westgate Shopping Mall attack ([[https://en.wikipedia.org/wiki/Westgate_shopping_mall_attack](https://en.wikipedia.org/wiki/Westgate_shopping_mall_attack)]([https://en.wikipedia.org/wiki/Westgate_shopping_mall_attack](https://en.wikipedia.org/wiki/Westgate_shopping_mall_attack)), accessed on 25 July 2021): This attack in Nairobi, Kenya, was on Twitter thirty-three minutes before any TV news channels. A detailed study of the literature shows that the information shared by an eyewitness can clearly explain the severity of an incident. An eyewitness cannot be identified by just finding disaster-related keywords. The keywords may have come as a reference in the past, or from replies, re-tweets, or hashtags. Removing noise from such tweets is vital for accurately identifying a disaster event's nature and severity. Furthermore, the institutions responsible for responding to disastrous events need to rely on credible information from eyewitness sources to estimate the actual intensity and impact. Based on this reliable information, the agencies must take adequate measures to take control of the situation. Therefore, credible information is vital for such systems or institutions, and an eyewitness to the event is key in providing such information. We identified from a comprehensive literature study that state-of-the-art approaches have adopted different techniques and features for identifying eyewitness reports. For instance, [PERSON] et al. [14] created an extensive feature list with domain experts' help for extracting feature words to classify eyewitness tweets. The features description of manually crafted categories with example words in each category is shown in Table 1. For example, if a tweet contains \"window shaking\" or \"water in basement\", that tweet is classified under \"reporting small details of surroundings\" and so on. In our previous work [15], we implemented the grammar-rules-based approach for all the features identified in the baseline work of the dictionary-based approach adopted by [PERSON] et al. [14]. We exploited the linguistic features, language structure, and existing word relationships within a sentence. We identified that the relationship among the words in a sentence could be exploited to extract the feature words automatically. For the automatic extraction of features, we defined grammar rules and adopted the manual rule-based classification approach to classify the category of eyewitness messages and achieved the maximum F-Score of 0.81 [15]. In this work, we further fine-tuned the grammar rules and proposed new features for the identification of eyewitness messages. We used feature characteristics and labeled datasets to train several machine learning and deep learning classifiers for classifying eyewitness messages and secured a maximum F-score of 0.93.