## 3 Materials and Methods
### Data Preparation  
The traffic dataset used in this project was obtained from a combination of the following datasets: GitHub, from the repository titled \"Traffic Data Classification Project\" available at GitHub ([[https://github.com/Akanksha242/Traffic-Data-Classification-Project](https://github.com/Akanksha242/Traffic-Data-Classification-Project)]([https://github.com/Akanksha242/Traffic-Data-Classification-Project](https://github.com/Akanksha242/Traffic-Data-Classification-Project)), accessed on 15 October 2024) and collected through the X streaming API; data collected by [PERSON] [10], also available on GitHub ([[https://github.com/sinadabiri/Tweet-Classification-Deep-Learning-Traffic](https://github.com/sinadabiri/Tweet-Classification-Deep-Learning-Traffic)]([https://github.com/sinadabiri/Tweet-Classification-Deep-Learning-Traffic](https://github.com/sinadabiri/Tweet-Classification-Deep-Learning-Traffic)), accessed on 15 October 2024) as well as the X dataset available at Kaggle ([[https://www.kaggle.com/datasets/mounicapremkumar/traffic-analysis-twitter-dataset](https://www.kaggle.com/datasets/mounicapremkumar/traffic-analysis-twitter-dataset)]([https://www.kaggle.com/datasets/mounicapremkumar/traffic-analysis-twitter-dataset](https://www.kaggle.com/datasets/mounicapremkumar/traffic-analysis-twitter-dataset)), accessed on 15 October 2024). The datasets were combined into one dataset that resulted in 140,000 tweets, stored in a CSV file. Each tweet in the dataset is represented by three parameters: label, X ID, and tweet. The label parameter indicates the classification of the tweet (where 1 denotes a traffic-related tweet and 0 denotes a non-traffic-related tweet), the X ID is a unique identifier for each tweet, and the tweet contains the text content of the tweet. To ensure consistency and compatibility across these datasets, we performed data cleaning and standardization procedures. This involved unifying data formats, resolving discrepancies, and eliminating duplicates. We first split all the tweets in the dataset into training and test sets, with 70% of the tweets for training and the remaining 30% for testing. Table 1 displays a sample of the data. In this sample, the label column indicates whether the tweet is related to a traffic incident (1) or not (0). A X snowflake unique ID encodes details such as the creation timestamp, datacenter, ID, and machine ID. The timestamp can be decoded to determine the exact time when the tweet or object was created. The tweet column contains the text of the tweet, which includes details about traffic conditions, incidents, or unrelated content. This dataset is valuable for training models to automatically classify tweets based on their relevance to traffic conditions. In the preprocessing pipeline for the dataset, several essential steps were implemented to prepare the data for the DL and NLP models. First, the tweets were converted to lowercase to ensure uniformity across the dataset. URLs, user mentions, and special characters were removed, as they do not contribute meaningful information to the context of the tweets. Hashtags were removed, but the associated words were retained to preserve context. Tokenization was employed to split the text into individual words or tokens, allowing the analysis of each word separately. A predefined list of English stop words from the NLTK library was used to remove common, non-informative words such as \"the\" and \"is\", which are often irrelevant to the core meaning of the tweets. Additionally, lemmatization was applied to reduce words to their base or root forms, ensuring consistency and minimizing dimensionality. Table 2 displays a sample of the pre-processed data.