## 3 Dataset Curation  
When we initiated this research effort, there was no dataset available for NLP tasks specific to cybersecurity literature [2]. As a result, we proposed a framework for curating a large repository of device information for ICS in our previous work [2]. Using this framework, we are able to identify ICS vendor websites, collect website-accessible documents, and identify documents relevant to the dataset using web scraping, data analytics, and natural language processing (NLP). Our framework starts by determining ICS vendor names by scraping CISA's ICS-Cert website at [[https://us-cert.cisa.gov/ics](https://us-cert.cisa.gov/ics)]([https://us-cert.cisa.gov/ics](https://us-cert.cisa.gov/ics)) (Access on 1 March 2021). These results are combined with the vendor name results obtained utilizing predefined keyword search queries applied to different search engines, including Google, Bing, AOL, and Baidu. The framework then expands vendor names into vendor websites by conducting web searches to identify their most likely website URLs. The resulting names and websites are then classified using a web content-based scoring metric along with Latent Dirichlet Allocation (LDA) to identify vendors-of-interest. This process evaluates the home page content of each vendor's website to determine if that vendor falls within the category of an ICS solutions vendor. Finally, our framework conducts a comprehensive web site crawl through all relevant pages to curate a list of downloadable documents, and proceeds to download each document it located. All downloaded documents are then associated with the corresponding ICS vendor. Based on our results from processing the identified vendor websites, we found that [2]:  
* 3% of the documents were unreadable;
* 5% were scanned documents;
* 29% where not related to ICS products;
* 63% of the downloaded documents are ICS product-related documents.  
When analyzing the 12,581 ICS product-related documents that were found, we could determine that  
* 25% were classified as \"manuals\";
* 69% were classified as \"brochures\";
* 6% were classified as \"catalogs\".  
Downloaded documents may contain regular paragraphs, tables, lists, and images. Text from readable documents was extracted via the PyMuPDF python package, and we leveraged python's Pytesseract package for performing optical character recognition (OCR) with any scanned PDFs. With this approach, we managed to extract 2,160,517 sequences with 41,073,376 words across our curated dataset of ICS documents [8].  
In order to label the sequences in our curated dataset we developed a new mobile cross-platform application based on Xamarin that allows us to manually label sequences extracted from these ICS documents. We first identified three types of claim sequences: generic, device, and cybersecurity claims. Specifying individual claim types facilitates future investigations into claim type detection. We categorized all of these types of claims as \"claim\" labels in this study and removed the sequences with the \"Not Sure\" label from the classification dataset. For more details on this labelling process please refer to [8].  
Figure 2 illustrates the labeled dataset we used to train our classifier models for the Tally-Vet project. In this paper we focus only on identifying \"Claim\" sequences. Figure 3 presents the final class count and distribution of all classes.