# Quality Dimensions for Automatic Assessment of Structured Cyber Threat Intelligence Data
## 4 Results Analysis  
### Experiment  
To summarize the consideration of the dimensions provided in the previous section, the architecture of the proposed CTI quality assessment solution is presented in Figure 2. The architecture of the proposed solution includes four major components: parser of structured CTI data, content assessment, provider assessment, and visualization. The parser of structured CTI data parses the file, extracts the STIX objects and calculates the values of indicators required for the measurement of the used dimensions. Initially, thirteen dimensions were selected. After design, the assessment was separated into two distinct parts: content and provider. The content assessment is based on eight dimensions that are enumerated in the second component. The initial dimensions of completeness and extensiveness were joined into a single dimension of completeness. The results of content assessment are presented in four aggregating dimensions that are as follows: actionability technical, actionability tactical, quality score technical, and quality score tactical. The initial dimension of actionability is presented as the finalizing indicator for CTI data quality, since it is an aggregator of several initial dimensions and it is defined as one of the fundamental building blocks of successful incident responses [41]. Actionability is defined at two levels to solve an everlasting dispute between timeliness and completeness, whichever dimension is more important. The quality score of CTI data is defined in a similar way to the actionability dimension. A confidence score for the provider is based on two dimensions: reputation and similarity. The initial dimension of reliability was truncated to the reputation of the provider. It means that the reliability of the provided CTI data is directly associated with the value of the reputation of the CTI provider. Finally, the calculated values are visualized. Visualization eases the understanding and interpretation of the obtained results. Our research indicates a lack of benchmark datasets related to structured CTI products. This problem was noticed by several authors [27, 28, 29] as well. However, only [PERSON] et al. [29] constructed a dataset of structured CTI products and placed it on GitHub [45]. We have used the dataset geosakel77 [45] (42.34 MB), constructed by [PERSON] et al. The purpose of using the public dataset is to provide the results of our method to enable the possibility of comparison for the future related studies. To assess the latest CTI products, we have constructed a dataset called bari24 (980 KB) that includes the newest pulses provided by AlienVault Open Threat Exchange [46] in the last quarter of 2024. To produce defined adversary tactics and techniques, some CTI data from MITRE ATT&CK knowledge base [47] are used to construct a dataset called bariMit (10.1 MB). The description of these CTI data differs greatly from the descriptions used in two previous datasets, since these descriptions include a lot of custom developed objects. Moreover, these descriptions are very extensive; one description on average occupies one 1 MB. So, using the dataset bariMit allows us to test the scalability of our method. In advance, we could say that our method passed the test of scalability, since the processing of 1 MB of CTI data description required 54 s of time (Intel Core i5-8500t processor (ANAFRA, Prague, Czech Republic), 16 GB RAM, SSD NVMe hard drive and Windows 11 operating system). All CTI data are presented in STIX v2.1 schema. Table 2 presents the statistics of used and constructed datasets. We can observe that the datasets are quite diverse if we look at the statistics of the datasets. The Java application was implemented to evaluate the cyber threat intelligence data quality by calculating key quality dimensions on STIX data files. While using the app, the user can upload STIX files in JSON format. The uploaded STIX file is parsed and objects are extracted. Then, dimensions are calculated for each file and detailed results are provided.