# The Gap between the Admitted and the Measured Technical Debt: An Empirical Study
## 4 The Experiment
### Results  
As explained earlier in Section 3, the projects were investigated three-times for the accumulated technical debt.  
Firstly, the developers who originally created the source code were asked to report on the self-admitted technical debt (SATD). To do so, an online form was created for them to track the technical debt items. Regarding the research focus, we did not ask them to insert a comprehensive record of the technical debt (such as collected by specialised platforms). Instead, we asked them to insert several focused technical debt items (TDI) for their project:  
* **TDI Title** (e.g., _Authorisation verification on front-end_);
* **TDI Short description** (e.g., _The authorisation is verified on every web page instead of having a central authorisation scheme validation_);
* **TDI Location** (e.g., _/src/components/UI.js_);
* **TDI Type** (_Design_, _Core_);
* **TDI Subtype** (_Code smell_, _Complex classes or methods_, _Grime_, _Incomplete design specification_, _Low-quality code_, _Duplicate code_, _Coding violations_, _Complex code_);
* **TDI Estimated mediation cost** (e.g., _4 h 30 min_).  
The TDIs for every project were reviewed, and only code-related TDIs were summarised (e.g., _incomplete specifications_ were not included). This is how we were given the total self-admitted code technical debt per project.  
Secondly, every project's source code was measured with two platforms: _the SonarQube_ and _the Vector Square_. Both were deliberately not fine-tuned for a special purpose (e.g., setting a custom code-smell or coding standard rules) but were used \"_as-is_\": settings were left in an \"_out-of-the-box_\" manner.  
We translated the reported technical debt amount from an absolute, time-based measure to a relative one. The time-based measure was normalised according to a particularproject total effort. This step was crucial in terms of the ability to compare the technical debt amount between different projects (e.g., _a 1 h technical debt in a 100 h project_ is of equal importance as _10 h in a 1000 h project_--1% in both cases).  
The admitted and the measured technical debt are shown clearly in Figure 2, where the admitted and twice-measured technical debt are shown for every project. The _SATD_ abbreviation in the figure is used for _\"self-admitted technical debt\"_.  
Figure 2 also demonstrates the diversity of the investigated projects in terms of the internal quality trade-off amount, recognised as a technical debt. However, the claimed (admitted and measured) accumulated technical debt amount for every project is below or at the border of 10%, which is commonly understood as a margin below which the accumulated technical debt is of a reasonable amount, without the need to panic. This was also expected, since our research was dealing with production or almost production-ready software. The mild exceptions are projects #3 and #4, where developers admitted to having slightly more technical debt than would be reasonable (11.7% in project #3 and 13.2% in project #4).  
In addition to the relative accumulated reported and admitted technical debt amounts, we are also reporting the technical debt ratings, according to the SQALE method, as explained in Section 2.1. Please see Table 3 for details.  
Figure 2: Measured and self-admitted technical debt values per product.