# An Automated Refactoring Approach to Improve IoT Software Quality
## 6 Evaluation
### Correctness Of Refactoring  
HSQLDB benchmark is evaluated by connecting the database under several connection modes, such as in-memory mode, standalone mode and server mode. The evaluation results show that HSQLDB under all modes can connect to the database. We also create database, run SQL statements to insert and delete data, and perform other database operations. They all execute correctly. We run the _JDBCBench_ and _TestBench_ in package _org.Isqldb.test_. The _JDBCBench_ is a test of JDBC connection and _TestBench_ is a stress test of transaction processing. They all run correctly and return a benchmark report.  
For Cassandra, we connect to the database and executed some CQL statements after the refactoring. They all work correctly, and we run all the unit tests in the test folder of Cassandra. A total of 648 unit tests, cover almost all classes in the source code. We find that they all run smoothly without reporting any errors. Cassandra has part of code that already use _ReentrantReadWriteLock_. We then manually refactor them back to synchronized locks, and use our tool to infer the original _ReentrantReadWriteLock_ usage. The method _mayReload()_ in class _CompactStrategyManager_ uses write lock before refactoring. Our tool infers this method uses splitting lock. After manual check, the splitting lock does not change the behavior of method _mayReload()_. The other locks are inferred as same as original usage.  
We used JGroups to deploy the cluster of three nodes after refactoring and successfully completed the communication between them. There are some test programs in JGroups, 49 of which were tested, and they all ran smoothly without reporting any errors.  
Because Freedomotic and MINA have fewer built-in monitors, we manually inspected all the refactored locks. We manually identified (1) if the refactoring had changed the behavior; (2) if a correct kind of lock was inferred, (3) if a lock was inserted to a correct position, (4) if a lock structure was used correctly, and (5) if the critical section was protected safely. During the inspection, we found that  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Benchmark** & **KSLOC** & **\#Built-In Monitors** & \begin{tabular}{c} **\#Downgrading** \\ **Locks** \\ \end{tabular} & \begin{tabular}{c} **\#Spliting** \\ **Locks** \\ \end{tabular} & \
\begin{tabular}{c} **\#Spliting** \\ **\#Read Locks** \\ \end{tabular} &
\begin{tabular}{c} **\#Write Locks** \\ \end{tabular} \\ \hline HSQLDB & 179 & 621 & 6 & 39 & 51 & 525 \\ Cassandra & 432 & 239 & 2 & 24 & 33 & 180 \\ JGroups & 123 & 179 & 5 & 33 & 28 & 113 \\ Freedomotic & 57 & 21 & 2 & 1 & 2 & 16 \\ MINA & 24 & 12 & 0 & 3 & 1 & 8 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Evaluation of the refactoring.  
the refactoring had not changed any behavior of the original programs, and each critical section had been inferred with the kind of lock according to the lock mode and almost all of them were accurate. The position that the lock inserted and the used lock structure were correct. Finally, the critical sections are surrounded by locks and the protection of the critical section is safe.