# Can People Really Do Nothing? Handling Annotation Gaps in ADL Sensor Data
## 5 Experimental Setup
### Evaluation Procedures and Metrics  
As illustrated in Section 5.1, the input sensor dataset is semi-annotated manually. The main objective of an activity recognition system is to achieve high recognition rates. The recognition performance is affected by different factors. In HMM-based models, the succession of states and the length of each state directly influence both the transition and emission models. This gives special importance to the length of the sampling time interval, \(\Delta t\). In particular, if the sampling time interval is much longer than the activity duration, some activities may be lost and not reflected to the transition model. On the other side, if it is too small, repetition overhead will be increased during the HMM model estimation affecting the state transition distribution. Therefore, a tradeoff is needed. However, the recall-precision pairs are highly dependent on both \(\Delta t\) and the activities' duration. This means that changing \(\Delta t\) to obtain a generalized stable recall-precision analysis such as ROC that is globally applicable to any activity pattern is questionable. Therefore, reaching a general recommendation of the best sampling interval in terms of the intervals of the sensor readings is the best investigation outcome rather than building a conclusion on an activity pattern-dependent outcome. Therefore, we opted to investigate the most proper value of \(\Delta t\) independently of the underlying activity duration. For the used dataset, the average activity interval between sensor readings is about 11 s. We tested different values for \(\Delta t\) and found that an interval of 7 s gives a good basis for all paradigms. Therefore, we recommend using \(\Delta t\) values between 50% and 65% of the average sensor inter-reading intervals. This sampling rate guarantees a relatively \"succinct\" coverage of almost all the recorded readings.  
\begin{table}
\begin{tabular}{c c  
We resampled the input sensor readings using \(\Delta t=7\) s. Then, we applied the Viterbi algorithm to estimate the emission and transition models. The size of the generated samples exceeded 2 M samples. 60% of these samples were used as the training dataset for model estimation. The remaining 40% were used to evaluate the performance.  
Since the main investigation target of the proposed paradigms is the activity recognition performance, we considered four major recognition evaluation metrics. Specifically, we used _recall_, _accuracy_, _precision_, and _specificity_ metrics for this purpose.  
_Recall_ measures the fraction of the correctly recognized instances of an activity to the total number of the retrieved instances of this specific activity, as shown in the following equation:  
\[Recall=\frac{TP}{TP+FN} \tag{5}\]  
where \(TP\) equals the number of true positives and \(FN\) equals the number of false negatives.  
The _Precision_ metric is used to quantify the positive predictive values of recognizing an activity.  
\[Precision=\frac{TP}{TP+FP} \tag{6}\]  
The closeness of the total retrieved activities to the true recognition values is measured by the _Accuracy_,  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{7}\]  
where \(TN\) and \(FP\) are the number of true negatives and false positives, respectively.  
Finally, the True Negative Rate (TNR), or _Specificity_, quantifies how specific the classifier is, with respect to the other activities. In other words, how much the activity does not influence other activities when they are classified.  
\[Specificity=\frac{TN}{TN+FP} \tag{8}\]