# Can People Really Do Nothing? Handling Annotation Gaps in ADL Sensor Data
## 1 Introduction  
Learning systems take decisions based on gathering experiences embedded in existing data [1]. However, an amount of labeled data intensely affects the performance of such systems. Learning methods are classified according to the availability of the labeled training data to supervised and unsupervised learning [2]. Acquiring complete labeled data for machine learning is regularly very challenging and prohibitive, therefore involving unlabeled data in the supervised learning process is vastly promising, since it expands the accuracy of those learning methods [3]. Moreover, a mutual hypothesis in supervised learning is that the collected labeled data represents a normal distribution of process characteristics. However, labeled datasets are frequently small due to the expensive labeling cost and/or sampling bias, which mislead the classification process and produce a non-generalized classifier [4]. Recently, hybrid learning methods that can learn from labeled and unlabeled data gained much interest hoping to construct better performing classifiers [5]. These methods can be categorized into three major approaches varying from fully supervised to unsupervised learning [6]: (1) Building an initial classifier based on a set of labeled data then use it for labeling the unlabeled data. Hereafter, a new classifier is built based on both the earlier and the new labeled data. Techniques such as aSelf-Organizing Map (SOM), neural network [7] and 'co-training' using Naive Bayes classifiers or Expectation Maximization (EM) are usually applied in this first approach [8]. (2) The second approach usually generates a data model using all available data by applying either a data density estimator or clustering procedure. Then, the labels are consequently applied to label entire clusters of data or estimate class densities. These class densities comprise labeling of the unlabeled data based on their relative position in the data space with respect to the originally labeled data [9]. One of the known techniques in this category is the probabilistic framework, which applies a mix of Gaussians or Parzen windows for learning [10]. (3) Both labeled and unlabeled data are processed together for creating semi or partially supervised classifier. This category is between the first and the second categories, since clustering depends on a proper similarity measure and is directed by labeled data. One of the techniques that belongs to this category is a General Fuzzy Min-Max (GFMM) neural network which iteratively processes both labeled and unlabeled data for adjusting hyper box fuzzy clusters [11]. Fairly diverse methods belong to this group are presented in [12]. Comparison between the classifiers based on only limited labeled data and those discussed above which use further unlabeled data shows the feasibility and the advances of the former techniques. One of the well-known methods for conducting unlabeled data is the Expectation Maximization (EM) algorithm. EM handles the unlabeled data as missing data and assuming the generative model such as a mixture of Gaussians for iteratively estimating the model parameters and assigning soft labels to unlabeled examples. Applications such as text classification [13], image retrieval [14] bioinformatics [15], and natural language processing [16] widely use EM. Co-training approach [17] is another common method for integrating unlabeled data, if the data can be classified into two different sets of attributes. Instead, the transduction approach maximizes the classification margins on both labeled and unlabeled data to generate labels to a set of unlabeled data. Nonetheless, it is commonly observed that these methods could reduce performance due to abused model assumptions or sticking in local maxima [18]. However, many empirical studies show that using unlabeled data does not always improve the performance of some classifiers [1].