dataesr/scientific-paragraphs-categorization huggingface dataset:
Outlines All handcrafted methods for extracting features have produced remarkable results in many HAR applications. These approaches assume that each dataset has a set of features that are representative, allowing a learning model to achieve the best performance. However, handcrafted features require extensive pre-processing. This is time consuming and inefficient because the dataset is manually selected and validated by experts. This reduces adaptability to various environments. This is why HAR algorithms must automatically extract the relevant representations. Methods based on deep learning allow better and higher quality features to be obtained from raw data. Moreover, these features can be learned for any dataset. They can be processed in a supervised or unsupervised manner, for example windows labelled or not with the name of the activity.
In addition, deep learning methods can be end-to-end, i. e. they extract features and perform classification. Thanks to deep learning, great advances have been made in the field of NLP. It allows to represent words, sentences or texts thanks to models, structures and learning methods. These models are able to interpret the semantics of words, to contextualize them, to make prior or posterior correlations between words and thus to increase their performance in terms of sentence or text classification. Moreover, these models are able to automatically extract the right features to accomplish their task. The NLP and HAR domains in smart homes both process data in the form of sequences. In smart homes, sensors generate a stream of events. This stream of events is sequential and ordered like words in a text.
Temporal Data In a smart home, sensors record the actions and interactions with the residents' environment over time. These recordings are the logs of events that capture the actions and activities of daily life. Most sensors only send their status when there is a change in status, to save battery power and also to not overload wireless communications. In addition, sensors may have different triggering times. This results in scattered sampling of the time series and irregular sampling. Therefore, recognizing human activity in a smart home is a pattern recognition problem in time series with irregular sampling, unlike recognizing human activity in videos or wearables. In this section, we describe literature methods for segmentation of the sensor data stream in a smart home. These segmentation methods provide a representation of sensor data for human activity recognition algorithms.
Dynamic Windows (DW) DW uses a non-fixed window size unlike the previous methods. It is a two-stage approach that uses an offline phase and an online phase  [54]. In the offline phase, the data stream is split into EW. From the EW, the "best-fit sensor group" is extracted based on rules and thresholds. Then, for the online phase, the dataset is streamed to the classification algorithm. When it identifies the "best-fit sensor group" in the stream, the classifier associates the corresponding label with the given input segment. Problems can arise if the source dataset is not properly annotated. Quigley et al. [47]  have shown that this approach is inefficient for modeling complex activities. Furthermore, rules and thresholds are designed by experts, manually, which is time consuming.
Time Series Classification The recognition of human activity in a smart home is a problem of pattern recognition in time series with irregular sampling. Therefore more specific machine learning for sequential data analysis have also proven efficient for HAR in smart homes. Indeed, statistical Markov models such as Hidden Markov Models  [29, 55]  and their generalisation, Probabilistic graphical models as Dynamic Bayesian Networks  [56]  can model spatiotemporal information. In the deep learning framework, they have been implemented as Recurrent Neural Networks (RNN). RNN show today a stronger capacity to learn features and represent time series or sequential multi-dimensional data. RNN are designed to take a series of inputs with no predetermined limit on size. RNN remembers the past and its decisions are influenced by what it has learnt from the past.
Interleave and Concurrent Activities Human activities are often carried out in a complex manner. Activities can be carried out in an interleave or concurrent manner. An individual may alternately cook and wash dishes, or cook and listen to music simultaneously, but could just as easily cook and wash dishes alternately while listening to music. The possibilities are infinite in terms of activity scheduling. However, some activities seem impossible to see appearing in the dataset and could be anomalous, such as cooking while the individual sleeps in his room. Researchers are working on this issue. Modeling this type of activity is becoming complex. But it could be modeled as a multi label classification problem. Safyan and. al [70]  have explored this problem using ontology. Their approach uses a semantic segmentation of sensors and activities.
Recent works in life-long learning incorporating deep learning as reviewed in  [84]  could help tackle this issue of temporal drift. In particular, one can imagine that an interactive system can from time to time request labelled data to users to continue to learn and adapt. Such algorithms have been developed under the names of interactive reinforcement learning or active imitation learning in robotics. In  [85]  they allowed the system to learn micro and compound actions while minimising the number of requests for labelled data by choosing when, what information to ask, and to whom to ask for help. Such principles could inspire a smart home system to continue to adapt its model while minimising user intervention and optimising his intervention by pointing out the missing key information.
Variability of Settings Beside these long-term evolutions, the data from one house to another are also very different, and the model learned in one house is hardly applicable in another because of the change in house configuration; sensors equipment; and families' compositions and habits. Indeed, the location, the number and the sensors type of smart homes can influence activity recognition systems performances. Each smart homes can be equipped in different ways and have different architecture in terms of sensors, room configuration, appliance. Some can have a lot of sensors, multiple bathrooms, or bedrooms and contain multiple appliances. When others can be smaller as a single apartment, where sensors can be fewer and have more overlaps and noisy sequences. Due to this difference in house configurations, a model that optimised in the first smart homes could perform poorly in another.
This issue could be solved by collecting a new dataset for each new household to train the models anew, however this is costly as explained in sec. 6. Another solution is to adapt the models learned in a household to another. Transfer learning methods have recently been developed to allow pre-trained deep learning models to be used with different data distributions, as reviewed in  [86]. Transfer learning using deep learning has been successfully applied to time series classification as reviewed in  [87]. For activity recognition, Cook et al. [88]  reviewed the different types of knowledge that could be transferred in traditional machine learning. These methods can be updated with deep learning algorithms and by benefiting from recent advances in transfer learning for deep learning. Furthermore, adaptation to new settings have recently been improved by the development of meta-learning algorithms.
Datasets Datasets are key to train, test and validate activity recognition systems. Datasets were first generated in laboratories. But these records don't allow enough variety and complexity of activities and were not real enough. To overcome these issues public datasets were created from recordings in real homes with volunteer residents. In parallel to being able to compare in the same condition and on the same data, some competitions were created such as Evaluating AAL Systems Through Competitive Benchmarking -AR (EvAAL-AR)  [90]  or UCAmI Cup  [91]. However, the production of datasets is a tedious task and recording campaigns are difficult to manage. They require volunteer actors and apartments or houses equipped with sensors. In addition, data annotation and post-processing take a lot of time.
Real Smart Home Dataset A variety of public real homes datasets exist  [43, 75, 76, 92, 93]. De-la-Hoz et al. [94]  provides an overview of sensor-based datasets used in HAR for smart homes. They compiled documentation and analysis of a wide range of datasets with a list of results and applied algorithms. But such dataset production implies some problems as: sensors type and placement, variability in term of user profile or typology of dwelling and the annotation strategy.
Sensor Type and Positioning Problem When acquiring data in a house, it is difficult to choose the sensors and their numbers and locations. It is important to select sensors that are as minimally invasive as possible in order to respect the privacy of the volunteers  [92]. No cameras or video recordings were used. The majority of sensor-oriented smart home datasets use so-called low-level sensors. These include infrared motion sensors (PIR), magnetic sensors for openings and closures, pressure sensors placed in sofas or beds, sensors for temperature, brightness, monitoring of electricity or water consumption. The location of these sensors is critical to properly capture activity. Strategic positioning allows to accurately capture certain activities. e. g. a water level sensor in the toilet to capture toilet usage or a pressure sensor under a mattress to know if a person is in bed.
There is no precise method or strategy for positioning and installing sensors in homes. CASAS  [75]  researchers have proposed and recommended a number of strategic positions. However, some of these strategic placements can be problematic in terms of evolution. It is possible to imagine that during the life of a house the organization or use of its rooms changes e. g. if a motion sensor is placed above the bed to capture its use. But if the bed is moved to a different place in the room, then the sensor will no longer be able to capture this information. In the context of a dataset and the use of the dataset to validate the algorithms, this constraint is not important. But it becomes important in the context of real applications to evaluate the resilience of algorithms, which must continue to function in case of loss of information.
In addition to positioning, it is important to choose enough sensors to cover a maximum of possible activities. The number of sensors can be very different from one dataset to another. For example, the MIT dataset  [76]  uses 77 and 84 sensors for each of these apartments. The Kasteren dataset  [43]  uses between 14 and 21 sensors. ARAS  [92]  has apartments with 20 sensors. Orange4Home  [93]  is based on an apartment equipped with 236 sensors. This difference can be explained by the different types of dwellings but also by the number and granularity of the activities that we want to recognize. Moreover some dataset are voluntarily over-equipped. There is still no method or strategy to define the number of sensors installed according to an activity list.
Profile and Typology Problem It is important to take into account that there are different typologies of houses: apartment, house, with garden, with floors, without floor, one or more bathrooms, one or more bedrooms. These different types and variabilities of houses lead to difficulties such as: the possibility that the same activity takes place in different rooms. That the investment in terms of number of sensors can be more or less important. Or that the network coverage of the sensors can be problematic. For example, Alerndar et al. [92]  faced a problem of data synchronization. One of their houses required two sensor networks to cover the whole house. They must synchronized the data for dataset needs. It is therefore necessary that the datasets can propose different house configurations in order to evaluate the algorithms in multiple configurations.
Several dataset with several houses exist  [43, 43, 76, 92]. CASAS  [75]  is one of them, with about 30 several houses configurations. These datasets are very often used in the literature  [94]. However the volunteers are mainly elderly people, and cover several age groups is important. A young resident does not have the same behavior as an older one. The Orange4Home dataset  [93]  cover the activity of a young resident. The number of residents is also important. The activity recognition is more complex in the case of multiple residents. This is why several datasets cover this field of research also  [43, 75, 92].
Annotation Problem Dataset annotation is something essential for supervised algorithm training. When creating these datasets it is necessary to deploy strategies to enable this annotation. Such as journal  [43] , smartphone applications  [93] , personal digital assistant (PDA)  [76] , Graphical User Interface (GUI)  [92]  or voice records to annotate the dataset  [43]. As these recordings are made directly by volunteers, they are asked to annotate their own activities. For the MIT dataset  [76] , residents used a PDA to annotate their activities. Every 15 minutes, the PDA beeped to prompt residents to answer a series of questions to annotate their activities, however, several problems were encountered with this method of user self-annotation. However, several problems were encountered with this method of self-annotation by the user, such as some short activities not being entered, errors in label selection, or omissions.
The Diary allowed the volunteers to complete some additional information when wearing a helmet was not possible. However, wearing a helmet all day long remains a constraint. The volunteers of the ARAS dataset  [92]  used a simple Graphical User Interface (GUI) to annotate their activities. Several instances were placed in homes to minimize interruptions in activities and avoid wearing an object such as a helmet all day long. Volunteers were asked to indicate only the beginning of each activity. It is assumed that residents will perform the same activity until the next start of the activity. This assumption reflects a bias that sees human activity as a continuous stream of known activity.
Synthetic Smart Home Dataset The cost to build real smart homes and the collection of datasets for such scenarios is expensive and sometimes infeasible for many projects. Measurements campaigns should include a wide variety of activities and actors. It should be done with sufficient rigor to obtain qualitative data. Moreover, finding the optimal placement of the sensors  [95] , finding appropriate participants  [96, 97]  and the lack of flexibility  [98, 99]  makes the dataset collection difficult. For these reasons researchers imagined smart homes simulation tools  [100]. These simulation tools can be categorized into two main approaches, model-based  [101]  and interactive  [102] , according to Synnott et. al. [103]. The model-based approach uses predefined models of activities to generate synthetic data.
In contrast, the interactive approach relies on having an avatar that can be controlled by a researcher, human participant or simulated participant. Some hybrid simulator as OpenSH  [100]  can combines advantages from both interactive and model-based approaches. In addition, smart homes simulation tool can be focusing on the dataset generation or data visualization. Some simulation tools provide multi-resident or fast forwarding to accelerate the time during execution. These tools allow you to quickly generate data and visualize it. But the capture of activities can be unnatural and not noisy. Some uncertainty may be missing.
Duration Sensor type Table  3 : Example of real datasets of the literature Real datasets such as Orange4Home  [93]  provide a large sensor set. That can help to determine which sensors can be useful for which activity. CASAS  [75]  propose many houses or apartment configurations and topologies with elderly people. Which allows evaluating the adaptability to house topologies. ARAS  [92]  propose younger people and multi-residents' livings. Useful to validate the noisy resilience and segmentation ability of the activity recognition system. The strength of real datasets is their variability, and their representativeness in number and execution of activities. But sensors can be placed too strategically and wisely choose to cover some specific kinds of activities. In some datasets PIR sensors are used as a grid or installed as a checkpoint to track residents trajectory.
# of Houses Year Synthetic datasets allow to quickly evaluate different configuration sensors and topologies. In addition they can produce large amounts of data without real setup or volunteer subjects. The annotation is more precise compared to real dataset methods (diary, smartphone apps, voice records). But activities provided by synthetic datasets are less realistic in terms of execution rhythm and variability. Every individual has its own rhythm in terms of action duration, interval or order. The design of the virtual smart homes can be a tedious task for a non-expert designer. Moreover, no synthetic datasets are publicly available. Only some dataset generation tools as OpenSH  [100]  are available. Today, even if smart sensors become cheaper and cheaper, real houses are not equipped with a wide range of sensors as it can be found in datasets.
It is not realistic to find an opening sensor on a kitchen cabinet. Real homes contains PIR to monitor wide areas with the security system. Temperature sensors to control the heat. More and more air qualitative or luminosity sensors can be found. Some houses are now equipped with smart lights or smart plugs. Magnetic sensors can be found on external openings. And now, some houses provide general electrical and water consumption. These datasets are not representative of the actual home sensor equipment. Another issue as shown above is the annotation. Supervised algorithms needs qualitative labels to learn correct features and classify activities. Residents' self-annotation can produce errors and lack of precision. Post processing to add annotations, adds uncertainty as they are always based on hypothesis, such as every activity is performed sequentially.
But the human activity flow is not always sequential. Very few datasets provide concurrent or interleaved activities. Moreover every dataset proposes its own taxonomy for annotations. Even if synthetic datasets try to overcome annotation issues, This section demonstrates the difficulty of providing a correct evaluation system or dataset. And the work already provided by all the scientific community is excellent. Thanks to this amount of work, it is possible to, in certain conditions, evaluate activity recognition systems. However, there are several areas of research that can be explored to help the field progress more quickly. A first possible research axis for data generation is, the generation of data from video games. Video games constitute a multi-billion dollar industry, where developers put great effort into build highly realistic worlds.
Recent works in the field of semantic video segmentation consider and use video games to generate datasets in order to train algorithms  [104, 105]. Recently Roitberg et al. [106]  studied a first possibility using a commercial game by Electronic Arts (EA) " The Sims 4", a daily life simulator game, to reproduce the video Toyota Smarthome dataset  [107]. The objective was to evaluate and train HAR algorithms from video produced by a video game and compare them to the original dataset. This work showed promising results. An extension of this work could be envisaged in order to generate datasets of sensor activity traces. Moreover, every dataset proposes its own taxonomy. Some are inspired by medical works such as, Katz et al. work  [108] , to define a list of basic and necessary activities.
Evaluation Methods In order to validate the performance of the algorithms, the researchers use datasets. But learning the parameters of a prediction function and testing it on the same data is a methodological error: a model that simply repeats the labels of the samples it has just seen would have a perfect score but could not predict anything useful on data that is still invisible. This situation is called overfitting. To avoid it, it is common practice in a supervised machine learning experiment to retain some of the available data as a dataset for testing. Several methods exist in the field of machine learning and deep learning. For the problem of HAR in smart houses, some of them have been used by researchers. The evaluation of these algorithms is not only related to the use of these methods.
It depends on the methodology but also on the datasets on which the evaluation is based. It is not uncommon that preprocessing is necessary. However this preprocessing can influence the final results. This section highlights some of the biases that can be induced by preprocessing the datasets as well as the application and choice of certain evaluation methods. Unbalanced datasets pose a challenge because most of the machine learning algorithms used for classification have been designed assuming an equal number of examples for each class. This results in models with poor predictive performance, especially for the minority class. This is a problem because, in general, the minority class is larger and the problem is therefore more sensitive to classification errors for the minority class than for the majority class.
To get around this problem some researchers will rebalance the dataset. By removing classes that are too little represented. By randomly removing examples for the most represented classes  [15]  These approaches allow to increase the performance of the algorithms but do not allow to represent the reality. Within the context of the activities of daily life, certain activities are performed more or less often during the course of the days. A more realistic approach is to group activities under a new, more general label. Ex: "preparing breakfast", "preparing lunch", "preparing dinner", "preparing a snack", can be grouped under the label "preparing a meal". Therefore, activities that are less represented but semantically close can be used as parts of example. This can allow fairer comparisons between datasets if the label names are shared.
The Other Class Issue In the field of HAR in smart houses, it is very frequent that a part of the dataset is not labeled. Usually the label "Other" is assigned to these unlabeled events. The class "Other" generally represents 50% of the dataset  [14, 16]. This makes it the most represented class in the dataset and unbalances the dataset. Furthermore, the "Other" class may represent several different activity classes or simply something meaningless. Some researchers choose to suppress this class, judged to be over-represented and containing too many random sequences. Others prefer to remove it from the training phase and therefore from the training set. However, they keep it in the test set in order to evaluate the system in a more real-life environment  [33]. Yala et al. [33]  evaluated performance with and without the "Other" class and showed that this choice has a strong impact on the final results.