# Context Classifier for Service Robots

Tiago Ferreira, Fábio Miranda, Pedro Sousa, José Barata, João Pimentão

► To cite this version:

Tiago Ferreira, Fábio Miranda, Pedro Sousa, José Barata, João Pimentão. Context Classifier for Service Robots. 6th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), Apr 2015, Costa de Caparica, Portugal. pp.196-203, 10.1007/978-3-319-16766-4_21. hal-01343483

HAL Id: hal-01343483

https://inria.hal.science/hal-01343483v1

Submitted on 8 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Context Classifier for Service Robots

Tiago Ferreira¹, Fábio Miranda², Pedro Sousa², José Barata² and João Pimentão²

¹ Holos SA, Caparica, Portugal
tiago.ferreira@holos.pt

² Faculdade de Ciências e Tecnologia – UNL Caparica, Portugal
f.miranda@campus.fct.unl.pt, pas@fct.unl.pt, jab@uninova.pt, pim@fct.unl.pt

**Abstract.** In this paper a context classifier for service robots is presented. Independently of the application, service robots need to have the notion of their context in order to behave appropriately. A context classification architecture that can be integrated in service robots reliability calculation is proposed. Sensorial information is used as input. This information is then fused (using Fuzzy Sets) in order to create a knowledge base that is used as an input to the classifier. The classification technique used is Bayes Networks, as the object of classification is partially observable, stochastic and has a sequential activity. Although the results presented refer to indoor/outdoor classification, the architecture is scalable in order to be used in much wider and detailed context classification. A community of service robots, contributing with their own contextual experience to dynamically improve the classification architecture, can use cloud-based technologies.

**Keywords:** Context, Service Robots, Reliability, Fuzzy Sets, Bayes Networks

# 1 Introduction

Aside from our language and the common understanding of the world and its operation, what contributes to a much more efficient exchange of knowledge between human beings, is the notion of context [1]. Humans are able to add and take into account situational information (context) in a conversation to better understand the subject, retain more information in a more efficient way and to respond appropriately to it, in other words, to increase the conversational bandwidth as said in [1]. When interacting with machines or between machines, that awareness can be lost and so can the information richness [2].

As the matter of environment (as a subset of context) regards the physical world, there is also uncertainty associated with its sensing. That uncertainty can be caused, for example, by inaccuracy or incompleteness of sensed information [3]. There is a need for models that manage and take into account that uncertainty [4]. Fuzzy logic is used to set transitions of an element between membership and non-membership in linguistic terms and it involves ambiguity [5]. It can be used to describe subjective contexts and perform a multi-sensor fusion between those contexts.

Bayesian networks consist in nodes (that represent events) and arcs (representing relationships/conditional dependencies). Nodes not connected represent variables that
are conditionally independent. Bayesian networks are used to process data sets and to deduce high level information [4], assigning the most likely class to a given observation, described by a set of attributes. According to [6] this technique is best suitable for a factored state representation in which a state consists of a vector of attributes.

## 2 Relationship to Cloud-based Solutions

The presented architecture was developed using Robotic Operating System (ROS)[7] on top of the Servrobot platform (servrobot.holos.pt). ROS allows for the exchange of information between software modules within the system using a messaging protocol. These software modules need not to be in the same machine or system and may send messages across the Internet. As the architecture is a software module itself it can receive and send information to other modules or robots. In this sense this architecture is scalable to receive inputs from several service robots. This is a typical application scenario for Cloud-based technologies use thus it can be used by a community of service robots contributing with their own contextual experience to dynamically improve the classification architecture and its performance. Robotearth is an example of cloud-based technologies use in robotics[8].

## 3 State of the Art

Context has been used in the robotic field in many applications using several context models and or techniques [9]. A detailed review was made focusing on the application of context in the field. The most relevant results are presented in Table 1.

<table><caption>**Table 1.** Overview of context models in robotic systems their application purpose</caption><thead><tr><td>Application Purpose</td><td>Context Model/Reasoning Technique</td></tr></thead><tbody><tr><td>Object Recognition</td><td>Markov Random Field (probabilistic classifier)[10], Ontologies[11, 12]</td></tr><tr><td>Safety</td><td>Dempster-Shafer[14], Ontologies[15]</td></tr><tr><td>Human-machine interface</td><td>Polynomial classifiers[16], Hidden Markov Model[17]</td></tr><tr><td>Localization</td><td>Ontologies + Bayesian model[18]</td></tr><tr><td>Navigation</td><td>Ontologies[19]</td></tr><tr><td>Smart Environments</td><td>Ontologies + Logic[20], Ontologies[21]</td></tr></tbody></table>

As it can be understood from Table 1, ontologies seem to be the most used technique to support machine learning in the robotics field. Applications include several areas and reliability seems overlooked.
# 4 Implementation

## 4.1 Problem Approach

As it was stated before there are several applications for context analysis in the robot-ics field but context can be integrated in other applications as well. It is believed that the reliability field hasn't explored context in complete way. Context can and should be integrated in the reliability field allowing reliability calculation optimization.

A classification hierarchy is proposed, in which the context is first classified as in-door environment or outdoor environment and then there will be another computa-tional model created to sub-classify in more detail the type of environment present.

## 4.2 Architecture Overview

<figure><img src="image_2.png" /><figcaption>Fig. 1. Reliability context integration architecture</figcaption></figure>

In Fig.1 the reliability context architecture is presented. It will first receive the sensor inputs and process its signals in order to reduce any noise and outlier values as well as to perform any necessary sensor fusion. In the knowledge base the information from sensors and sensor fusion are stored. A context classifier is created that will provide contextual information to the reliability calculation. Those classifications will be also stored in a context history module. This history can be re-fed to the classification module to improve performance. From that history the last state will be considered for the current classification.

It is believed that the environmental conditions will influence a great deal the reli-ability calculation and estimation. The key elements present in the surroundings may be crucial for optimizing reliability assessment, for example if a mobile robot is work-ing under a very dusty outdoor environment it may be more prone to have mechanical failures thus reducing its reliability.

## 4.3 Architecture Implementation

### Managing Input Sources.

The physical sensors taken into account in the architecture and their fuzzy linguistic terms are:

* Thermometer for temperature ("very hot", "hot", "comfortable" and "cold");

* Hygrometer for humidity ("very high", "high", "comfortable" and "dry");

* Photosensor for light ("very high", "high", "medium", "low" and "very low");
* GPS module for the number of satellites ("high", "medium", "low" and "none");

* IMU for the soil type ("concrete", "pavement" or "smooth");

* GAS detector for gas level ("dangerous" or "non-dangerous" atmosphere);

* Time used to distinguish the "day" and "night" formalisms.

**Fuzzy Logic Membership Functions.**

Fuzzy logic was implemented to transform a sensed numeric value into a linguistic term (a qualitative semantic), therefore fuzzification was used to aggregate the sensor inputs. This will increase the flexibility and modularity of the overall algorithm over the slight changes in the sensor values (also decreasing its complexity and load).

The fuzzification was applied to temperature, humidity and light intensity. For each of them tests were conducted during various times of the day and weather conditions to achieve the necessary linguistic terms and membership functions. In this work the functions were chosen with no specific form, but there are many forms that can be assigned (triangular, trapezoidal, Gaussian, etc.).

**Choosing the Classification Model.**

Since ontologies are often specialized to some domain-specific applications (although they are used for many different purposes), it was preferred not to model the world around the robot "statically" by describing its relations "manually" in the beginning. Instead, the main interest is in one technique that could be taught and that can adapt to new situations.

The environment is partially observable because the robots sensors cannot give access to all the complete state of the environment at each point in time. Since its being dealt with a stochastic environment, the classification task is a sequential activity, a single classification at a time considering the previous state is performed, there is a dynamic environment and can be considered discrete, a factored representation is present the technique should support probabilities and uncertainty. Therefore the most suitable technique for classification is a Bayes classifier (Bayes network). It seemed to be a good start since it has good performances, even with less training data, is highly scalable, can adapt to a wide variety of classification tasks and is appropriate for a factored representation of data.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Bayes Network</figcaption></figure>
In this work the WEKA [22] framework was used to generate the Bayes network for this classification. In Fig.2 the Bayes network is represented with the different attributes and relations to the proposed output classification.

**Reliability.**

To integrate this work in reliability calculation a context factor must be added in the calculation.

$$R = e^{-\lambda \cdot \pi \cdot t} \quad (1)$$

Where *R* is the reliability, *λ* is the module hazard rate, *π* is the terrain factor and *t* is the time. The terrain factor can be calculated by:

$$\pi = \sum \pi_{tx} \times t_{tx} \quad (2)$$

Where $\pi_{tx}$ is the terrain factor asserted to terrain *x* and $t_{tx}$ is the total percentage of the operating time in terrain *x*. The proposed architecture results can be integrated in this factor by substituting the terrain concept by the context concept.

**5 Field Tests and Experiment Results**

From the first dataset collected, an 87.5% of correctly classified instances were obtained, using a 10-fold cross-validation evaluation on the training dataset. The confusion matrix represents TP (true positives), FP (false positives), FN (false negatives) and TN (true negatives). From Table 2 it can be understood that 119 instances were correctly classified as “indoor” out of a total of 121 indoor instances (true positives). A more detailed accuracy table is presented below (Table 3).

<table><caption>**Table 2. Confusion Matrix**</caption><thead><tr><td>Indoor</td><td>Outdoor</td><td>Classified as</td></tr></thead><tbody><tr><td>119</td><td>2</td><td>Indoor</td></tr><tr><td>17</td><td>14</td><td>Outdoor</td></tr></tbody></table>

<table><caption>**Table 3. Detailed performance measures**</caption><thead><tr><td>TP Rate</td><td>FP Rate</td><td>Precision</td><td>Specificity</td><td>F-Measure</td><td>Class</td></tr></thead><tbody><tr><td>0.983</td><td>0.548</td><td>0.875</td><td>0.452</td><td>0.926</td><td>INDOOR</td></tr></tbody></table>

TPR (True Positive Rate), also known as sensitivity or recall measures the proportion of positives that are correctly classified as such. FPR (False Positive Rate), also known as false alarm rate or fallout it's the complementary with the specificity (1-specificity). Precision represents the positive predictive values. Specificity (also known as TNR – True Negative Rate) measures the proportion of negatives, which are correctly identified as such (complementary of FPR). F-measure a combined measure for precision and recall.
It was found that the number of GPS satellites is the most relevant attribute, alt- hough all other attributes are needed to increase systems consistency and accuracy.

A mixed environment test including the “previous state” attribute was done. In Fig.3, a mixed testing path is presented. The indoor path is represented in blue color, the outdoor path is represented in red color and the transition areas are represented in orange. In Table 4, the obtained confusion matrix is presented.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Holos mixed testing path</figcaption></figure>

<table><caption>**Table 4.** Confusion Matrix from the mixed testing path</caption><thead><tr><td>Indoor</td><td>Outdoor</td><td>Classified as</td></tr></thead><tbody><tr><td>119</td><td>19</td><td>Indoor</td></tr><tr><td>2</td><td>146</td><td>Outdoor</td></tr></tbody></table>

In this test the percentage of correctly classified instances was increased to 92.66% of overall accuracy. From the incorrectly classified instances, the 2 FP cases are located on step 6 of the Fig.3 and they occurred due to the presence of a glass wall in the room. Near the glass wall, the classifier can be deceived. Regarding the 19 false negative cases, 2 of them occurred on the step 5 of the path (Fig.3) which is the transition step, 6 of them occurred in step 18 due to the presence of a porch and the fact that the GPS attribute is slower to acquire satellites when moving from indoor to outdoor environments. The majority of the FN cases (11) occurred in step 17, which is another transition step.
<table><caption>**Table 5.** Detailed performance measures</caption><thead><tr><td>TP Rate</td><td>FP Rate</td><td>Precision</td><td>Specificity</td><td>F-Measure</td><td>Class</td></tr></thead><tbody><tr><td>0.862</td><td>0.014</td><td>0.885</td><td>0.986</td><td>0.873</td><td>INDOOR</td></tr></tbody></table>

It's now possible to remake all the calculations for the real application scenario. In Table 5, those calculations are presented. Very good results were achieved in specificity and sensitivity (a perfect classifier should be 100% specific and 100% sensitive) as well as a decrease in the FPR. Considering the “previous state” attribute in the classification, the most relevant attributes where: “light”, “GPS satellites”, “previous state” and “temperature web”.

As can be seen by these results this architecture is able to achieve similar results to those in [23] even without the previous state optimization (around 88%). When the previous state is included the results improve by approximately 5% in comparison.

## 6 Conclusions and Future Work

There are several context models, techniques and frameworks. From all of those possibilities the ones based on ontologies seem to be the ones with most widespread use. From all existing frameworks and for the intent of this architecture, machine-learning techniques, particularly Bayesian networks, seemed to be the more appropriate method for solving the systems classification problem. This is verified by the good results achieved by this system. One big reason for the good results in the system was the inclusion, in the architecture, of the previous state in the Bayes network. Future experiments should be considered with more linguistic terms in the fuzzy sets (for example, “morning”, “afternoon”, “late afternoon” or “night” instead of just “day” or “night” for the time). Increasing the attribute quantity can add a valuable improvement in the overall performance. However, increasing the number of attributes doesn't necessarily improve the systems accuracy.

It is believed that a good base was created for a more complex and detailed classification. As knowing if the robot is indoor or outdoor with good accuracy allows a detailed context sub-classification.

## References

1. Dey, A.K., Abowd, G.D.: Towards a Better Understanding of Context and Context-Awareness. Handheld Ubiquitous Comput. 304–307 (1999).

2. Dey, A.K.: Understanding and Using Context. Pers. Ubiquitous Comput. 5, 4–7 (2001).

3. Haghighi, P., Krishnaswamy, S., Zaslavsky, A., Gaber, M.M.: Reasoning about context in uncertain pervasive computing environments. Smart Sens. Context. 5279, 112–125 (2008).

4. Bettini, C., Brdiczka, O., Henricksen, K., Indulska, J., Nicklas, D., Ranganathan, A., Riboni, D.: A survey of context modelling and reasoning techniques. Pervasive Mob. Comput. 6, 161–180 (2010).

5. Lotfi Zadeh: Fuzzy Sets. Inf. Control. 8, 338–353 (1965).

6. Russell, S., Norvig, P., Canny, J., Malik, J., Edwards, D.: Artificial intelligence: a modern approach. (2009).
7. Quigley, M., Conley, K.: ROS: an open-source Robot Operating System. ... open source .... (2009).

8. Kehoe, B., Sachin, P.: A Survey of Research on Cloud Robotics and Automation. IEEE Trans. Autom. Sci. Eng. 1–11 (2010).

9. Miranda, F., Ferreira, T.: Review on Context Classification in Robotics. Rough Sets Intell. .... (2014).

10.Varvadoukas, T., Giannakidou, E., Gomez, J. V., Mavridis, N.: Indoor Furniture and Room Recognition for a Robot Using Internet-Derived Models and Object Context. Front. Inf. Technol. 122–128 (2012).

11.Wang, E., Kim, Y.S., Kim, H.S., Son, J.H., Lee, S., Suh, I.H.: Ontology Modeling and Storage System for Robot Context Understanding. Knowledge-Based Intell. Inf. Eng. Syst. 3683, 922–929 (2005).

12.Choi, J., Park, Y., Lim, G., Lee, S.: Ontology-Based Semantic Context Modeling for Object Recognition of Intelligent Mobile Robots. Recent Prog. Robot. Viable Robot. Serv. to Hum. 370, 399–408 (2008).

13.Wibisono, W., Zaslavsky, A., Ling, S.: Improving situation awareness for intelligent on-board vehicle management system using context middleware. IEEE Intell. Veh. Symp. 1109–1114 (2009).

14.Nienhiiser, D., Gumpp, T., Zollner, J.M.: A Situation Context Aware Dempster-Shafer Fusion of Digital Maps and a Road Sign Recognition System. Intell. Veh. Symp. 2009 IEEE. 1401–1406 (2009).

15.Schlenoff, C., Messina, E.: A robot ontology for urban search and rescue. Proc. 2005 ACM Work. Res. Knowl. Represent. Auton. Syst. - KRAS '05. 27–34 (2005).

16.Trovato, G., Zecca, M., Kishi, T., Endo, N., Hashimoto, K., Takanishi, A.: Generation of Humanoid Robot’s Facial Expressions for Context-Aware Communication. Int. J. Humanoid Robot. 10, 23 (2013).

17.Mou, W., Chang, M., Liao, C.: Context-aware assisted interactive robotic walker for Parkinson’s disease patients. Intell. Robot. Syst. 329 – 334 (2012).

18.Yi, C., Suh, I.H., Lim, G.H., Choi, B.-U.: Bayesian robot localization using spatial object contexts. 2009 IEEE/RSJ Int. Conf. Intell. Robot. Syst. 3467–3473 (2009).

19.Provine, R., Uschold, M., Smith, S., Stephen, B., Schlenoff, C.: Observations on the use of ontologies for autonomous vehicle navigation planning. Rob. Auton. Syst. (2004).

20.Mastrogiovanni, F., Sgorbissa, a., Zaccaria, R.: Context assessment strategies for Ubiquitous Robots. 2009 IEEE Int. Conf. Robot. Autom. 2717–2722 (2009).

21.Hristoskova, A., E., C., Veloso, M., De, F.: Heterogeneous Context-Aware Robots Providing a Personalized Building Tour. Int. J. Adv. Robot. Syst. 10, 13 (2013).

22.Bouckaert, R.R., Frank, E., Hall, M., Kirkby, R., Reutemann, P., Seewald, A., Scuse, D.: WEKA Manual for Version 3-7-11. (2014).

23.Zhou, P., Zheng, Y., Li, Z., Li, M., Shen, G.: IODetector: A generic service for indoor outdoor detection. ... Embed. Netw. Sens. .... (2012).