# Autonomous Navigation Framework for Intelligent Robots Based on a Semantic Environment Modeling
## 3 TOSM-Based Autonomous Navigation Framework
### Semantic Information Processing
In the testing phase, weights that were generated during model training, are used to recognize objects in a real-world environment. In our SIP module, after object recognition using the CNN model, inference for semantic information is performed at run-time by integrating the recognized objects with their semantic information that contains spatial relationships between objects, described in Table 2. Through these spatial relationships among recognized objects, we infer the relationship information of the robot by leveraging with the semantic objects and space stored in the on-demand database as follows:  
Assume \(x\) is an object recognized in the environment by the CNN model, while \(X\) is the subclass of _EnvironmentalElement_ (e.g., _Furniture_, _Door_, and _Corridor_). Then the class of object \(x\) is defined by a class definition function \(IsA(x,X)\), which indicates that object \(x\) is class \(X\). As shown in Figure 4, there are two objects in the environment: table and chair. The class definition function \(IsA(table,Table)\) and \(IsA(chair,Chair)\) states that in both cases, the class of the table is _Table_ and the chair is _Chair_. When the class is defined, we can get the datatype properties of the object from the database based on the robot's pose and relative pose of the object. In other words, _Chair_ is specified _chair2_ (we call this \(X_{n}\)) that is an instance of _Chair_ by the database query such as illustrated in the gray table inside Figure 4.  
We also use a relationship inference function \(SR=rel(X_{n},X_{m})\) to define relationships between recognized objects, other objects, and places in the database. According to the class definition function, \(X_{n}\) indicates an instance of class \(X\). \(SR\) describes a spatial relationship between \(X_{n}\) and \(X_{m}\), such that \"\(X_{n}\)\(isNextTo/isInsideOf/isInFromOf\)\(X_{m}\)\" (e.g., \"\(table1\)\(isNextTo\)chair2\".) Similarly, we can infer the robot's relationship in the environment; in that case, the \(X_{n}\) is an instance of \(Robot\). So, when the robot navigates in a real-world environment, and it finds two objects, table and chair, shown in Figure 4, the robot recognizes these two objects using the CNN model and infers these objectswith their semantic information that is saved in the database. As a result, we get spatial relationships such as \"_table1 isNextTo chair2_\", \"_table1 isInsideOf corridor2_\", \"_robot isLookingAt chair2_\", and \"_robot isLocatedAt corridor2_\". We can also see that the recognition model and the TOSM-based on-demand database share human-like memory architecture when working together for visual semantic recognition. The SIP module is processed in working memory (Short Term Memory, STM), while prior semantic knowledge stored in the on-demand database is hosted by Long-Term Memory (LTM), as shown in Figure 5. When the mobile robot navigates in a real-world environment, it sends the sensory data to SIP, which passes the pre-processed data to the recognition model that is stored in working memory and gets the periodic visual updates as robot navigates.