Patent Description:
The following discussion of the background is intended to facilitate an understanding of the present invention only. It may be appreciated that the discussion is not an acknowledgement or admission that any of the material referred to was published, known or part of the common general knowledge of the person skilled in the art in any jurisdiction as at the priority date of the present invention.

Conventionally, technologies for maintenance on geographical information such as a map have been focused on autonomous vehicles which may require lane level maps. The lane level maps generally comprise a list of line segments which may define location of scene elements like lane markers, kerbs, and drivable routes through intersections.

For indoor robotics, most work has been focused on occupancy grid maps. These may generally be easy to build and maintain given that a laser scanner is available on the robot. The grid basically represents which cells contain obstacles. However, small mobile robots may require high resolution semantic grid maps which have many more degrees of freedom and are harder to maintain incrementally.

Cells may contain a wide range of categories relevant for navigation, such as footpath, road, kerb, grass, traffic light crossing etc. Errors tend to cause more problems in the navigation of the robot compared to occupancy grid maps (for example, when used indoors).

Generally, in robotics applications, the semantic grid maps are generally built using the following process:.

Therefore, for maintenance on the semantic grid maps, manual operations to automatically generated maps may be required to correct or update flaws in the map due to issues like sensor errors, misalignment, or fundamental ambiguities.

Document <CIT> discloses an autonomous control system configured to generate synthetic data that reflect simulated environments, and to use the generated synthetic data to train computer models for various detection and control algorithms.

As such, there exists a need to develop technologies which specifically train a neural network to learn how to build large scale consistent maps or update the semantic grid maps over time, in order to reduce the reliance on the mapping expert to constantly correct updated maps.

The present invention seeks to provide a system and a method that addresses the aforementioned need at least in part.

The technical solution is provided in the form of a system as defined in claim <NUM>, and a method as defined in claim <NUM>.

In some embodiments, each training step may span multiple iterations, and the output of the neural network at earlier iterations may become the second data in subsequent iterations.

Since the neural network can construct the predicted geographical information and compare the actual geographical information and the predicted geographical information, the present invention can train the neural network to learn how to build large scale consistent maps or update the semantic grid maps over time, without manual operations. Accordingly, the present invention is able to reduce costs and efforts required to maintain the semantic grid maps.

In accordance with an aspect of the present invention, there is a system for training a neural network for geographical information comprising: the neural network; and a processor operable to generate an environment from actual geographical information, and to simulate a robot moving in the environment to obtain first data from the simulation of the robot, characterised in that: the processor is operable to produce corrupted geographical information from the actual geographical information to obtain second data from the corrupted geographical information, and the neural network is operable to receive the first and the second data, construct predicted geographical information based on the first and second data, and compare the actual geographical information and the predicted geographical information to train the neural network.

In some embodiments, the generated environment includes a 3D (three-dimensional) environment.

In some embodiments, the processor is operable to generate the 3D environment from a top-down view of the actual geographical information.

In some embodiments, the processor is operable to delete at least a part of the actual geographical information, add a spatial distortion on the actual geographical information, or add incorrect categories or noise on the actual geographical information, to produce the corrupted geographical information.

According to the invention, the neural network is operable to predict at least one region in the actual geographical information that needs to be updated.

According to the invention, the neural network is operable to construct the predicted geographical information based on information relevant to the at least one region that needs to be updated.

In some embodiments, the neural network is operable to correct or update at least one flaw in the actual geographical information by the training.

In some embodiments, the first data includes simulated sensory data from the robot's point of view.

In some embodiments, the actual geographical information includes an actual map, the corrupted geographical information includes a corrupted map, and the predicted geographical information includes a predicted map.

In some embodiments, each training may span multiple iterations. The output of the neural network at earlier iterations may become the second data in subsequent iterations.

In accordance with another aspect of the present invention, there is a method of training a neural network for geographical information comprising steps of: generating an environment from actual geographical information; simulating a robot moving in the environment to obtain first data from the simulation of the robot; producing corrupted geographical information from the actual geographical information to obtain second data from the corrupted geographical information; receiving, by the neural network, the first and the second data; constructing predicted geographical information based on the first and second data; and comparing the actual geographical information and the predicted geographical information to train the neural network.

In some embodiments, the step of generating an environment from actual geographical information comprising a step of: generating a 3D environment from a top-down view of the actual geographical information.

According to the invention, the step of constructing predicted geographical information based on the first and second data comprising a step of: predicting at least one region in the actual geographical information that needs to be updated.

According to the invention, the step of constructing predicted geographical information based on the first and second data comprising a step of: constructing the predicted geographical information based on information relevant to the at least one region that needs to be updated.

In some embodiments, the method further comprises a step of: correcting or updating at least one flaw in the actual geographical information by the training.

Other aspects of the invention will become apparent to those of ordinary skilled in the art upon review of the following description of specific embodiments of the present invention in conjunction with the accompanying drawings.

Other arrangements of the present invention are possible and, consequently, the accompanying drawings are not to be understood as superseding the generality of the preceding description of the invention.

<FIG> is a block diagram in accordance with an embodiment of the present invention.

As shown in <FIG>, a system <NUM> comprises a processor <NUM> and a neural network <NUM>.

In some embodiments, the processor <NUM> may be a controller or a control unit of a device. In some other embodiments, the processor <NUM> may be a stand-alone device such as a computing device. The computing device may include, but not be limited to, a smartphone, a tablet computer, a laptop computer, a desktop computer and a wearable device. The wearable device may include, but not be limited to, a smart watch, a smart glasses or a mobile virtual reality headset. Although not shown, in some other embodiments, the processor <NUM> may be a part of or an element of the neural network <NUM>.

In some embodiments, it may be appreciated that the processor <NUM> may include a plurality of controllers or a plurality of computing devices. In some embodiments, the plurality of computing devices <NUM> may communicate with each other.

The neural network <NUM> may be a computer system modelled on a human brain and/or a nervous system. The processor <NUM> and the neural network <NUM> are operable to communicate with each other.

The processor <NUM> may generate an environment from actual geographical information, for example an actual map. The actual map is also referred to as a global (i.e. of an arbitrarily large area) semantic (i.e. with classification of different obstacle/ground surface types) map. In some embodiments, the "actual map" is referred to as either a template map (which is manually specified via a step of "manually specifying a template semantic map - to be described below) or an actual map for real environments (which is not obtainable - the user can only make predictions for this via the output of the neural network when executed on real data).

The generated environment includes a 3D (three-dimensional) environment. In some embodiments, the processor <NUM> may generate the 3D environment from a top-down view of the actual geographical information, for example the actual map.

The processor <NUM> may simulate a robot moving in the environment, to obtain data (hereinafter referred to as "first data") from the simulation of the robot. In some embodiments, the first data includes simulated sensory data from the robot's point of view.

Specifically, by simulating the robot, the processor <NUM> may generate sensory data for which a user knows exact parameters of the underlying actual map that was used to construct the simulation. By generating and utilising such sensory data, the present invention can train the neural network <NUM> which can solve an inverse problem (i.e. instead of a problem of transforming the semantic grid to the sensory data, performs a relatively harder problem of transforming the sensory data to the semantic grid).

As described above, the first data may be obtained from the simulation of the robot. In some embodiments, the processor <NUM> may obtain the first data and provide the neural network <NUM> with the first data, and thus the neural network <NUM> may receive the first data from the processor <NUM>. In some other embodiments, the neural network <NUM> may obtain the first data directly.

The processor <NUM> may produce corrupted geographical information, for example a corrupted map, from the actual geographical information, for example the actual map, to obtain data (hereinafter referred to as "second data") from the corrupted geographical information.

In some embodiments, the processor <NUM> may delete at least a part of the actual geographical information, for example the actual map, in order to produce the corrupted geographical information, for example the corrupted map. In some other embodiments, the processor <NUM> may add a spatial distortion on the actual geographical information, for example the actual map, in order to produce the corrupted geographical information, for example the corrupted map. In some other embodiments, the processor <NUM> may add incorrect categories or noise on the actual geographical information, for example the actual map, in order to produce the corrupted geographical information, for example the corrupted map.

Specifically, the second data may be obtained from the corrupted geographical information, for example the corrupted map. In some embodiments, the processor <NUM> may obtain the second data and provide the neural network <NUM> with the second data, and thus the neural network <NUM> may receive the second data from the processor <NUM>. In some other embodiments, the neural network <NUM> may obtain the second data directly.

The neural network <NUM> may construct predicted geographical information, for example a predicted map, based on the first data and second data.

In some embodiments, the neural network <NUM> may predict at least one region in the actual geographical information, for example the actual map, that needs to be updated. The neural network <NUM> may then construct the predicted geographical information, for example the predicted map, based on information relevant to the at least one region that needs to be updated.

The neural network <NUM> may then compare the actual geographical information, for example the actual map, and the predicted geographical information, for example the predicted map, so as to train the neural network <NUM>. In some embodiments, the neural network <NUM> may correct or update at least one flaw in the actual geographical information, for example the actual map, by the training of the neural network <NUM>.

In some embodiments, the actual geographical information, for example the actual map, is a mutable buffer that may get incrementally updated over time. For example, if a vehicle trajectory covers the same area twice, the neural network <NUM> may update the buffer twice. It may be appreciated that the mutable buffer is not necessarily needed during the training of the neural network <NUM>.

The present invention may comprise the three (<NUM>) phases as follows. <FIG> is a conceptual diagram showing phase <NUM> in accordance with an embodiment of the present invention.

<FIG> is a conceptual diagram showing a conventional system architecture for neural network <NUM> based on grid mapping (also referred to as "local mapper"). <FIG> is a conceptual diagram showing a system architecture in which a global map is added, in accordance with an embodiment of the present invention (also referred to as "local/global data fusion mapper").

<FIG> and <FIG> elucidate the difference between the commonly used system architecture for the neural network <NUM> based on the grid mapping (shown in <FIG>), and the system architecture in accordance with an embodiment of the present invention (shown in <FIG>).

As shown in <FIG>, the method in accordance with an embodiment of the present invention may make use of a global semantic map buffer, which allows the neural network <NUM> to perform information fusion between the sensor data and the prior map. In addition, the method in accordance with an embodiment of the present invention may make use of a novel training mechanism which specifies a method of training based on an artificially corrupted global map, and a method to unwrap the training process over multiple iterations of the map update mechanism. As such, the neural network <NUM> can better learn to perform incremental updates to the map.

<FIG> is a flowchart in accordance with an embodiment of the present invention.

First, the processor <NUM> may generate the environment, for example the 3D environment, from the actual geographical information (S110). The processor <NUM> may generate the 3D environment from the top-down view of the actual geographical information.

The processor <NUM> may then simulate the robot moving in the environment, for example the 3D environment (S120). Thus, the processor <NUM> may generate the first data, for example the sensory data from the robot's point of view. The neural network <NUM> may receive the first data.

The processor <NUM> may produce corrupted geographical information from the actual geographical information (S130). Thus, the processor <NUM> may generate the second data from the corrupted geographical information. The neural network <NUM> may receive the second data.

The neural network <NUM> which has received the first data and the second data may construct the predicted geographical information based on the first data and the second data (S140). The neural network <NUM> may predict at least one region in the actual geographical information that needs to be updated, and then construct the predicted geographical information based on information relevant to the at least one region that needs to be updated.

The neural network <NUM> may then compare the actual geographical information and the predicted geographical information so as to train the neural network <NUM> (S150). The neural network <NUM> may correct or update at least one flaw in the actual geographical information, by the training.

In accordance with the present invention, by training the neural network <NUM> for the exact task of determining corrective updates to apply to the actual grid map, minor flaws that may otherwise be introduced by traditional sensor fusion methods can be automatically mitigated.

It may be appreciated that the present invention can also be used for creation of detailed maps for purposes other than robot navigation (for example, constructing and/or maintaining precise maps of construction sites, agricultural fields, search and rescue sites, etc.).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the present invention as defined by the appended claims. However this is merely an exemplarily embodiment, and those skilled in the art will recognize that various modifications and equivalents are possible in light of the above embodiments.

Claim 1:
A system for training a neural network (<NUM>) for geographical information the neural network (<NUM>); and a processor (<NUM>) operable to generate an environment from actual geographical information, and to simulate a robot moving in the environment to obtain first data from the simulation of the robot, characterised in that:
the processor is operable to produce corrupted geographical information from the actual geographical information to obtain second data from the corrupted geographical information, and
the neural network is operable to receive the first and the second data, construct predicted geographical information based on the first and second data, and compare the actual geographical information and the predicted geographical information to train the neural network, wherein the neural network is operable to predict at least one region in the actual geographical information that needs to be updated and to construct the predicted geographical information based on information relevant to the at least one region that needs to be updated.