Patent ID: 12253844

MODE FOR CARRYING OUT THE INVENTION

<Outline of Present Technology>

The present technology focuses on a point that it is difficult to perform sensing using an optimal algorithm in a sensor device mounted on various devices such as a robot, a moving body, and a smartphone due to the following factors, and achieves a solution thereof.

Factors

There are many types of elemental technologies.Maturity of elemental technologies vary.There are many variations in cost.There are many variations in applications.System design and implementation are challenging.There are many types of user requests.There are many mounting restrictions such as processor power, power consumption, and circuit size.There are many types of sensing targets.

Especially, the present technology enables a sensor device that performs environment sensing to perform the sensing using an optimal algorithm as a sensing algorithm that is an algorithm regarding the sensing.

Hereinafter, a mode for carrying out the present technology is described. The description is given in the following order.1. Program Providing System2. Use Case of Sensing Program3. Configuration of Conveyance Robot4. Operation of Conveyance Robot5. Variation
<Program Providing System>
System Configuration

FIG.1is a view illustrating a configuration example of a program providing system according to one embodiment of the present technology.

The program providing system inFIG.1is formed by connecting various devices such as a mobile terminal2-1, an arm robot2-2, a moving body2-3, a cooking robot2-4, and a conveyance robot2-5to a program management server1via a network11including the Internet and the like.

The mobile terminal2-1is a smartphone.

The arm robot2-2is a dual-arm robot. A carriage is provided in a casing of the arm robot2-2. The arm robot2-2is a movable robot.

The moving body2-3is an automobile. A function of automated driving and the like is mounted on the moving body2-3.

The cooking robot2-4is a kitchen-type robot. The cooking robot2-4has a function of cooking by driving a plurality of cooking arms. An operation similar to a cooking operation performed by a human is reproduced by the cooking arm.

The conveyance robot2-5is a robot on a top plate of which prepared as a placing table a conveyance object may be placed, the robot capable of moving to a destination position in this state. A wheel is provided on a base of the conveyance robot2-5.

Each device illustrated inFIG.1is equipped with a sensor device used for sensing of environment, sensing of object, sensing of human and the like.

FIG.2is a view illustrating a configuration example of the sensor device.

As illustrated inFIG.2, a sensor device21is provided with a controller31and a sensor group32.

The controller31controls each sensor forming the sensor group32to perform sensing of various targets such as the sensing of environment, sensing of object, and sensing of human. The sensing by the controller31is performed on the basis of sensor data output from each sensor forming the sensor group32.

The controller31outputs a sensing result to a device on a host side. Various types of processing are performed by the device on the host side on the basis of the sensing result by the controller31. In a case where the sensor device21is mounted on the mobile terminal2-1, a central processing unit (CPU) of the smartphone serves as the device on the host side. The controller31also has a function of communicating with the device on the host side.

The sensor group32includes a plurality of sensors that performs the sensing of various targets. In the example inFIG.2, the sensor group32includes an RGB camera32A, a stereo camera32B, a ToF sensor32C, a structured-light sensor32D, and a LiDAR32E.

The RGB camera32A includes an image sensor for an RGB image. The RGB camera32A images a peripheral state by driving the image sensor, and outputs the RGB image acquired by the imaging as the sensor data.

The stereo camera32B is a distance sensor of a stereo camera system, and includes two image sensors for a distance image. The stereo camera32B outputs the distance image indicating a distance to a target as the sensor data.

The ToF sensor32C is a distance sensor of a time of flight (ToF) system. The ToF sensor32C measures the distance to the target by the ToF system, and outputs distance information as the sensor data.

The structured-light sensor32D is a distance sensor of a structured-light system. The structured-light sensor32D measures the distance to the target by the structured-light system, and outputs distance information as the sensor data.

A light detection and ranging (LiDAR)32E measures a three-dimensional position of each point of the target, and outputs information indicating a measurement result as the sensor data.

Sensors different from the sensors illustrated inFIG.2such as a positioning sensor, a gyro sensor, an acceleration sensor, a temperature sensor, and an illuminance sensor may be included in the sensor group32.

A type of the sensor forming the sensor group32is appropriately changed depending on the device on which the sensor device21is mounted. One sensor may form the sensor group32.

The sensor device21may include a substrate on which the controller31and the sensor group32are arranged, or may be formed as a device in which the substrate on which each sensor is arranged is accommodated in a casing21A as illustrated inFIG.3.

In the sensor device21having such a configuration, the controller31executes a sensing program, which is a program for sensing, and implements a sensing function of various targets such as the environment, object, and human. The sensing function of the controller31is implemented on the basis of an output of one sensor forming the sensor group32or on the basis of a combination of outputs of a plurality of sensors.

The environment sensing (sensing of environment) includes, for example, the following.Imaging of RGB image using RGB camera32AMeasurement of distance to target using outputs of stereo camera32B, ToF sensor32C, and structured-light sensor32DGeneration of three-dimensional map using output of LiDAR32EEstimation of self-position using three-dimensional map

The environment sensed by the sensor device21includes various physical states that are states outside the sensor device21or outside the device on which the sensor device21is mounted that may be expressed as quantitative data by performing the sensing.

The object sensing (sensing of object) includes, for example, the following.Recognition and identification of target using RGB image imaged by RGB camera32AMeasurement of characteristic of target such as shape, size, color, and temperature

The object sensed by the sensor device21includes various stationary objects and moving objects around the sensor device21or around the device on which the sensor device21is mounted.

The human sensing (sensing of human) includes, for example, the following.Recognition of human, recognition of face of human, identification of human using RGB image imaged by RGB camera32ARecognition of specific parts of person such as head, arms, hands, eyes, and noseEstimation of position of specific parts including bone estimationEstimation of physical feature of human such as body height and body weightEstimation of attribute of human such as age and sex

The human sensed by the sensor device21includes a human around the sensor device21or around the device on which the sensor device21is mounted.

The controller31includes a plurality of programs with different algorithms as the sensing programs for implementing the respective sensing functions.

FIG.4is a view illustrating an example of the sensing program prepared in the sensor device21.

In the example inFIG.4, a ranging program A, a ranging program B, and a ranging program C are prepared as firmware that operates on an operating system (OS). The ranging program A, the ranging program B, and the ranging program C are the sensing programs that implement a ranging function as an environment sensing function.

The ranging program A, the ranging program B, and the ranging program C are the sensing programs that implement the same ranging function by different sensing algorithms. The ranging program A, the ranging program B, and the ranging program C define the different sensing algorithms.

The ranging program A is the sensing program that performs ranging by a ranging algorithm A. The ranging program B is the sensing program that performs ranging by a ranging algorithm B. The ranging program C is the sensing program that performs ranging by a ranging algorithm C.

For example, the ranging algorithms A to C are the sensing algorithms to perform the ranging using different parameters: different parameters are set in the same sensor and the distance is calculated by performing the same calculation on the basis of the output of the sensor.

Furthermore, the ranging algorithms A to C are the sensing algorithms to perform the ranging using different calculation methods: the same parameter is set in the same sensor and the distance is calculated by performing different calculations on the basis of the output of the sensor.

In a case where a plurality of distance sensors such as the stereo camera32B, the ToF sensor32C, and the structured-light sensor32D is prepared, the ranging algorithms A to C may be the sensing algorithms to perform ranging using different distance sensors.

In this case, for example, the ranging algorithm A performs the ranging on the basis of the output of the stereo camera32B, and the ranging algorithm B performs the ranging on the basis of the output of the ToF sensor32C. Furthermore, the ranging algorithm C performs the ranging on the basis of the output of the structured-light sensor32D.

In this manner, in the sensor device21, a plurality of programs with different sensing algorithms is prepared as the sensing programs for implementing the same ranging function. The sensor used for the ranging is associated with at least any one of each sensing algorithm or a sensing program that defines each sensing algorithm. In a case where the sensing program is executed, an operation of the associated sensor is controlled in conjunction with this.

For example, in the sensor device21mounted on the conveyance robot2-5, the sensing algorithm corresponding to a sensing condition is selected and the ranging is performed. The sensing condition is a condition of selection of the sensing algorithm determined according to a situation of the conveyance robot2-5.

For example, in a case where the situation is suitable for the ranging algorithm A, the ranging program A is executed, and the ranging is performed using the ranging algorithm A. Furthermore, in a case where the situation is suitable for the ranging algorithm B, the ranging program B is executed, and the ranging is performed using the ranging algorithm B. In a case where the situation is suitable for the ranging algorithm C, the ranging program C is executed, and the ranging is performed using the ranging algorithm C.

Since the ranging is performed by adaptively selecting the sensing algorithm (sensing program) corresponding to the sensing condition, the ranging by an optimal sensing algorithm becomes possible. The same applies to a case where the sensing target is other than the distance.

One sensing program defines one sensing algorithm. Selecting the sensing program corresponds to selecting the sensing algorithm.

Note that, adaptively selecting the sensing algorithm means selecting the sensing algorithm associated with the sensing condition when this condition is detected. The sensing algorithm considered to be suitable is associated with the sensing condition corresponding to each assumed situation. The association between the sensing condition and the sensing algorithm may be dynamically changed.

FIG.5is a view illustrating another example of the sensing program.

In the example inFIG.5, an ingredient recognition program A, an ingredient recognition program B, and an ingredient recognition program C are prepared as the firmware that operates on the OS. The ingredient recognition programs A to C are the sensing programs that implement an ingredient recognition function as the object sensing function.

For example, in the sensor device21mounted on the cooking robot2-4, the sensing algorithm corresponding to the sensing condition determined by a cooking process and the like is selected and the ingredient is recognized.

FIG.6is a view illustrating still another example of the sensing program.

In the example inFIG.6, a face recognition program A, a face recognition program B, and a face recognition program C are prepared as the firmware that operates on the OS. The face recognition programs A to C are the sensing programs that implement a face recognition function as the human sensing function.

For example, in the sensor device21mounted on the cooking robot2-4, the sensing algorithm corresponding to the sensing condition determined by the cooking process and the like is selected and the face is recognized.

Updating of Sensing Program

In the program providing system inFIG.1, the sensing program prepared as the firmware in the sensor device21of each device may be updated.

FIG.7is a view illustrating an example of updating of the sensing program.

As indicated by an arrow inFIG.7, the program management server1provides the sensing program to each device. The program management server1includes a data base (DB) of the sensing program to be provided to each device.

In the example inFIG.7, a ranging program D that performs the ranging by a ranging algorithm D is provided to the mobile terminal2-1, and a face recognition program H that performs the face recognition by a face recognition algorithm H is provided to the arm robot2-2.

Furthermore, a self-position estimation program J that performs self-position estimation by a self-position estimation algorithm J is provided to the moving body2-3, and an object recognition program K that performs object recognition by an object recognition algorithm K is provided to the cooking robot2-4. A person recognition program M that performs person recognition by a person recognition algorithm M is provided to the conveyance robot2-5.

FIG.8is a view illustrating an example of updating of the sensing program.

In the sensor device21of each device, the sensing program may be added as illustrated in A ofFIG.8. In the example in A ofFIG.8, the ranging program D that performs the ranging by the ranging algorithm D is added to the ranging programs A to C that perform the ranging by the ranging algorithms A to C, respectively.

In a default state, the sensing program that defines the sensing algorithm corresponding to a general situation is prepared in the sensor device21of each device. Even in a case of a situation to which the sensor device21of each device cannot respond with the sensing program prepared in advance, this may respond to a special situation by adding the sensing program that defines the sensing algorithm corresponding to such special situation.

Furthermore, as illustrated in B ofFIG.8, it is also possible to delete (uninstall) an unnecessary program. In the example in B ofFIG.8, the ranging program C out of the ranging programs A to C is deleted as indicated by a broken line frame.

FIG.9is a view illustrating another example of updating of the sensing program.

As illustrated inFIG.9, the updating may be performed in units of sensing program set including a plurality of sensing programs. In the example inFIG.9, a sensing program set including the ranging program D that performs the ranging by the ranging algorithm D, a ranging program E that performs the ranging by a ranging algorithm E, and a ranging program F that performs the ranging by a ranging algorithm F is provided by the program management server1to be added.

In the DB of the program management server1, as illustrated inFIG.10, a plurality of sensing program sets in which a plurality of sensing programs is collected for each use condition such as a place, a situation, and a purpose is prepared.

In the example inFIG.10, a sensing program set for indoor ranging and a sensing program set for outdoor ranging are prepared. These sensing program sets are the sensing program sets corresponding to places.

The sensing program set corresponding to the place is, for example, the set used in the sensor device21mounted on a device having a moving function. Among the same indoors, the sensing program sets may be prepared in units of finer places such as a sensing program set for a kitchen and a sensing program set for a dining room.

It is also possible to prepare the sensing program sets for various places such as a sensing program set for the sea, a sensing program set for a mountain, and a sensing program set for inside of a train.

Furthermore, in the example inFIG.10, a sensing program set for ranging in fine weather and a sensing program set for ranging in rainy weather are prepared. These sensing program sets are the sensing program sets corresponding to weather.

The sensing program set corresponding to weather is, for example, the set used in the sensor device21mounted on a device having a moving function and may move outdoors. It is also possible to prepare the sensing program sets for various changing situations such as a sensing program set for each time of the day such as morning, noon, and night, a sensing program set for each brightness, and a sensing program set for each temperature.

It is also possible to prepare the sensing program sets for various purposes such as a sensing program set when running, a sensing program set when playing baseball, a sensing program set when cooking curry, and a sensing program set when cooking salad.

The sensor device21of each device may collectively add the sensing programs by specifying an ID of the sensing program set corresponding to a use condition. The ID as identification data is set in each sensing program set. The ID as the identification data is also set in each sensing program forming the sensing program set.

In place of the set of the sensing programs that implement the same ranging function by different sensing algorithms, the set of the sensing programs that implements different functions may be added as illustrated inFIG.11.

In the example inFIG.11, the sensing program set includes the ranging program D, the face recognition program H, and the object recognition program K. The ranging program D is the sensing program that performs the ranging by the ranging algorithm D, and the face recognition program H is the sensing program that performs the face recognition by the face recognition algorithm H. The object recognition program K is the sensing program that performs the object recognition by the object recognition algorithm K.

FIG.12is a view illustrating an example of the sensing program set.

The sensing program set illustrated inFIG.12includes an algorithm manager that is a program that controls adaptive selection of the algorithm.

The sensor device21executes the algorithm manager and selects the sensing algorithm corresponding to the sensing condition. In the algorithm manager, a combination of information indicating a type of the sensing program that controls the execution and information indicating execution order of the sensing programs is set.

FIG.13is a view illustrating an example of updating of the sensing program.

The sensing program may be executed in each of the sensor device21and a controller51, which is the device on the host side, and a predetermined function may be implemented. In this case, the sensing program of the controller51may be updated similarly to the sensing program of the sensor device21. The controller51is, for example, a data processing device on the host side such as the CPU of the mobile terminal2-1and a CPU of a PC mounted on the arm robot2-2.

A sensing program that updates the firmware of the sensor device21and a sensing program that updates the firmware of the controller51may be included in one sensing program set to be provided.

The sensing program and the sensing program set may be provided for a fee or for free. One sensing program set may include both a paid sensing program and a free sensing program.

When updating the sensing program as described above, the sensor device21may be authenticated by the program management server1on the basis of key information for authentication, and the updating may be performed in a case where it is confirmed that the sensor device is a legitimate device. The key information for authentication is prepared as unique information in each sensor device21.

The authentication of the sensor device21using the key information for authentication may be performed not when the sensing program is updated but when the sensing program is executed.

Provision Source of Sensing Program

FIG.14is a view illustrating an example of a provision source of the sensing program.

As illustrated inFIG.14, the sensing program provided from the program management server1to each device is developed by, for example, a developer that performs user registration of a service in the program providing system. Each developer is provided with information regarding a specification of the sensor device21and a development tool such as a software development kit (SDK) by a service provider that operates the service using the program providing system.

Each developer develops the sensing program or the sensing program set by using the SDK and the like, and uploads the same from its own computer to the program management server1. The uploaded sensing program and sensing program set are stored in the sensing program DB to be managed.

The program management server1manages a using situation of each sensing program and sensing program set such as the number of times of installation and the number of times of execution in each device. A predetermined incentive such as payment of an amount of money corresponding to the using situation and issuance of points may be provided from the service provider to the developer.

FIG.15is a view illustrating an example of generation of the sensing program set.

The sensing program set may be generated by any user by putting together the sensing programs developed and uploaded by each developer.

In the example inFIG.15, the indoor ranging program set is generated by putting together three sensing programs: the ranging program D, the ranging program E, and the ranging program F among the ranging programs A to G.

The indoor ranging program set generated in this manner is released by the program management server1as an installable sensing program set, and is appropriately installed on a predetermined device similarly to the sensing program set developed by the developer.

An incentive may be provided to the user who generates the program set by putting together the plurality of sensing programs.

<Use Case of Sensing Program>

Use Case of Conveyance Robot

Here, a use case of the environment sensing is described.

In a case where the conveyance robot2-5conveys the conveyance object, the sensor device21mounted on the conveyance robot2-5performs the environment sensing by executing the sensing program. In order to safely move to a destination, detection of an obstacle, measurement of a distance to the obstacle, estimation of a direction of the obstacle, estimation of a self-position and the like are performed as the environment sensing.

FIG.16is a view illustrating a state of conveyance by the conveyance robot2-5.

FIG.16illustrates a state of the conveyance robot2-5that moves in a kitchen in a building. A cooked dish is placed on the top plate prepared as the placing table for the conveyance object. In this example, the conveyance robot2-5is used for serving the dish.

The conveyance robot2-5plans a moving route, avoids the obstacle and the like on the basis of a result of the environment sensing by the sensor device21, moves to the destination, and serves the dish.

FIG.17is an enlarged view of an appearance of the conveyance robot2-5.

As illustrated inFIG.17, the conveyance robot2-5is formed by connecting an annular base101and a circular thin plate-shaped top plate102with a thin rod-shaped support arm103. A plurality of tires is provided on a bottom surface side of the base101. The base101serves as a moving unit that implements the movement of the conveyance robot2-5.

A radial length of the base101and a radial length of the top plate102are substantially the same. In a case where there is the top plate102substantially directly above the base101, the support arm103is in an oblique state as illustrated inFIG.17.

The support arm103includes an arm member103-1and an arm member103-2. A diameter of the arm member103-1on the top plate102side is slightly smaller than a diameter of the arm member103-2on the base101side. When the arm member103-1is accommodated inside the arm member103-2at an extension/contraction unit103A, a length of the support arm103is adjusted as indicated by a bidirectional arrow.

An angle of the support arm103may be adjusted at each of a connection between the base101and the support arm103and a connection between the top plate102and the support arm103.

FIG.18is a view illustrating an example of an attitude of the conveyance robot2-5when the dish is placed thereon.

In the example inFIG.18, by setting the support arm103substantially vertically and setting the length thereof to a maximum length, a height of the top plate102is adjusted to be substantially the same as a height of a top plate of the cooking robot2-4.

When the conveyance robot2-5is in such a state, the cooking arm of the cooking robot2-4places the dish on the top plate102. In the example inFIG.18, the dish completed by the cooking operation of the cooking robot2-4is placed by the cooking arm.

As illustrated inFIG.18, the cooking robot2-4is provided with a plurality of cooking arms that performs various cooking operations such as cutting of an ingredient and roasting of the ingredient. The cooking operation by the cooking arm is performed in accordance with cooking data that defines contents and order of the cooking operations. The cooking data includes information regarding each cooking process until the dish is completed.

In this manner, the dish served by the conveyance robot2-5is the dish cooked by the cooking robot2-4. A dish made by a human may be placed on the top plate102by the human and served.

FIG.19is a plan view illustrating a layout of a space in which the conveyance robot2-5moves.

As illustrated inFIG.19, each room including a kitchen #1, a dining room #2, and a party room #3is prepared in a building in which the conveyance robot2-5moves. There is a corridor #11between the kitchen #1and the dining room #2, and a corridor #12between the kitchen #1and the party room #3.

Outside the building inFIG.19a range of which is indicated by a broken line, a garden #21is provided so as to face the dining room #2and the party room #3. As indicated with a hatch, a large window is provided on a wall of the party room #3including a wall on a side of the garden #21. Windows are also provided on both sides of the corridor #12.

A sensing algorithm used in a case where the conveyance robot2-5moves in such a space and serves a dish is described.

Specific Example of Sensing Algorithm

FIG.20is a view illustrating an example of the sensing algorithm defined by the sensing program prepared in the sensor device21of the conveyance robot2-5.

As illustrated inFIG.20, the sensor device21of the conveyance robot2-5is provided with a program that defines each of algorithms A1to A9. The algorithms A1to A9are the sensing algorithms for environment sensing used when serving the dish.

Note that, in the following, for convenience of description, a case where the environment sensing is performed using the stereo camera32B provided on the sensor device21is mainly described. Depending on the algorithm, the output of the ToF sensor32C is used.

As illustrated inFIG.20, the algorithm A1is divided into an algorithm A1-1and an algorithm A1-2.

The algorithm A1is used as, for example, a default sensing algorithm.

The algorithm A1-1is the sensing algorithm with high accuracy although this may be used only in a narrow place. Responsiveness of the algorithm A1-1is lower than a reference speed. The responsiveness is a time required for the environment sensing.

The environment sensing by the algorithm A1-1is performed with imaging resolution of the stereo camera32B set higher than reference resolution. Since the environment sensing is performed on the basis of an image having high resolution and a large data amount, a processing speed decreases and the responsiveness decreases, but the accuracy increases.

Furthermore, the output of the ToF sensor32C is also used for the environment sensing by the algorithm A1-1. At the time of environment sensing by the algorithm A1-1, a detection range of the ToF sensor32C is set as a range narrower than a reference size. Furthermore, power for driving the ToF sensor32C is set as power stronger than reference power.

The algorithm A1-2is the sensing algorithm with lower accuracy than that of the algorithm A1-1although this may be used in a wider place than that with the algorithm A1-1.

The environment sensing by the algorithm A1-2is performed with imaging resolution of the stereo camera32B set lower than the imaging resolution of the algorithm A1-1. Since the environment sensing is performed on the basis of an image having slightly lower resolution and a small data amount, a processing speed increases and the responsiveness increases as compared to the algorithm A1-1, but the accuracy decreases.

Furthermore, the environment sensing by the algorithm A1-2is performed with a baseline length of the stereo camera32B set longer than the baseline length with the algorithm A1-1. A distance is measured using the stereo camera32B using parallax between two cameras. By increasing the baseline length represented as the distance between the two cameras, it is possible to measure the distance to a distant target.

The algorithm A2is the sensing algorithm that may be used in a wide place and has sufficient responsiveness to capture a moving object.

The environment sensing by the algorithm A2is performed with the baseline length of the stereo camera32B set longer than the baseline length with the algorithm A1-2.

The algorithm A3is a sensing algorithm resistant to a noise such as rain.

The environment sensing by the algorithm A3is performed such that image processing to remove noise is performed on the distance image imaged by the stereo camera32B, and then the distance is calculated on the basis of the distance image acquired after the noise removal. Specifically, since image processing of removing raindrops as the noise is added, it is possible to cope with rain although the processing speed decreases and responsiveness decreases.

A known technology is used for the noise removal. The technology for noise removal is disclosed in, for example, “https://digibibo.com/blog-entry-3422.html, and http://www.robot.t.u-tokyo.ac.jp/˜yamashita/paper/A/A025Final.pdf”.

The algorithm A4is a sensing algorithm resistant to direct sunlight.

The environment sensing by the algorithm A4is performed while adjusting imaging parameters so as to increase the shutter speed and decrease the sensitivity of the stereo camera32B.

The algorithm A5is a sensing algorithm adaptable to a darkish place.

The environment sensing by the algorithm A5is performed while adjusting the imaging parameters so as to decrease the shutter speed and increase the sensitivity. Although ranging of the moving object becomes difficult, accuracy is secured by decreasing the moving speed of the conveyance robot2-5.

Furthermore, the environment sensing by the algorithm A5may be performed using another sensor. For example, another sensor such as the ToF sensor32C that does not depend on visible light is used. By using the ToF sensor32C, it becomes difficult to perform ranging into distance, but it becomes possible to perform ranging even in a dark place. A collision risk may be avoided by decreasing the moving speed of the conveyance robot2-5for the fact that the ranging into distance cannot be performed.

The algorithm A6is a sensing algorithm resistant to a shadow.

The environment sensing by the algorithm A6is performed while adjusting the imaging parameters so as to extend a dynamic range of luminance of the stereo camera32B. By extending the dynamic range of luminance, simultaneous ranging of a bright place and a dark place becomes possible.

The algorithm A7is a sensing algorithm resistant to ranging of a reflector such as a mirror.

The environment sensing by the algorithm A7is performed using a sensor that does not depend on visible light. For example, a sensor that performs ranging using a sound wave is used.

The algorithm A8is a sensing algorithm capable of detecting a transparent object.

The environment sensing by the algorithm A8is performed with imaging resolution of the stereo camera32B set higher than reference resolution.

The algorithm A9is a sensing algorithm capable of coping with a completely dark place.

In the environment sensing by the algorithm A9, ranging is performed using an active sensor such as the ToF sensor32C or the LiDAR32E.

With reference toFIGS.21to25, an environment sensing condition that is a condition of selection of the sensing algorithm as described above is described.

As illustrated in a balloon inFIG.21, the kitchen #1is a space having the following features: there are many objects and the space is narrow, there are many stationary objects, and there are many dangerous objects such as knife and glass. Although there are few moving objects, there is a high possibility of colliding with an obstacle because the space is narrow. In order for the conveyance robot2-5to move safely, it is required that accuracy be guaranteed even if the responsiveness is poor.

The sensing algorithm suitable for performing the environment sensing in the kitchen #1having such features is the algorithm A1-1. In a case where the conveyance robot2-5is in the kitchen #1, the algorithm A1-1is used for the environment sensing as indicated by a destination of an open arrow.

In a case where the conveyance robot2-5detects a situation that the conveyance robot2-5itself is in the kitchen #1, this selects to execute the sensing program that defines the algorithm A1-1according to the environment sensing condition to perform the environment sensing in the kitchen #1.

The environment sensing condition is a condition of the selection of the sensing algorithm, that is, the selection of the sensing program.

As illustrated in a balloon inFIG.22, the corridor #11is a space having the following features: it is sometimes completely dark due to electricity saving, and a reflector such as a mirror is put. Although there is no moving object, there is a high possibility that the conveyance robot2-5collides with a wall because this is the completely dark place, or this erroneously recognize itself reflected by the mirror as an obstacle. In order for the conveyance robot2-5to move safely, this is required to be resistant to reflection by the reflector such as the mirror and to cope with the completely dark place.

The sensing algorithms suitable for performing the environment sensing in the corridor #11having such features are the algorithms A7and A9. In a case where the conveyance robot2-5is in the corridor #11, the algorithms A7and A9are used for the environment sensing as indicated by a destination of an open arrow.

In a case where the conveyance robot2-5detects a situation that the conveyance robot2-5itself is in the corridor #11, this selects to execute the sensing programs that define the algorithms A7and A9according to the environment sensing condition to perform the environment sensing in the corridor #11.

For example, the environment sensing using the algorithm A7and the environment sensing using the algorithm A9are executed in parallel or alternately. In this manner, in a case where a plurality of sensing algorithms is set as the sensing algorithms suitable for the environment sensing condition, the environment sensing is performed by switching the plurality of sensing algorithms.

As illustrated in a balloon inFIG.23, the party room #3is a space having the following features: it is viewable into distance, the space is wide, there are few objects, there are many people, and there is a large window. Although this is wide, there is a high possibility of collision with an obstacle because there are many people and many moving objects, or a high possibility of collision because the large window cannot be recognized. Furthermore, direct sunlight entering through the large window is likely to cause erroneous recognition. In order for the conveyance robot2-5to move safely, it is required that this may be used in a wide place, resistant to direct sunlight, and may detect a transparent obstacle.

The sensing algorithms suitable for performing the environment sensing in the party room #3having such features are the algorithms A2, A4, and A8. In a case where the conveyance robot2-5is in the party room #3, the algorithms A2, A4, and A8are used for the environment sensing as indicated by a destination of an open arrow.

In a case where the conveyance robot2-5detects a situation that the conveyance robot2-5itself is in the party room #3, this selects to execute the sensing programs that define the algorithms A2, A4, and A8according to the environment sensing condition to perform the environment sensing in the party room #3.

As illustrated in a balloon inFIG.24, the garden #21is a space having the following features: this is exposed to direct sunlight, it might be darkish, it might rain, and a wild animal such as a bird might jump in. There is a high possibility of erroneous recognition due to direct sunlight, rain, or darkishness. Furthermore, there is a high possibility of colliding with a wild animal that suddenly jumps in. In order for the conveyance robot2-5to move safely, this is required to be resistant to rain, direct sunlight, and shadow and to cope with the darkish place.

The sensing algorithms suitable for performing the environment sensing in the garden #21having such features are the algorithms A3, A4, A5, and A6. In a case where the conveyance robot2-5is in the garden #21, the algorithms A3, A4, A5, and A6are used for the environment sensing as indicated by a destination of an open arrow.

In a case where the conveyance robot2-5detects a situation that the conveyance robot2-5itself is in the garden #21, this selects to execute the sensing programs that define the algorithms A3, A4, A5, and A6according to the environment sensing condition to perform the environment sensing in the garden #21.

FIG.25is a view illustrating an example of correspondence between a serving situation and the sensing algorithm.

For example, the sensing algorithms that may be used in a case where the conveyance object is carried from the kitchen #1to another space are the algorithms A1-1, A1-2, A2, A4, A7, A8, and A9. Out of the algorithms A1-1, A1-2, A2, A4, A7, A8, and A9, the sensing algorithm corresponding to the environment sensing condition for performing the environment sensing in each use case is selected.

In a case of moving from the kitchen #1to the party room #3, transition of the sensing algorithm is transition from the algorithm A1-1to the algorithm A2.

In a case of moving from the kitchen #1to the garden #21, transition of the sensing algorithm is transition from the algorithm A1-1to the algorithm A4.

In a case of moving from the kitchen #1to the party room #3through the glass-walled corridor #12, transition of the sensing algorithm is transition from the algorithm A1-1to the algorithm A8, and further from the algorithm A8to the algorithm A2.

In a case of moving from the kitchen #1to the party room #3through the corridor with the mirror, transition of the sensing algorithm is the transition from the algorithm A1-1to the algorithm A7, and further from the algorithm A7to the algorithm A2.

In a case of moving from the kitchen #1to the dining room #2through the dark corridor #11without lighting, transition of the sensing algorithm is the transition from the algorithm A1-1to the algorithm A9and further from the algorithm A9to the algorithm A2.

The sensing algorithms that may be used in a case of serving in the garden #21are the algorithms A3, A4, A5, and A6. In a situation of serving in the garden #21similarly, the sensing algorithm corresponding to the environment sensing condition that the environment sensing is performed in each use case is selected.

In this manner, in the sensor device21, various situations of the conveyance robot2-5such as a place and an action are detected, and the sensing algorithm is selected according to the environment sensing condition that the environment sensing is performed in such situation.

<Configuration of Conveyance Robot>

FIG.26is a block diagram illustrating a configuration example of hardware of the conveyance robot2-5.

The conveyance robot2-5is formed by connecting a top plate lift drive unit122, a tire drive unit123, a sensor group124, and a communication unit125to a controller121. The sensor device21is also connected to the controller121.

The controller121includes a CPU, a ROM, a RAM, a flash memory and the like. The controller121executes a predetermined program and controls an entire operation of the conveyance robot2-5including the sensor device21. The controller121corresponds to the controller51on the host side (FIG.13).

The top plate lift drive unit122includes a motor and the like provided on the connection between the base101and the support arm103, the connection between the top plate102and the support arm103and the like. The top plate lift drive unit122drives the respective connections.

Furthermore, the top plate lift drive unit122includes a rail or a motor provided inside the support arm103. The top plate lift drive unit122extends and contracts the support arm103.

The tire drive unit123includes a motor that drives the tires provided on the bottom surface of the base101.

The sensor group124includes various sensors such as a positioning sensor, a gyro sensor, an acceleration sensor, a temperature sensor, and an illuminance sensor. Sensor data indicating a detection result by the sensor group124is output to the controller121.

The communication unit125is a wireless communication module such as a wireless LAN module and a mobile communication module compatible with a long term evolution (LTE). The communication unit125communicates with an external device such as the program management server1.

FIG.27is a block diagram illustrating a functional configuration example of the conveyance robot2-5.

At least a part of functional units illustrated inFIG.27is implemented by executing a predetermined program by the CPU forming the controller121and the CPU forming the controller31of the sensor device21.

In the controller121, a route information acquisition unit151, a positioning control unit152, a movement control unit153, an attitude control unit155, an environment data acquisition unit156, and a peripheral state recognition unit157are implemented.

In contrast, in the controller31of the sensor device21, a situation detection unit201and a sensing control unit202are implemented. The sensor device21is a data processing device that controls the sensing algorithm.

The route information acquisition unit151of the controller121controls the communication unit125and receives information of a destination and a moving route transmitted from a control device not illustrated. The information received by the route information acquisition unit151is output to the movement control unit153.

The route information acquisition unit151may plan the moving route on the basis of the destination and a current position of the conveyance robot2-5at a timing when the conveyance object is prepared and the like.

In this case, the route information acquisition unit151serves as an operation plan setting unit that plans the operation of the conveyance robot2-5and sets an operation plan.

The positioning control unit152detects the current position of the conveyance robot2-5. For example, the positioning control unit152generates a map of a space in which the conveyance robot2-5is installed on the basis of a detection result by a distance sensor forming the sensor device21. Sensor data, which is an output of the sensor device21, is acquired by the environment data acquisition unit156and supplied to the positioning control unit152.

The positioning control unit152detects the current position by specifying its own position on the generated map. Information on the current position detected by the positioning control unit152is output to the movement control unit153. The detection of the current position by the positioning control unit152may be performed on the basis of an output of the positioning sensors forming the sensor group124.

The movement control unit153controls the tire drive unit123to control the movement of the conveyance robot2-5on the basis of the information supplied from the route information acquisition unit151and the current position detected by the positioning control unit152.

Furthermore, the movement control unit153controls the movement so as to avoid an obstacle in a case where information regarding the obstacle around the same is supplied from the peripheral state recognition unit157. The obstacle includes various moving objects and stationary objects such as people, furniture, and home appliances. In this manner, the movement control unit153controls the movement of the conveyance robot2-5accompanying the conveyance of the conveyance object on the basis of the result of the environment sensing by the sensor device21.

The attitude control unit155controls the top plate lift drive unit122to control the attitude of the conveyance robot2-5. Furthermore, in conjunction with the control by the movement control unit153, the attitude control unit155controls the attitude of the conveyance robot2-5during the movement so as to keep the top plate102horizontal.

The attitude control unit155controls the attitude of the conveyance robot2-5according to the peripheral state recognized by the peripheral state recognition unit157. For example, the attitude control unit155controls the attitude of the conveyance robot2-5so that the height of the top plate102approaches the height of the top plate of the cooking robot2-4or a height of a top plate of a dining table recognized by the peripheral state recognition unit157.

The environment data acquisition unit156controls the sensor device21to perform the environment sensing and acquires the sensor data indicating the result of the environment sensing. The sensor data acquired by the environment data acquisition unit156is supplied to the positioning control unit152and the peripheral state recognition unit157.

The peripheral state recognition unit157recognizes the peripheral state on the basis of the sensor data indicating the result of the environment sensing supplied from the environment data acquisition unit156. Information indicating a recognition result by the peripheral state recognition unit157is supplied to the movement control unit153and the attitude control unit155.

In a case where detection of the obstacle, measurement of a distance to the obstacle, estimation of a direction of the obstacle, estimation of a self-position and the like are performed as the environment sensing by the sensor device21, the peripheral state recognition unit157outputs the information regarding the obstacle as information indicating the recognition result of the peripheral state.

The detection of the obstacle, the measurement of the distance to the obstacle, the estimation of the direction of the obstacle, the estimation of the self-position and the like may be performed by the peripheral state recognition unit157on the basis of the result of the environment sensing by the sensor device21. In this case, the sensor data used for each processing performed by the peripheral state recognition unit157is detected by the environment sensing by the sensor device21.

In this manner, contents of the processing performed by the sensor device21as the environment sensing are arbitrary. That is, raw data detected by the sensor provided on the sensor device21may be directly supplied to the controller121as the sensor data, or processing and analysis of the raw data may be performed on the sensor device21side, and a result of the processing and analysis may be supplied to the controller121as the sensor data.

The situation detection unit201on the sensor device21side detects the situation of the conveyance robot2-5. The situation of the conveyance robot2-5is detected on the basis of, for example, the sensor data output from the sensor forming the sensor group124or the sensor data output from the sensor provided on the sensor device21.

The situation of the conveyance robot2-5includes, for example, the operation of the conveyance robot2-5such as the operation performed by the same, the place where the conveyance robot2-5is located, the weather, temperature, humidity, and brightness in the place where the conveyance robot2-5is located. Furthermore, the situation of the conveyance robot2-5also includes an external situation such as a situation of a person with whom the conveyance robot2-5is communicating and a situation of the obstacle around the conveyance robot2-5.

The situation detection unit201outputs information indicating such situation of the conveyance robot2-5to the sensing control unit202.

The sensing control unit202selects the sensing algorithm according to the environment sensing condition to perform the environment sensing in the situation detected by the situation detection unit201, and executes the sensing program that defines the selected sensing algorithm.

For example, the sensing algorithm or the sensing program is associated with each environment sensing condition. The sensing control unit202selects the sensing algorithm or the sensing program corresponding to the environment sensing condition using the ID as the identification data. The sensing program set may be selected according to the environment sensing condition.

The sensing control unit202drives each sensor provided on the sensor device21by executing the sensing program, and outputs the sensor data to the controller121on the basis of the output of each sensor.

<Operation of Conveyance Robot>

Here, the operation of the conveyance robot2-5having the above-described configuration is described.

Basic Processing

First, basic sensing processing of the conveyance robot2-5is described with reference to a flowchart inFIG.28.

At step S1, the situation detection unit201of the sensor device21performs the environment sensing using a default sensing algorithm.

At step S2, the situation detection unit201detects the situation of the conveyance robot2-5on the basis of the sensor data from the sensor group124or the sensor data from the sensor device21.

At step S3, the sensing control unit202determines whether or not to switch the sensing algorithm. For example, in a case where the situation detection unit201detects that the situation of the conveyance robot2-5changes, it is determined that the sensing algorithm is to be switched.

In a case where it is determined at step S3that the sensing algorithm is to be switched, at step S4, the sensing control unit202selects the sensing algorithm according to the environment sensing condition to perform the environment sensing in the changed situation, and executes the sensing program that defines the selected sensing algorithm.

Thereafter, the procedure returns to step S2and processing of detecting the situation of the conveyance robot2-5is performed, and the above-described processing is repeated. In a case where it is determined at step S3that the sensing algorithm is not switched, similarly, the procedure returns to step S2and the above-described processing is repeated.

Specific Processing

Next, serving processing of the conveyance robot2-5is described with reference to a flowchart inFIG.29.

At step S11, the situation detection unit201detects a place of the conveyance robot2-5on the basis of the sensor data from the sensor group124or the sensor data from the sensor device21.

At step S12, the sensing control unit202determines whether or not the conveyance robot2-5is located indoors on the basis of a detection result by the situation detection unit201.

In a case where it is determined at step S12that the conveyance robot2-5is located indoors, at step S13, the sensing control unit202performs the environment sensing using an indoor basic algorithm.

The indoor basic algorithm is the sensing algorithm to adjust imaging parameters of the stereo camera32B such as a shutter speed and sensitivity according to intensity of ambient light and perform the environment sensing. The shutter speed is set to a standard to lower speed, and the sensitivity is set to standard to higher sensitivity.

At step S14, the sensing control unit202performs indoor processing. In the indoor processing, the sensing algorithm is selected according to an indoor situation, and the environment sensing is performed. The sensing algorithm used for the environment sensing is appropriately switched from the indoor basic algorithm to another sensing algorithm. The indoor processing is described later in detail with reference to a flowchart inFIG.30.

In contrast, in a case where it is determined at step S12that the conveyance robot2-5is not located indoors, that is, located outdoors, the sensing control unit202performs the environment sensing using an outdoor basic algorithm at step S15.

The outdoor basic algorithm is the sensing algorithm to adjust imaging parameters of the stereo camera32B such as a shutter speed and sensitivity according to intensity of ambient light and perform the environment sensing. The shutter speed is set to a standard to higher speed, and the sensitivity is set to standard to lower sensitivity.

At step S16, the sensing control unit202performs outdoor processing. In the outdoor processing, the sensing algorithm is selected according to an outdoor situation, and the environment sensing is performed. The sensing algorithm used for the environment sensing is appropriately switched from the outdoor basic algorithm to another sensing algorithm. The outdoor processing is described later in detail with reference to a flowchart inFIG.31.

After the indoor processing is performed at step S14or after the outdoor processing is performed at step S16, the procedure returns to step S11, and the subsequent processing is repeated.

Here, the indoor processing performed at step S14inFIG.29is described with reference to the flowchart inFIG.30.

At step S21, the sensing control unit202determines whether or not the conveyance robot2-5is located in the kitchen #1.

In a case where it is determined at step S21that the conveyance robot2-5is located in the kitchen #1, at step S22, the sensing control unit202selects the algorithm A1-1according to the environment sensing condition that the environment sensing is performed in the kitchen #1and performs the environment sensing.

In a case where it is determined at step S21that the conveyance robot2-5is not located in the kitchen #1, at step S23, the sensing control unit202determines whether or not the conveyance robot2-5is located in the dining room #2.

In a case where it is determined at step S23that the conveyance robot2-5is located in the dining room #2, at step S24, the sensing control unit202selects the algorithm A1-2according to the environment sensing condition that the environment sensing is performed in the dining room #2and performs the environment sensing.

In a case where it is determined at step S23that the conveyance robot2-5is not located in the dining room #2, at step S25, the sensing control unit202determines whether or not the conveyance robot2-5is located in the party room #3.

In a case where it is determined at step S25that the conveyance robot2-5is located in the party room #3, at step S26, the sensing control unit202selects the algorithm A2according to the environment sensing condition that the environment sensing is performed in the party room #3and performs the environment sensing.

In a case where it is determined at step S25that the conveyance robot2-5is not located in the party room #3, at step S27, the sensing control unit202determines whether or not the conveyance robot2-5is located in the corridor #11.

In a case where it is determined at step S27that the conveyance robot2-5is located in the corridor #11, at step S28, the sensing control unit202selects the algorithm A5according to the environment sensing condition that the environment sensing is performed in the corridor #11and performs the environment sensing.

After the environment sensing is performed using any one of the algorithms A1-1, A1-2, and A2or the algorithm A5according to the place of the conveyance robot2-5, or in a case where it is determined at step S27that the conveyance robot2-5is not located in the corridor #11, the procedure shifts to step S29.

At step S29, the sensing control unit202determines whether or not there is a transparent object near the conveyance robot2-5. It is determined whether or not there is the transparent object on the basis of the detection result of the situation by the situation detection unit201.

In a case where it is determined at step S29that there is a transparent object near the conveyance robot2-5, at step S30, the sensing control unit202selects the algorithm A8according to the environment sensing condition that the environment sensing is performed in a place where there is the transparent obstacle and performs the environment sensing.

After the environment sensing is performed at step S30, or in a case where it is determined at step S29that there is no transparent object near the conveyance robot2-5, the procedure shifts to step S31.

At step S31, the sensing control unit202determines whether or not there is a reflecting object near the conveyance robot2-5. It is determined whether or not there is the reflecting object on the basis of the detection result of the situation by the situation detection unit201.

In a case where it is determined at step S31that there is the reflecting object near the conveyance robot2-5, at step S32, the sensing control unit202selects the algorithm A7according to the environment sensing condition that the environment sensing is performed in a place where there is the reflector and performs the environment sensing.

After the environment sensing is performed at step S32, or in a case where it is determined at step S31that there is no reflecting object near the conveyance robot2-5, the procedure shifts to step S33.

At step S33, the sensing control unit202determines whether or not brightness in the place of the conveyance robot2-5is sufficient. It is determined whether or not the brightness is sufficient on the basis of the detection result of the situation by the situation detection unit201.

In a case where it is determined at step S33that the brightness in the place of the conveyance robot2-5is not sufficient, at step S34, the sensing control unit202selects the algorithm A9according to the environment sensing condition that the environment sensing is performed in the completely dark place and performs the environment sensing.

After the environment sensing is performed at step S34, or in a case where it is determined at step S33that the brightness in the place of the conveyance robot2-5is sufficient, the procedure shifts to step S35.

At step S35, the sensing control unit202determines whether or not the place of the conveyance robot2-5changes. It is determined whether or not the place of the conveyance robot2-5changes on the basis of the detection result of the situation by the situation detection unit201.

In a case where it is determined at step S35that the place of the conveyance robot2-5does not change, the procedure returns to step S29, and the above-described processing is repeated.

In contrast, in a case where it is determined at step S35that the place of the conveyance robot2-5changes, the procedure returns to step S14inFIG.29, and the subsequent processing is performed.

Next, the outdoor processing performed at step S16inFIG.29is described with reference to the flowchart inFIG.31.

At step S51, the sensing control unit202determines whether or not the weather in the place of the conveyance robot2-5is fine. It is determined whether or not the weather is fine on the basis of the detection result of the situation by the situation detection unit201.

In a case where it is determined at step S51that the weather in the place of the conveyance robot2-5is fine, at step S52, the sensing control unit202determines whether or not this is a place where a shadow is likely to appear.

In a case where it is determined at step S52that the place is the place where the shadow is likely to appear, at step S53, the sensing control unit202selects the algorithm A6according to the environment sensing condition to perform the environment sensing in the place where the shadow is likely to appear and performs the environment sensing.

In a case where it is determined at step S52that the place is not the place where the shadow is likely to occur, at step S54, the sensing control unit202selects the algorithm A4according to the environment sensing condition to perform the environment sensing under direct sunlight and performs the environment sensing.

In contrast, in a case where it is determined at step S51that the weather is not fine, the procedure shifts to step S55.

At step S55, the sensing control unit202determines whether or not it is raining. It is determined whether or not it is raining on the basis of the detection result of the situation by the situation detection unit201.

In a case where it is determined at step S55that it is raining, at step S56, the sensing control unit202selects the algorithm A3according to the environment sensing condition that the environment sensing is performed in a place where it is raining and performs the environment sensing.

In a case where it is determined at step S55that it is not raining, at step S57, the sensing control unit202selects the algorithm A5according to the environment sensing condition that the environment sensing is performed in a darkish place and performs the environment sensing.

After the environment sensing is performed using any one of the algorithms A3to A6, at step S58, the sensing control unit202determines whether or not the place of the conveyance robot2-5changes.

In a case where it is determined at step S58that the place of the conveyance robot2-5does not change, at step S59, the sensing control unit202determines whether or not the weather changes.

In a case where it is determined at step S59that the weather changes, the procedure returns to step S51, and the above-described processing is repeated.

In a case where it is determined at step S59that the weather does not change, at step S60, the sensing control unit202keeps the selected sensing algorithm and performs the environment sensing. Thereafter, the procedure returns to step S58, and the above-described processing is repeated.

In a case where it is determined at step S58that the place changes, the procedure returns to step S16inFIG.29, and the subsequent processing is repeated. By the above-described processing, the conveyance robot2-5may adaptively select the sensing algorithm according to the situation and perform the highly accurate environment sensing.

In a case of a timing to perform the environment sensing using a certain sensing algorithm, when the sensing program that defines the sensing algorithm is not prepared, it is possible to access the program management server1to acquire the sensing program from the program management server1.

Although the processing performed in the sensor device21mounted on the conveyance robot2-5is described, the above-described processing of performing the environment sensing while switching the sensing algorithm is performed in each device on which the sensor device21is mounted such as the mobile terminal2-1.

<Variation>

Example of Case where Sensing Algorithm is Externally Selected

The selection of the sensing algorithm corresponding to the environment sensing condition is performed in the sensor device21, but this may be performed by a device outside the device on which the sensor device21is mounted.

FIG.32is a view illustrating an example of control of the sensing algorithm.

In the example inFIG.32, the selection of the sensing algorithm corresponding to the environment sensing condition is performed by the program management server1, which is the external device. In this case, the configuration of the controller31inFIG.27is implemented in the program management server1. The program management server1is a data processing device that controls the sensing program executed by the sensor device21mounted on the conveyance robot2-5.

As indicated by an arrow #1, from the conveyance robot2-5to the program management server1, sensor data used for detecting a situation is transmitted and the sensing program is requested.

The situation detection unit201of the program management server1detects the situation of the conveyance robot2-5on the basis of the sensor data transmitted from the conveyance robot2-5. Furthermore, the environment sensing condition corresponding to the situation of the conveyance robot2-5is determined by the sensing control unit202, and the sensing algorithm is selected.

The sensing control unit202of the program management server1transmits the sensing program that defines the sensing algorithm corresponding to the environment sensing condition to the sensor device21mounted on the conveyance robot2-5, and causes the same to execute the sensing program as indicated by an arrow #2. In this case, the sensing control unit202of the program management server1serves as a data processing unit that transmits the sensing program that defines the sensing algorithm corresponding to the environment sensing condition to the conveyance robot2-5.

In this manner, the sensing algorithm may be controlled by a device outside the sensor device21. For example, the controller121of the conveyance robot2-5on which the sensor device21is mounted may be made an external device, and the sensing algorithm may be controlled by the controller121.

The sensing program that defines the sensing algorithm corresponding to the environment sensing condition may be executed by the program management server1or the controller121, which is the external device, and information indicating an execution result may be transmitted to the sensor device21.

FIG.33is a block diagram illustrating a configuration example of hardware of a computer that implements the program management server1.

A central processing unit (CPU)1001, a read only memory (ROM)1002, and a random access memory (RAM)1003are connected to one another by a bus1004.

An input/output interface1005is further connected to the bus1004. An input unit1006including a keyboard, a mouse and the like, and an output unit1007including a display, a speaker and the like are connected to the input/output interface1005. Furthermore, a storage unit1008including a hard disk, a nonvolatile memory and the like, a communication unit1009including a network interface and the like, and a drive1010that drives a removable medium1011are connected to the input/output interface1005.

The control of the sensing algorithm as described above is implemented by execution of a predetermined program by the CPU1001.

Example of Program

The above-described series of processing may be executed by hardware or may be executed by software. In a case where the series of processing is executed by software, a program forming the software is installed on a computer incorporated in dedicated hardware, a general-purpose personal computer or the like.

The program to be installed is recorded in the removable medium1011illustrated inFIG.33including an optical disk (compact disc-read only memory (CD-ROM), digital versatile disc (DVD) and the like), a semiconductor memory and the like to be provided. Furthermore, this may be provided via a wired or wireless transmission medium such as a local area network, the Internet, and digital broadcasting. The program may be installed in advance on the ROM1002and the storage unit1008.

Note that, the program executed by the computer may be the program of which processing is performed in chronological order in the order described in this specification or may be the program of which processing is performed in parallel or at required timing such as when a call is issued.

Note that, in this specification, the system is intended to mean assembly of a plurality of components (devices, modules (parts) and the like) and it does not matter whether or not all the components are in the same casing. Therefore, a plurality of devices accommodated in different casings and connected via a network and one device in which a plurality of modules is accommodated in one casing are the systems.

The effects described in this specification are illustrative only; the effects are not limited thereto and there may also be another effect.

The embodiments of the present technology are not limited to the above-described embodiments and various modifications may be made without departing from the gist of the present technology.

For example, the present technology may be configured as cloud computing in which one function is shared by a plurality of devices via the network to process together.

Furthermore, each step described in the above-described flowchart may be executed by one device or executed by a plurality of devices in a shared manner.

Moreover, in a case where a plurality of processes is included in one step, the plurality of processes included in one step may be executed by one device or by a plurality of devices in a shared manner.

REFERENCE SIGNS LIST

1Program management server2-1Mobile terminal2-2Arm robot2-3Moving body2-4Cooking robot2-5Conveyance robot21Sensor device31Controller32Sensor group121Controller124Sensor group201Situation detection unit202Sensing control unit