Patent Description:
Conventionally, a robot that outputs different types of output sounds corresponding to a plurality of motions has been widely used.

Furthermore, conventionally, there has been proposed a game apparatus that determines priority of sound data at each time point during progress of a game according to priority information set on the basis of musical analysis, and opens a channel corresponding to sound data with the lowest priority in a case where there is no empty channel (see, for example, Patent Document <NUM>).

Moreover, conventionally, there has been proposed a voice dialogue apparatus that guides user's utterance volume to a desired level by adjusting volume of music to be output (see, for example, Patent Document <NUM>).

Meanwhile, in a case where different types of output sounds are output corresponding to a plurality of motions of a robot, when the plurality of motions has been simultaneously detected, the output sounds corresponding to the motions are simultaneously output. Therefore, there is a case where a user cannot recognize individual output sounds and user experience is deteriorated.

The present technology has been made in view of such a situation, and aims to improve user experience by an output sound of an autonomous mobile body such as a robot and the like. Prior art includes: <CIT>, which describes a robot which detects a voice using a microphone array, and identifies a sound source direction. The robot directs a head portion in the sound source direction.

Hereinafter, a mode for carrying out the present technology will be described. Note that the description will be given in the following order.

First, an embodiment of the present technology will be described with reference to <FIG>.

<FIG> illustrates an embodiment of the information processing system <NUM> to which the present technology is applied.

The information processing system <NUM> includes an autonomous mobile body <NUM>, an information processing server <NUM>, and an operated device <NUM>. The autonomous mobile body <NUM>, the information processing server <NUM>, and the operated device <NUM> are connected via a network <NUM>.

The autonomous mobile body <NUM> is an information processing apparatus that performs autonomous operation not under the control of the information processing server <NUM> or under the control of the information processing server <NUM>. For example, the autonomous mobile body <NUM> includes various robots such as a traveling type, a walking type, a flying type, and a swimming type.

Furthermore, the autonomous mobile body <NUM> is an agent apparatus that can realize communication with a user more naturally and effectively. One of the features of the autonomous mobile body <NUM> is to actively execute various operation that triggers communication with the user (hereinafter also referred to as trigger operation).

For example, the autonomous mobile body <NUM> can actively present information to the user on the basis of environment recognition. Furthermore, for example, the autonomous mobile body <NUM> actively executes various trigger operation that urges the user to perform a predetermined action.

Furthermore, the trigger operation by the autonomous mobile body <NUM> can be said to be active and positive interference with a physical space. The autonomous mobile body <NUM> can move in the physical space and execute various physical operation on a user, a living thing, an article, and the like. According to the features described above of the autonomous mobile body <NUM>, the user can comprehensively recognize the operation of the autonomous mobile body through vision, hearing, and tactile sense, and can realize advanced communication as compared with a case of simply interacting with the user using voice.

Moreover, the autonomous mobile body <NUM> can express its own state or communicate with the user or another autonomous mobile body by outputting an output sound. The output sound of the autonomous mobile body <NUM> includes an operation sound output corresponding to a motion of the autonomous mobile body <NUM> and an utterance sound for communicating with a user, another autonomous mobile body, and the like. The utterance sound is not necessarily a voice representing a language that can be understood by a person, and may be a voice representing a non-language imitating animal cry and the like.

The information processing server <NUM> is an information processing apparatus that controls operation of the autonomous mobile body <NUM>. For example, the information processing server <NUM> has a function of causing the autonomous mobile body <NUM> to execute various trigger operation that triggers communication with a user.

The operated device <NUM> is various devices operated by the autonomous mobile body <NUM> and the information processing server <NUM>. The autonomous mobile body <NUM> can operate the various operated devices <NUM> not under the control of the information processing server <NUM> or under the control of the information processing server <NUM>. The operated device <NUM> includes, for example, a home appliance such as a lighting device, a game device, a television device, and the like.

The network <NUM> has a function of connecting each configuration included in the information processing system <NUM>. For example, the network <NUM> may include a public network such as the Internet, a telephone network, a satellite communication network, and the like, various local area networks (LANs) including Ethernet (registered trademark), a wide area network (WAN), and the like. For example, the network <NUM> may include a dedicated line network such as an Internet protocol-virtual private network (IP-VPN) and the like. For example, the network <NUM> may include a wireless communication network such as Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like.

Next, a configuration example of the autonomous mobile body <NUM> will be described with reference to <FIG>. The autonomous mobile body <NUM> can be various devices that perform autonomous operation based on environment recognition. Hereinafter, a case where the autonomous mobile body <NUM> is a long-ellipsoidal agent-type robot device that autonomously travels by wheels will be described as an example. The autonomous mobile body <NUM> realizes various communication including information presentation, for example, by performing autonomous operation according to a user, the surroundings, and its own situation. The autonomous mobile body <NUM> is, for example, a small robot having such a size and weight that the user can easily lift the autonomous mobile body with one hand.

First, an example of an exterior of the autonomous mobile body <NUM> will be described with reference to <FIG>.

<FIG> is a front view of the autonomous mobile body <NUM>, and <FIG> is a rear view of the autonomous mobile body <NUM>. A and B of <FIG> are perspective views of the autonomous mobile body <NUM>. <FIG> is a side view of the autonomous mobile body <NUM>. <FIG> is a top view of the autonomous mobile body <NUM>. <FIG> is a bottom view of the autonomous mobile body <NUM>.

As illustrated in <FIG>, the autonomous mobile body <NUM> includes an eye portion <NUM> and an eye portion 101R corresponding to a left eye and a right eye at an upper portion of a main body. The eye portion <NUM> and the eye portion 101R are realized by, for example, an LED or the like, and can express a line of sight, a blink, and the like. Note that the eye portion <NUM> and the eye portion 101R are not limited to the example described above, and may be realized by, for example, two single or independent organic light emitting diodes (OLEDs) or the like.

Furthermore, the autonomous mobile body <NUM> includes a camera <NUM> and a camera 102R above the eye portion <NUM> and the eye portion 101R. The camera <NUM> and the camera 102R have a function of imaging a user and surrounding environment. At that time, the autonomous mobile body <NUM> may realize simultaneous localization and mapping (SLAM) on the basis of images captured by the camera <NUM> and the camera 102R.

Note that the eye portion <NUM>, the eye portion 101R, the camera <NUM>, and the camera 102R are arranged on a substrate (not illustrated) arranged inside an exterior surface. Furthermore, the exterior surface of the autonomous mobile body <NUM> is basically formed using an opaque material, but a head cover <NUM> using a transparent or translucent material is provided in a portion corresponding to the substrate on which the eye portion <NUM>, the eye portion 101R, the camera <NUM>, and the camera 102R are arranged. Therefore, a user can recognize the eye portion <NUM> and the eye portion 101R of the autonomous mobile body <NUM>, and the autonomous mobile body <NUM> can image the outside world.

Furthermore, as illustrated in <FIG>, <FIG>, and <FIG>, the autonomous mobile body <NUM> includes a time of flight (ToF) sensor <NUM> at a front lower portion. The ToF sensor <NUM> has a function of detecting a distance to an object existing ahead. The autonomous mobile body <NUM> can detect, for example, distances to various objects with high accuracy or detect a step and the like by the ToF sensor <NUM> to prevent overturning or falling.

Furthermore, as illustrated in <FIG>, <FIG>, and the like, the autonomous mobile body <NUM> includes a connection terminal <NUM> to an external device and a power switch <NUM> on a back surface. The autonomous mobile body <NUM> is connected to the external device via the connection terminal <NUM>, for example, and can perform information communication.

Furthermore, as illustrated in <FIG>, the autonomous mobile body <NUM> includes a wheel <NUM> and a wheel 107R on a bottom surface. The wheel <NUM> and the wheel 107R are driven by different motors (not illustrated). Therefore, the autonomous mobile body <NUM> can realize moving operation such as forward movement, backward movement, turning, and rotation.

Furthermore, the wheel <NUM> and the wheel 107R can be stored inside the main body and protruded to the outside. For example, the autonomous mobile body <NUM> can perform jumping operation by vigorously protruding the wheel <NUM> and the wheel 107R to the outside. Note that <FIG> illustrates a state in which the wheel <NUM> and the wheel 107R are stored inside the main body.

Note that, hereinafter, in a case where it is not necessary to distinguish the eye portion <NUM> and the eye portion 101R individually, they are simply referred to as an eye portion <NUM>. Hereinafter, in a case where it is not necessary to individually distinguish the camera <NUM> and the camera 102R, they are simply referred to as a camera <NUM>. Hereinafter, in a case where it is not necessary to distinguish the wheel <NUM> and the wheel 107R individually, they are simply referred to as a wheel <NUM>.

<FIG> and <FIG> are schematic diagrams illustrating an internal structure of the autonomous mobile body <NUM>.

As illustrated in <FIG>, the autonomous mobile body <NUM> includes an inertial sensor <NUM> and a communication device <NUM> disposed on an electronic substrate. The inertial sensor <NUM> detects acceleration and angular velocity of the autonomous mobile body <NUM>. Furthermore, the communication device <NUM> is a configuration for realizing wireless communication with the outside, and includes, for example, Bluetooth (registered trademark), a Wi-Fi (registered trademark) antenna, and the like.

Furthermore, the autonomous mobile body <NUM> includes, for example, a speaker <NUM> inside a side surface of the main body. The autonomous mobile body <NUM> can output various sounds by the speaker <NUM>.

Furthermore, as illustrated in <FIG>, the autonomous mobile body <NUM> includes a microphone <NUM>, a microphone <NUM>, and a microphone 124R inside the upper portion of the main body. The microphone <NUM>, the microphone <NUM>, and the microphone 124R collect a user's utterance and an ambient environmental sound. Furthermore, since the autonomous mobile body <NUM> includes the plurality of microphones <NUM>, microphones <NUM>, and microphones 124R, it is possible to collect sounds generated in the surroundings with high sensitivity and detect a position of a sound source.

Furthermore, as illustrated in <FIG> and <FIG>, the autonomous mobile body <NUM> includes motors 125A to 125E (whereas the motor 125E is not illustrated). The motor 125A and the motor 125B drive, for example, the substrate on which the eye portion <NUM> and the camera <NUM> are arranged in a vertical direction and a horizontal direction. The motor 125C realizes a forward tilting posture of the autonomous mobile body <NUM>. The motor 125D drives the wheel <NUM>. The motor 125E drives the wheel 107R. The autonomous mobile body <NUM> can express rich operation by the motors 125A to 125E.

Note that, hereinafter, in a case where it is not necessary to individually distinguish the microphones <NUM> to 124R, they are simply referred to as a microphone <NUM>. Hereinafter, in a case where it is not necessary to distinguish the motors 125A to 125E individually, they are simply referred to as a motor <NUM>.

<FIG> illustrates a configuration example of functions of the autonomous mobile body <NUM>. The autonomous mobile body <NUM> includes a control unit <NUM>, a sensor unit <NUM>, an input unit <NUM>, a light source <NUM>, a sound output unit <NUM>, a drive unit <NUM>, and a communication unit <NUM>.

The control unit <NUM> has a function of controlling each configuration included in the autonomous mobile body <NUM>. The control unit <NUM> controls, for example, activation and stop of each configuration. Furthermore, the control unit <NUM> supplies a control signal and the like received from the information processing server <NUM> to the light source <NUM>, the sound output unit <NUM>, and the drive unit <NUM>.

The sensor unit <NUM> has a function of collecting various data regarding a user and a surrounding situation. For example, the sensor unit <NUM> includes the camera <NUM>, the ToF sensor <NUM>, the inertial sensor <NUM>, the microphone <NUM>, and the like described above. Furthermore, the sensor unit <NUM> may include, for example, various sensors such as a geomagnetic sensor, a touch sensor, various optical sensors including an infrared (IR) sensor and the like, a temperature sensor, a humidity sensor, and the like, in addition to the sensors described above. The sensor unit <NUM> supplies sensor data output from each sensor to the control unit <NUM>.

The input unit <NUM> includes, for example, a button, a switch, and the like such as the above-described power switch <NUM> and the like, and detects a physical input operation by a user.

The light source <NUM> includes, for example, the above-described eye portion <NUM> and the like, and expresses eyeball operation of the autonomous mobile body <NUM>.

The sound output unit <NUM> includes, for example, the above-described speaker <NUM>, an amplifier, and the like, and outputs an output sound on the basis of output sound data supplied from the control unit <NUM>.

The drive unit <NUM> includes, for example, the wheel <NUM>, the motor <NUM>, and the like described above, and is used for expressing body operation of the autonomous mobile body <NUM>.

The communication unit <NUM> includes, for example, the connection terminal <NUM>, the communication device <NUM>, and the like described above, and communicates with the information processing server <NUM>, the operated device <NUM>, and the other external device. For example, the communication unit <NUM> transmits the sensor data supplied from the sensor unit <NUM> to the information processing server <NUM>, and receives, from the information processing server <NUM>, the control signal for controlling operation of the autonomous mobile body <NUM> and the output sound data for outputting the output sound from the autonomous mobile body <NUM>.

<FIG> illustrates a configuration example of an information processing unit <NUM> realized by the control unit <NUM> of the autonomous mobile body <NUM> executing a predetermined control program.

The information processing unit <NUM> includes a recognition unit <NUM>, an action planning unit <NUM>, an operation control unit <NUM>, and a sound control unit <NUM>.

The recognition unit <NUM> has a function of recognizing a user and environment around the autonomous mobile body <NUM> and various information regarding the autonomous mobile body <NUM> on the basis of the sensor data supplied from the sensor unit <NUM>.

For example, the recognition unit <NUM> performs user identification, recognition of user's expression or line of sight, object recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, and the like. For example, the recognition unit <NUM> performs emotion recognition related to user's voice, word understanding, recognition of a position of a sound source, and the like. For example, the recognition unit <NUM> recognizes ambient temperature, presence of an animal body, posture and motion of the autonomous mobile body <NUM>, and the like.

Furthermore, the recognition unit <NUM> has a function of estimating and understanding an environment and a situation in which the autonomous mobile body <NUM> is placed on the basis of the recognized information. At this time, the recognition unit <NUM> may comprehensively perform situation estimation using environmental knowledge stored in advance.

The recognition unit <NUM> supplies data indicating a recognition result to the action planning unit <NUM>, the operation control unit <NUM>, and the sound control unit <NUM>.

The action planning unit <NUM> has a function of planning an action performed by the autonomous mobile body <NUM> on the basis of the recognition result by the recognition unit <NUM> and learning knowledge. The action planning unit <NUM> executes an action plan using, for example, a machine learning algorithm such as deep learning and the like. The action planning unit <NUM> supplies information indicating the action plan to the operation control unit <NUM> and the sound control unit <NUM>.

The operation control unit <NUM> performs operation control of the autonomous mobile body <NUM> by controlling the light source <NUM> and the drive unit <NUM> on the basis of the recognition result by the recognition unit <NUM> and the action plan by the action planning unit <NUM>. For example, the operation control unit <NUM> moves the autonomous mobile body <NUM> in a forward tilting posture, or performs a back-and-forth motion, a turning motion, a rotary motion, and the like. Furthermore, the operation control unit <NUM> causes the autonomous mobile body <NUM> to actively execute trigger operation that triggers communication between a user and the autonomous mobile body <NUM>. Furthermore, the operation control unit <NUM> supplies information regarding the operation performed by the autonomous mobile body <NUM> to the sound control unit <NUM>.

The sound control unit <NUM> performs output control of an output sound by controlling the sound output unit <NUM> on the basis of the recognition result by the recognition unit <NUM> and the action plan by the action planning unit <NUM>. For example, the sound control unit <NUM> determines an output sound to be output from the autonomous mobile body <NUM> on the basis of the recognition result by the recognition unit <NUM> and the action plan by the action planning unit <NUM>. Furthermore, the sound control unit <NUM> generates output sound data for outputting the determined output sound, and supplies the data to the sound output unit <NUM>. Furthermore, the sound control unit <NUM> supplies information regarding the output sound output from the autonomous mobile body <NUM> to the operation control unit <NUM>.

Next, a method for generating a synthesis sound in the sound control unit <NUM> will be described.

The sound control unit <NUM> generates an output sound including a synthesis sound using, for example, an FM sound source. At this time, the sound control unit <NUM> can variously express an impression and emotional connotation of the synthesis sound by dynamically and continuously changing various parameters related to synthesis of the FM sound source.

<FIG> is a diagram for explaining parameters related to the synthesis sound. <FIG> illustrates a relationship between a configuration included in a synthesizer that synthesizes the FM sound source and an output mode expressed by the synthesis sound by a change in a parameter related to each configuration.

For example, the sound control unit <NUM> can change a basic texture of sound by changing a parameter related to an oscillator. As an example, the sound control unit <NUM> can express a soft impression by using a sine wave as a waveform of the sound, and can express a sharp impression by using a sawtooth shape.

Furthermore, the sound control unit <NUM> can express a difference in gender, intonation, emotional ups and downs, and the like, for example, by controlling parameters of a pitch controller, that is, pitch.

<FIG> is a diagram illustrating an example of emotions that can be expressed by controlling pitch and speed of sound. Note that a size (an area) of a hatched region in <FIG> indicates volume. It is known that pitch and speed of sound strongly affect recall of an emotion expressed by the sound. The sound control unit <NUM> can express a degree of joy, anger, and the like by setting the pitch and the speed to be relatively high, for example. Conversely, the sound control unit <NUM> can express sorrow by setting the pitch and the speed to be relatively low. As described above, the sound control unit <NUM> can express various emotions and degrees thereof by controlling the pitch and the speed of the sound.

Returning to <FIG>, the sound control unit <NUM> can express clarity of sound (a way of opening a mouth) by controlling parameters of a filter. For example, the sound control unit <NUM> can express muffled sound or opened sound by increasing or decreasing frequency of a high cut filter.

Furthermore, the sound control unit <NUM> can change an accent of voice volume and an impression of a way of rising or ending by a temporal change of an amplifier.

Furthermore, the sound control unit <NUM> can express shivering of voice and smoothness by controlling parameters of a modulator.

As described above, the sound control unit <NUM> can express various impressions and emotional connotations by changing each parameter related to the oscillator, the modulator, the pitch controller, the filter, the amplifier, or the like.

<FIG> illustrates a functional configuration example of the information processing server <NUM>.

The information processing server <NUM> includes a communication unit <NUM>, a recognition unit <NUM>, an action planning unit <NUM>, an operation control unit <NUM>, and a sound control unit <NUM>.

The communication unit <NUM> communicates with the autonomous mobile body <NUM> and the operated device <NUM> via the network <NUM>. For example, the communication unit <NUM> receives sensor data from the autonomous mobile body <NUM>, and transmits, to the autonomous mobile body <NUM>, a control signal for controlling operation of the autonomous mobile body <NUM> and output sound data for outputting an output sound from the autonomous mobile body <NUM>.

The recognition unit <NUM>, the action planning unit <NUM>, the operation control unit <NUM>, and the sound control unit <NUM> have functions similar to those of the recognition unit <NUM>, the action planning unit <NUM>, the operation control unit <NUM>, and the sound control unit <NUM> of the autonomous mobile body <NUM>. That is, the recognition unit <NUM>, the action planning unit <NUM>, the operation control unit <NUM>, and the sound control unit <NUM> can perform various types of processing instead of the recognition unit <NUM>, the action planning unit <NUM>, the operation control unit <NUM>, and the sound control unit <NUM> of the autonomous mobile body <NUM>.

Therefore, the information processing server <NUM> can remotely control the autonomous mobile body <NUM>, and the autonomous mobile body <NUM> can perform various operation and output various output sounds under the control of the information processing server <NUM>.

Next, processing of the autonomous mobile body <NUM> will be described with reference to <FIG>.

Note that, hereinafter, an example of a case where the autonomous mobile body <NUM> independently performs various operation or outputs various output sounds without being controlled by the information processing server <NUM> will be described.

First, a basic example of operation sound output control processing executed by the autonomous mobile body <NUM> will be described with reference to a flowchart of <FIG>.

In step S1, the recognition unit <NUM> converts sensor data into an intermediate parameter.

For example, sensor data of an acceleration sensor included in the inertial sensor <NUM> includes a gravitational acceleration component. Therefore, in a case where an operation sound is output using the sensor data of the acceleration sensor as it is, the operation sound is always output even if the autonomous mobile body <NUM> is not moving.

Furthermore, since the sensor data of the acceleration sensor includes acceleration in three axial directions of an x-axis, a y-axis, and a z-axis, components corresponding to vibration and noise are also included in addition to a component corresponding to movement of the autonomous mobile body <NUM>. Therefore, in a case where the operation sound is output using the sensor data of the acceleration sensor as it is, the operation sound is output in response to the vibration and the noise in addition to the movement of the autonomous mobile body <NUM>.

On the other hand, the recognition unit <NUM> converts the sensor data of each sensor included in the sensor unit <NUM> into an intermediate parameter that corresponds to a motion of the autonomous mobile body <NUM> to be a target for outputting the operation sound and can be understood by a person.

Specifically, the recognition unit <NUM> acquires the sensor data from each sensor included in the sensor unit <NUM>, and performs arithmetic and logical operations such as filter processing, threshold processing, and the like on each sensor data, thereby converting each sensor data into a predetermined type of intermediate parameter.

<FIG> illustrates a specific example of a method of converting the sensor data into the intermediate parameter.

For example, the recognition unit <NUM> acquires sensor data indicating rotation speed of the motor 125D or the motor 125E of the autonomous mobile body <NUM> from a rotation sensor <NUM> included in the sensor unit <NUM>. The recognition unit <NUM> calculates a movement amount of the autonomous mobile body <NUM> by calculating odometry on the basis of the rotation speed of the motor 125D or the motor 125E. Furthermore, the recognition unit <NUM> calculates speed of the autonomous mobile body <NUM> in translation directions (front, rear, left, and right directions) on the basis of the movement amount of the autonomous mobile body <NUM>. Therefore, the sensor data is converted into the speed that is the intermediate parameter.

For example, the recognition unit <NUM> acquires, from an IR sensor <NUM> (not illustrated in <FIG>) included in the sensor unit <NUM> and provided on the bottom surface of the autonomous mobile body <NUM>, sensor data indicating whether or not an object (for example, a floor surface) approaches the bottom surface. Furthermore, the recognition unit <NUM> acquires sensor data indicating acceleration of the autonomous mobile body <NUM> from an acceleration sensor 121A included in the inertial sensor <NUM>. The recognition unit <NUM> detects whether or not the autonomous mobile body <NUM> is lifted on the basis of whether or not the object approaches the bottom surface of the autonomous mobile body <NUM> and the acceleration of the autonomous mobile body <NUM>. Therefore, the sensor data is converted into the presence or absence of lifting that is the intermediate parameter.

For example, the recognition unit <NUM> acquires the sensor data indicating the acceleration of the autonomous mobile body <NUM> from the acceleration sensor 121A. Furthermore, the recognition unit <NUM> acquires sensor data indicating angular velocity of the autonomous mobile body <NUM> from an angular velocity sensor 121B included in the inertial sensor <NUM>. The recognition unit <NUM> detects a motion amount after the autonomous mobile body <NUM> is lifted on the basis of the acceleration and the angular velocity of the autonomous mobile body <NUM>. This motion amount indicates, for example, an amount by which the autonomous mobile body <NUM> is shaken after being lifted. Therefore, the sensor data is converted into the motion amount after the autonomous mobile body <NUM> is lifted, which is the intermediate parameter.

For example, the recognition unit <NUM> acquires the sensor data indicating the angular velocity of the autonomous mobile body <NUM> from the angular velocity sensor 121B. The recognition unit <NUM> detects rotation in a yaw direction (lateral rotation) about an axis in a vertical direction of the autonomous mobile body on the basis of the angular velocity of the autonomous mobile body <NUM>. Therefore, the autonomous mobile body <NUM> whose sensor data is the intermediate parameter is converted into the lateral rotation.

In step S2, the sound control unit <NUM> generates an operation sound on the basis of the intermediate parameter.

For example, in a case where the speed of the autonomous mobile body <NUM> is a predetermined threshold value or more, the sound control unit <NUM> generates a translation sound which is an operation sound corresponding to the translation of the autonomous mobile body <NUM>. At this time, for example, the sound control unit <NUM> changes some of the parameters such as frequency, volume, a degree of modulation, and the like of the translation sound on the basis of an amount of change in the speed of the autonomous mobile body <NUM>.

For example, in a case where the autonomous mobile body <NUM> is lifted, the sound control unit <NUM> generates a lifting sound which is an operation sound corresponding to the lifting of the autonomous mobile body <NUM>. At this time, for example, the sound control unit <NUM> changes some of the parameters such as frequency, volume, a degree of modulation, and the like of the lifting sound on the basis of a change in the motion amount after the autonomous mobile body <NUM> is lifted.

For example, in a case where rotation speed of the autonomous mobile body <NUM> in the lateral rotation is equal to or more than a predetermined threshold, the sound control unit <NUM> generates a rotation sound which is an operation sound corresponding to the lateral rotation of the autonomous mobile body <NUM>. At this time, for example, the sound control unit <NUM> changes some of the parameters such as frequency, volume, a degree of modulation, and the like of the rotation sound on the basis of a change in the rotation speed of the autonomous mobile body <NUM> in the lateral direction.

In step S3, the autonomous mobile body <NUM> outputs an operation sound. Specifically, the sound control unit <NUM> generates output sound data for outputting the generated operation sound, and supplies the data to the sound output unit <NUM>. The sound output unit <NUM> outputs the operation sound on the basis of the acquired output sound data.

As described above, the operation sound is output in accordance with the motion of the autonomous mobile body <NUM>.

As described above, in a case where the autonomous mobile body <NUM> outputs different operation sounds for a plurality of motions, when the plurality of motions has been simultaneously recognized, the plurality of operation sounds is simultaneously output. In this case, since the plurality of operation sounds is simultaneously output, a user cannot recognize individual operation sounds, and there is a possibility that user experience is deteriorated.

Furthermore, for example, there is a case where a motion of the autonomous mobile body <NUM> recognized by the user is different from a motion of the autonomous mobile body <NUM> recognized by the recognition unit <NUM> of the autonomous mobile body <NUM>.

For example, even if the autonomous mobile body <NUM> appears to be moving straight from the user, there is a case where the recognition unit <NUM> of the autonomous mobile body <NUM> detects lateral rotation of the autonomous mobile body <NUM> on the basis of the sensor data of the angular velocity sensor 121B. In this case, if a rotation sound is output from the autonomous mobile body <NUM>, there is a possibility that the user feels uncomfortable.

Furthermore, for example, in a case where the user lifts and shakes the autonomous mobile body <NUM>, the recognition unit <NUM> of the autonomous mobile body <NUM> sometimes detects lateral rotation of the autonomous mobile body <NUM> on the basis of the sensor data of the angular velocity sensor 121B. In this case, if a rotation sound is output from the autonomous mobile body <NUM>, there is a possibility that the user feels uncomfortable.

On the other hand, the autonomous mobile body <NUM> controls output of the operation sound on the basis of priority as follows, thereby outputting a natural operation sound without giving a sense of discomfort to the user.

Here, a modified example of the operation sound output control processing executed by the autonomous mobile body <NUM> will be described with reference to a flowchart of <FIG>.

For example, this processing is started when power of the autonomous mobile body <NUM> is turned on and is ended when the power is turned off.

In step S51, the recognition unit <NUM> determines whether or not a motion of the autonomous mobile body <NUM> has been recognized. This determination processing is repeatedly executed until it is determined that the recognition unit <NUM> has recognized the motion of the autonomous mobile body <NUM> to be a target for outputting an operation sound on the basis of sensor data from the sensor unit <NUM>. Then, in a case where it is determined that the motion of the autonomous mobile body <NUM> to be the target for outputting the operation sound has been recognized, the processing proceeds to step S52.

In step S52, the recognition unit <NUM> recognizes a surrounding situation on the basis of the sensor data from the sensor unit <NUM>.

For example, the recognition unit <NUM> recognizes presence or absence of a user around the autonomous mobile body <NUM>, identification of the user, a state of the user, and the like. At this time, for example, a target user may be limited to a specific person such as an owner of the autonomous mobile body <NUM> and the like, or not limited to a specific person, and all the people around the autonomous mobile body <NUM> may be regarded as the users.

Furthermore, for example, the recognition unit <NUM> recognizes presence or absence, a type, a position, a distance, a feature, a state, and the like of an object around the autonomous mobile body <NUM>. The object around the autonomous mobile body <NUM> includes, for example, another autonomous mobile body.

Note that the distance between the autonomous mobile body <NUM> and the surrounding object is detected on the basis of, for example, sensor data from at least one of the camera <NUM> or the ToF sensor <NUM>.

Alternatively, for example, the communication unit <NUM> of the autonomous mobile body <NUM> may detect the distance to the surrounding object by performing short-range wireless communication with the surrounding object.

Alternatively, for example, the communication unit <NUM> of the autonomous mobile body <NUM> may receive information indicating a current position of the surrounding object from the information processing server <NUM> and the like, and the recognition unit <NUM> may detect the distance to the surrounding object on the basis of a current position of the autonomous mobile body <NUM> and the current position of the surrounding object.

Moreover, for example, the recognition unit <NUM> recognizes volume, a feature, a content, a position of a sound source, and the like of a sound around the autonomous mobile body <NUM> on the basis of sensor data from the microphone <NUM>.

The recognition unit <NUM> supplies information indicating a recognition result of the motion of the autonomous mobile body <NUM> and the surrounding situation to the sound control unit <NUM>.

In step S53, the sound control unit <NUM> determines whether or not the surroundings are noisy. For example, in a case where a level of a recognized surrounding sound is a predetermined threshold or more, the sound control unit <NUM> determines that the surroundings are noisy. The processing returns to step S51, and the processing in step S51 and subsequent steps is executed.

That is, in a case where the surroundings are noisy, the operation sound is difficult to hear and there is a possibility that it disturbs the surrounding sound, so that the operation sound is not output.

On the other hand, in step S53, in a case where the level of the recognized surrounding sound is less than the predetermined threshold, the sound control unit <NUM> determines that the surroundings are not noisy, and the processing proceeds to step S54.

In step S54, the sound control unit <NUM> determines whether or not a user's utterance has been recognized. In a case where it is determined that the user's utterance has not been recognized, the processing proceeds to step S55.

In step S55, the sound control unit <NUM> determines whether or not a user's face has been detected. In a case where it is determined that the user's face has not been detected, the processing proceeds to step S55.

In step S56, the sound control unit <NUM> determines whether or not a trigger for a start of a user's utterance has been recognized. As the trigger for the start of the user's utterance, for example, a gesture by which a user tries to talk to the autonomous mobile body <NUM>, an operation in which the user pushes the autonomous mobile body <NUM> or touches the autonomous mobile body <NUM>, and the like are assumed. In a case where it is determined that the trigger for the start of the user's utterance has been recognized, the processing proceeds to step S57.

On the other hand, in a case where it is determined in step S55 that the user's face has been detected, the processing in step S56 is skipped, and the processing proceeds to step S57.

Furthermore, in a case where it is determined in step S54 that the user's utterance has been recognized, the processing in steps S55 and S56 is skipped, and the processing proceeds to step S57.

In other words, in a case where the user talks to or tries to talk to the autonomous mobile body <NUM>, the processing proceeds to step S57.

In step S57, the sound control unit <NUM> determines whether or not an operation sound having a higher priority than the user's utterance is an output target.

For example, in the autonomous mobile body <NUM>, priority is set in advance for each output sound such as an operation sound and the like. For example, with respect to the output sound of the autonomous mobile body <NUM>, priority is set such that lifting sound > utterance sound (of the autonomous mobile body <NUM>) > translation sound > rotation sound. Note that an output sound on a left side of an inequality sign has a higher priority than an output sound on a right side. Then, an output sound having a higher priority is preferentially output.

Furthermore, for example, with respect to the user's utterance, priority of each output sound is set such that lifting sound > user's utterance > utterance sound (of the autonomous mobile body <NUM>) > translation sound > rotation sound. In other words, the lifting sound has a higher priority than the user's utterance, and the utterance sound, the translation sound, and the rotation sound have a lower priority than the user's utterance.

Therefore, in a case of the priority described above, in a case where lifting of the autonomous mobile body <NUM> is not recognized, the sound control unit <NUM> determines that the operation sound (lifting sound) having the higher priority than the user's utterance is not the output target. The processing returns to step S51, and the processing in step S51 and subsequent steps is executed. That is, in this case, the user's utterance is prioritized, and the operation sound is not output.

On the other hand, in step S57, for example, in a case where lifting of the autonomous mobile body <NUM> is detected, the sound control unit <NUM> determines that the operation sound having the higher priority than the user's utterance is the output target, and the processing proceeds to step S58.

Furthermore, in a case where it is determined in step S56 that the trigger for the start of the user's utterance has not been recognized, the processing in step S57 is skipped, and the processing proceeds to step S58. That is, in a case where the user does not talk to or does not try to talk to the autonomous mobile body <NUM>, the processing of step S57 is skipped, and the processing proceeds to step S58.

In step S58, the sound control unit <NUM> determines whether or not the autonomous mobile body <NUM> is uttering. In a case where the autonomous mobile body <NUM> is outputting an utterance sound, the sound control unit <NUM> determines that the autonomous mobile body <NUM> is uttering, and the processing proceeds to step S59.

In step S59, the sound control unit <NUM> determines whether or not an operation sound having a higher priority than the utterance of the autonomous mobile body <NUM> is an output target. For example, in a case where the above-described priority is set, in a case where lifting of the autonomous mobile body <NUM> is not detected, the sound control unit <NUM> determines that the operation sound (lifting sound) having the higher priority than the utterance of the autonomous mobile body <NUM> is not the output target. The processing returns to step S51, and the processing in and after step S51 is executed. That is, in this case, the utterance of the autonomous mobile body <NUM> is prioritized, and the operation sound is not output.

On the other hand, in step S59, for example, in a case where lifting of the autonomous mobile body <NUM> is detected, the sound control unit <NUM> determines that the operation sound having the higher priority than the utterance of the autonomous mobile body <NUM> is the output target, and the processing proceeds to step S60.

Furthermore, in a case where it is determined in step S58 that the autonomous mobile body <NUM> is not uttering, the processing proceeds to step S60.

In step S60, the sound control unit <NUM> determines whether or not there is a plurality of operation sounds to be output. In a case where a plurality of motions of the autonomous mobile body <NUM> to be a target for outputting the operation sound is detected, the sound control unit <NUM> determines that there is a plurality of operation sounds to be output, and the processing proceeds to step S61.

In step S61, the sound control unit <NUM> selects an operation sound to be output. For example, the sound control unit <NUM> selects an operation sound with the highest priority among the operation sounds to be output.

Thereafter, the processing proceeds to step S62.

On the other hand, in step S60, in a case where only one motion of the autonomous mobile body <NUM> to be the target for outputting the operation sound is detected, the sound control unit <NUM> determines that there is one operation sound to be output. The processing in step S61 is skipped, and the processing proceeds to step S62.

In step S62, the autonomous mobile body <NUM> outputs an operation sound. Specifically, in a case where there is a plurality of operation sounds to be output, the sound control unit <NUM> generates output sound data for outputting the operation sound selected in the processing of step S60. On the other hand, in a case where there is only one operation sound to be output, the sound control unit <NUM> generates output sound data for outputting the operation sound. The sound control unit <NUM> supplies the generated output sound data to the sound output unit <NUM>.

The sound output unit <NUM> outputs the operation sound on the basis of the acquired output sound data.

Thereafter, the processing returns to step S51, and the processing in and after step S51 is executed.

As described above, the operation sound with the high priority is preferentially output, and user experience is improved.

For example, after the autonomous mobile body <NUM> is lifted, the operation sound other than the lifting sound is not output. For example, in a case where the autonomous mobile body <NUM> moves in one of front, rear, left, and right directions, the rotation sound is not output. As a result, an operation sound that corresponds to a motion of the autonomous mobile body <NUM> recognized by the user and is natural and comfortable for the user is output.

Furthermore, in a case where the user talks or tries to talk, the operation sound other than the lifting sound and the utterance sound are not output. Therefore, it is possible to give the user an impression that the autonomous mobile body <NUM> listens to a user's utterance content without blocking the user's utterance.

On the other hand, even in a case where the user is talking, the lifting sound is output as it is without being changed. Therefore, for example, in a case where the user lifts the autonomous mobile body <NUM> while talking to it or talks to the autonomous mobile body <NUM> after lifting it, the autonomous mobile body <NUM> can express surprise caused by being lifted by the user by the lifting sound.

Note that, in the above description, an example has been described in which only the operation sound with the highest priority is output in a case where the plurality of motions has been detected. However, for example, output of another operation sound may be suppressed by lowering volume of the other operation sound with a low priority.

Furthermore, the operation sound with the highest priority may be changed on the basis of another motion other than the motion corresponding to the operation sound with the highest priority.

For example, an operation sound obtained by modulating the operation sound with the highest priority on the basis of magnitude, speed, and the like of another motion may be generated and output. For example, in a case where translation and lateral rotation of the autonomous mobile body <NUM> have been detected, frequency of a translation sound may be changed on the basis of magnitude, speed, and the like of the lateral rotation, or an overtone component of the translation sound may be increased to make a muddy sound.

Moreover, for example, an operation sound may be generated and output on the basis of an algorithm in which an algorithm for generating the operation sound with the highest priority is changed on the basis of magnitude, speed, and the like of another motion.

Furthermore, for example, the algorithm for generating the operation sound may be changed depending on a case where the autonomous mobile body <NUM> is lifted or not.

Next, processing in which the autonomous mobile body <NUM> changes an output sound according to a user's utterance or another autonomous mobile body will be described.

Note that a type of the other autonomous mobile body is not limited as long as the other autonomous mobile body is a mobile body that autonomously operates similarly to the autonomous mobile body <NUM> and can output an utterance sound in order to communicate with a user or another autonomous mobile body. For example, the other autonomous mobile body may be the same type of mobile body as the autonomous mobile body <NUM>, or may be a different type of mobile body.

Here, the output sound changing processing executed by the autonomous mobile body <NUM> will be described with reference to a flowchart of <FIG>.

In step S101, a surrounding situation is recognized, similarly to the processing in step S52 in <FIG>.

In step S102, similarly to the processing in step S54 in <FIG>, it is determined whether or not a user's utterance has been recognized. In a case where it is determined that the user's utterance has not been recognized, the processing proceeds to step S103.

In step S103, it is determined whether or not a user's face has been detected, similarly to the processing in step S55 in <FIG>. In a case where it is determined that the user's face has not been detected, the processing proceeds to step S104.

In step S104, similarly to the processing in step S56 in <FIG>, it is determined whether or not a trigger for a start of a user's utterance has been recognized. In a case where it is determined that the trigger for the start of the user's utterance has been recognized, the processing proceeds to step S105.

On the other hand, in a case where it is determined in step S103 that the user's face has been detected, the processing in step S104 is skipped, and the processing proceeds to step S105.

Furthermore, in a case where it is determined in step S102 that the user's utterance has been recognized, the processing in steps S103 and S104 is skipped, and the processing proceeds to step S105.

In step S105, the sound control unit <NUM> changes an output sound in accordance with the user's utterance.

For example, the sound control unit <NUM> changes volume, intensity, frequency, tone, sound quality, and the like of the output sound (operation sound or utterance sound) in order to prevent the user's utterance from being disturbed or give the user an impression that the autonomous mobile body <NUM> listens to the user's utterance.

For example, the sound control unit <NUM> reduces the volume of the output sound or stops the output sound.

For example, the sound control unit <NUM> reduces a component of a frequency band of human voice (for example, <NUM> to <NUM>) among frequency components of the output sound. Specifically, for example, the sound control unit <NUM> shifts a frequency band of the output sound to a band not including the frequency band of the human voice, or reduces a component included in the frequency band of the human voice among the frequency components of the output sound.

Thereafter, the processing proceeds to step S105.

On the other hand, in a case where it is determined in step S104 that the trigger for the start of the user's utterance has not been recognized, the processing in step S105 is skipped, and the processing proceeds to step S106.

In step S106, the sound control unit <NUM> determines whether or not another autonomous mobile body has been detected. In a case where it is determined that the other autonomous mobile body has been detected, the processing proceeds to step S107.

In step S107, the sound control unit <NUM> changes the output sound in accordance with the other autonomous mobile body.

For example, the sound control unit <NUM> changes the output sound (operation sound or utterance sound) output from the autonomous mobile body <NUM> so that an impression that the autonomous mobile body <NUM> is conscious of the other autonomous mobile body can be given to the user.

For example, the sound control unit <NUM> outputs the output sound so as to give an impression that the autonomous mobile body <NUM> talks (performs communication by sound) with the other autonomous mobile body. For example, when the other autonomous mobile body outputs an utterance sound, the sound control unit <NUM> suppresses the output sound. That is, the sound control unit <NUM> reduces volume of the utterance sound or the operation sound, or stops the utterance sound or the operation sound. On the other hand, for example, when the other autonomous mobile body does not output the utterance sound, the sound control unit <NUM> outputs the utterance sound or the operation sound without suppressing it. Therefore, for example, the autonomous mobile body <NUM> and the other autonomous mobile body alternately output the utterance sounds, and it is possible to give the user an impression that both of them are talking.

For example, the sound control unit <NUM> performs control to output the output sound in synchronization with the other autonomous mobile body. For example, the sound control unit <NUM> performs control so as to output an output sound having a rhythm or a tempo matched with an output sound of the other autonomous mobile body, or an output sound including a sound in harmony with an output sound of the other autonomous mobile body.

For example, the sound control unit <NUM> controls to output an utterance sound similar to an utterance sound of the other autonomous mobile body by bringing the utterance sound close to the utterance sound of the other autonomous mobile body. For example, the sound control unit <NUM> controls to output the utterance sound that mimics the utterance sound output by the other autonomous mobile body. Specifically, for example, in a case where the other autonomous mobile body is a dog-shaped robot, the sound control unit <NUM> performs control to output an utterance sound close to dog barking. Furthermore, for example, the sound control unit <NUM> brings an algorithm for outputting the utterance sound closer to an algorithm for outputting the utterance sound by the other mobile body. Moreover, for example, the sound control unit <NUM> recognizes pitch of the utterance sound of the other autonomous mobile body, and performs control so as to output an utterance sound that changes in a manner similar to the recognized pitch.

Note that information regarding the algorithm of the other autonomous mobile body is acquired, for example, by the communication unit <NUM> of the autonomous mobile body <NUM> directly communicating with the other autonomous mobile body. Alternatively, the communication unit <NUM> may acquire the information from the information processing server <NUM> and the like.

Thereafter, the processing returns to step S101, and the processing in and after step S101 is executed.

On the other hand, in a case where it is determined in step S106 that the other autonomous mobile body has not been detected, the processing returns to step S101, and the processing in and after step S101 is executed.

As described above, user experience is improved by changing the output sound according to the user's utterance and the other autonomous mobile body. For example, communication between the user and the autonomous mobile body <NUM> becomes smooth. Furthermore, for example, communication between the autonomous mobile body <NUM> and the other autonomous mobile body is realized, and the user can be entertained.

Hereinafter, modified examples of the above-described embodiment of the present technology will be described.

The type and priority of the operation sound described above are examples, and can be changed. For example, it is possible to increase or decrease the type of the operation sound.

Furthermore, the output sound may be changed using conditions other than those described above.

For example, the sound control unit <NUM> of the autonomous mobile body <NUM> may change the output sound on the basis of presence/absence of or a distance to a surrounding person, or presence/absence of, a distance to, or a type of a surrounding object, and the like.

For example, the sound control unit <NUM> may change the output sound as a distance to a person decreases.

For example, the sound control unit <NUM> may change the output sound by different methods in a case where a distance to a person is decreased by movement of the person, in a case where it is increased by movement of the person, in a case where it is decreased by movement of the autonomous mobile body <NUM>, and in a case where it is increased by movement of the autonomous mobile body <NUM>.

For example, the sound control unit <NUM> may change the output sound in a case where an object or another autonomous mobile body is recognized in the surroundings.

For example, in a case where the sound control unit <NUM> recognizes a person who has not been recognized in the past (for example, a suspicious person), the output sound may be changed. For example, the sound control unit <NUM> may increase volume of a translation sound or a rotation sound. For example, the sound control unit <NUM> may increase an overtone component of the translation sound or the rotation sound or increase intensity of modulation of a synthesis sound constituting the translation sound or the rotation sound to make a sharp sound.

For example, the sound control unit <NUM> may change the output sound in a case of approaching an unmovable place such as a wall and the like.

For example, the sound control unit <NUM> may change the output sound when approaching a specific object. As the specific object, for example, an autonomous mobile body of the same type, an autonomous mobile body of a different type, a specific device, and the like are assumed. In this case, the sound control unit <NUM> may change the output sound by different methods depending on whether the approaching object is the autonomous mobile body of the same type, the autonomous mobile body of the different type, or the specific device.

Furthermore, as described above, the information processing server <NUM> can receive the sensor data from the autonomous mobile body <NUM> and control the operation and the output sound of the autonomous mobile body <NUM> on the basis of the received sensor data.

Moreover, in a case where the information processing server <NUM> controls the output sound of the autonomous mobile body <NUM>, the information processing server <NUM> may generate the output sound, or the autonomous mobile body <NUM> may generate the output sound under the control of the information processing server <NUM>.

The series of processing described above can be executed by hardware or software. In a case where the series of processing is executed by the software, a program constituting the software is installed on a computer. Here, the computer includes a computer incorporated in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like, for example.

<FIG> is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.

In a computer <NUM>, a central processing unit (CPU) <NUM>, a read only memory (ROM) <NUM>, and a random access memory (RAM) <NUM> are mutually connected by a bus <NUM>.

Moreover, an input/output interface <NUM> is connected to the bus <NUM>. An input unit <NUM>, an output unit <NUM>, a recording unit <NUM>, a communication unit <NUM>, and a drive <NUM> are connected to the input/output interface <NUM>.

The input unit <NUM> includes an input switch, a button, a microphone, an imaging element, and the like. The output unit <NUM> includes a display, a speaker, and the like. The recording unit <NUM> includes a hard disk, a nonvolatile memory, and the like. The communication unit <NUM> includes a network interface and the like. The drive <NUM> drives a removable medium <NUM> such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer <NUM> configured as described above, for example, the CPU <NUM> loads a program stored in the recording unit <NUM> into the RAM <NUM> via the input/output interface <NUM> and the bus <NUM> and executes the program, whereby the above-described series of processing is performed.

The program executed by the computer <NUM> (CPU <NUM>) can be provided by recording on the removable medium <NUM> as a package medium and the like, for example. Furthermore, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

In the computer <NUM>, the program can be installed in the recording unit <NUM> via the input/output interface <NUM> by attaching the removable medium <NUM> to the drive <NUM>. Furthermore, the program can be received by the communication unit <NUM> via a wired or wireless transmission medium and installed in the recording unit <NUM>. In addition, the program can be installed in the ROM <NUM> or the recording unit <NUM> in advance.

Note that the program executed by the computer may be a program in which processing is performed in time series in the order described in the present specification, or may be a program in which processing is performed in parallel or at necessary timing such as when a call is made, and the like.

Furthermore, in the present specification, the system means a set of a plurality of components (devices, modules (parts), and the like), and it does not matter whether or not all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and one device housing a plurality of modules in one housing are both systems.

Moreover, an embodiment of the present technology is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the present technology.

For example, the present technology can be configured as cloud computing in which one function is shared and jointly processed by a plurality of devices via a network.

Furthermore, each step described in the above-described flowcharts can be executed by one device or shared and executed by a plurality of devices.

Moreover, in a case where one step includes a plurality of processing, the plurality of processing included in the one step can be executed by one device or shared and executed by a plurality of devices.

Claim 1:
An autonomous mobile body (<NUM>) in the form of a robot comprising:
a recognition unit (<NUM>) that recognizes a motion of its own device; and
a sound control unit (<NUM>) that controls an output sound output from the own device,
wherein
the sound control unit controls output of a plurality of operation sounds that is the output sound corresponding to a plurality of the motions of the own device, and changes the operation sound in a case where the plurality of motions has been recognized.