Patent ID: 12214492

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, (a) preferred embodiment(s) of the present disclosure will be described in detail with reference to the appended drawings. Note that, in this specification and the appended drawings, structural elements that have substantially the same function and structure are denoted with the same reference numerals, and repeated explanation of these structural elements is omitted.

In addition, description will be provided in the following order.1. Overview of information processing system according to one embodiment of present disclosure2. Configuration3. Operation processing3-1. Generation processing3-2. Notification processing4. Application examples4-1. Notification processing through use of radio waves4-2. Notification processing through use of smell4-3. Operation processing in accordance with emotion5. Conclusion

1. OVERVIEW OF INFORMATION PROCESSING SYSTEM ACCORDING TO ONE EMBODIMENT OF PRESENT DISCLOSURE

First, an overview of an information processing system according to one embodiment of the present disclosure will be described.FIG.1is a diagram describing an overview of the information processing system according to the present embodiment. As shown inFIG.1, the information processing system according to the present embodiment is implemented by a pet robot1(information processing device) having an entertainment nature, whose appearance has a shape formed simulating an animal such as a dog. In addition, the pet robot1is capable of autonomously moving eyes, legs, and the like to exhibit animal-like behaviors.

BACKGROUND

Here, in the daily life, one may have to answer a suddenly coming visitor in a hurry without being ready. In a general detached house, it is relatively possible to sense ambient sounds and feels even in the house, but as is often the case, although being careful about the sound of a motorbike thinking about when pizza delivery comes, for example, one may be late in noticing the sound while being distracted with another thing unintentionally. In addition, even in a condominium, it is possible to open the window to listen to an external downstairs sound and sense a feel, but it is inconvenient to leave the window open all the time.

Thus, the information processing system according to the present embodiment is capable of sensing approach of an object early and notifying a user using the pet robot1having various sensors and having an excellent sensory organ. In general, the human audible range is approximately 12 Hz to 23 KHz, while the audible range of dogs, for example, is approximately 15 Hz to 60 KHz. High-frequency acoustic waves are significantly produced by hitting sounds (stones, paving stones, touching sounds of gatepost metal) or the like in the case where an object is rubbed (rustling, footsteps, a rubbing sound when opening the gate). Therefore, by providing the pet robot1with a frequency sensor to sense these sounds, it is possible to sense approach of a person or an object in the vicinity and to notify the user in advance.

This allows the user to become aware of the visit early and make preparations. In addition, in the case where an intruder or a suspicious person at the back door or in the garden is sensed early, the user can take appropriate measures immediately to ensure security and safety. In addition, if it is clarified who the visitor is, it is possible to make a choice of not answering the door in some cases.

As shown inFIG.1, for example, the pet robot1present in a house2plays a watchdog-like role of sensing a feel of a person or an object in the vicinity including the outside of the house2mainly with a passive ultrasonic sensor and notifying the user. In the example shown inFIG.1, the pet robot1senses a sound of a motorbike3approaching the house2with the ultrasonic sensor, and notifies the user. Specifically, in the case of pizza delivery, for example, the doorbell is pressed after a sound that the motorbike3comes running, a sound of stopping in front of the house2, a sound of taking out pizza from a carrier, a sound of preparing for a slip, footsteps headed for the entrance door, a sound of walking on gravel or a stone pavement of the approach, and a sound of standing in front of the entrance door are heard, and the user becomes aware that the pizza delivery has come. The pet robot1can sense a series of these sounds (mainly, ultrasound) at an early stage, and capture sound frequency features and a sound chronological pattern to notify the user that “A pizza delivery person is coming soon” before the pizza delivery person reaches the entrance door. The user having received the notification can make preparations in advance, such as a preparation of a charge for pizza.

An overview of the information processing system according to the present embodiment has been described above. Note that, in the present embodiment, the pet robot1is used as an example of the information processing device, whilst the present embodiment is not limited to this, but may be an imaginary entity displayed on a display terminal such as a tablet terminal or a smartphone, for example, that is, a tiny robot based on a software program.

Next, a configuration of the pet robot1which is an example of the information processing device according to the present embodiment will be specifically described with reference toFIG.2.

2. CONFIGURATION

FIG.2is a block diagram showing an example of a configuration of the pet robot1according to the present embodiment. As shown inFIG.2, the pet robot1has a control section10, a communication section11, a camera15, a sound input section16, an ultrasonic sensor17, a driving section18, a touch sensor19, a storage section20, a sound output section22, an acceleration sensor23, and an angular velocity sensor24.

The control section10functions as an arithmetic processing device and a control device, and controls the overall operations in the pet robot1in accordance with various programs. The control section10is implemented by, for example, an electronic circuit such as a central processing unit (CPU) or a microprocessor. In addition, the control section10may include a read only memory (ROM) that stores programs, operation parameters, and the like to be used and a random access memory (RAM) that temporarily stores parameters and the like varying as appropriate.

In addition, the control section10according to the present embodiment may perform autonomous control of automatically operating in accordance with various types of information obtained from the communication section11, the camera15, the sound input section16, the ultrasonic sensor17, the touch sensor19, the acceleration sensor23, the angular velocity sensor24, or the like.

The communication section11is a communication module for transmitting/receiving data to/from another device. For example, the communication section11is connected to various types of servers via a network (not shown) to transmit/receive data, or is directly connected to a peripheral device (not shown) by Bluetooth (registered trademark) or Wi-Fi (registered trademark) to transmit/receive data.

The camera15has a lens system including an imaging lens, an aperture, a zoom lens, a focus lens, and the like, a driving system that causes the lens system to perform a focusing operation and a zooming operation, a solid-state image sensor array that photoelectrically converts imaged light obtained by the lens system to generate an imaging signal, and the like. The solid-state image sensor array may be implemented by, for example, a charge coupled device (CCD) sensor array or a complementary metal oxide semiconductor (CMOS) sensor array.

The sound input section16collects a user's voice or an ambient environmental sound, and outputs a sound signal to the control section10. The sound input section16is implemented by a microphone, a microphone amplifier section that subjects a sound signal obtained with the microphone to amplification processing, and an A/D converter for subjecting the sound signal to digital conversion, and outputs the sound signal to the control section10.

The ultrasonic sensor17is a passive sensor that receives ambient ultrasound, and outputs a sensed ultrasound signal to the control section10.

The driving section18is a functional module for achieving the degree of freedom at each joint of the pet robot1, and includes a plurality of driving units provided respectively for axes such as roll, pitch, and yaw at each joint. Each of the driving units includes a combination of a motor that performs a rotation operation about a predetermined axis, an encoder that detects a rotated position of the motor, and a driver that adaptively controls the rotated position and rotational speed of the motor on the basis of the output of the encoder.

The touch sensor19detects a pressure applied by a physical action from the user, such as “patting” or “clapping”. Note that the pet robot1may have a pressure sensor instead of or in addition to the touch sensor19.

The storage section20stores programs for the control section10to execute various types of processing, and the like. In addition, the storage section20includes a storage device including a storage medium, a recording device that records data on the storage medium, a readout device that reads out data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like.

The sound output section22is implemented by a speaker and an amplifier circuit for that speaker. In addition, the sound output section22outputs a sound such as a cry.

The acceleration sensor23and the angular velocity sensor24detect the orientation and motion acceleration of the pet robot1.

A configuration of the pet robot1according to the present embodiment has been specifically described above. Note that the configuration of the pet robot1according to the present embodiment is not limited to the example shown inFIG.2, but may further have a positional information acquisition section that measures the position of the pet robot1, for example.

In addition, the pet robot1may use the camera15described above as a distance sensor for measuring the distance to an object positioned ahead, or may separately include a distance sensor through use of a system of infrared rays or the like.

In addition, the pet robot1may include a body unit200, leg units201respectively coupled to the front, back, left, and right of the body unit200, and a head unit202and a tail unit203respectively coupled to the front end and the back end of the body unit200, as shown inFIG.1, for example. In the body unit200, the control section10formed by connecting a central processing unit (CPU), a dynamic random access memory (DRAM), a flash read only memory (ROM), a personal computer (PC) card interface circuit, and a signal processing circuit to one another via an internal bus and a battery as a power source of the pet robot1are stored. In addition, in the body unit200, the communication section11, the ultrasonic sensor17, the storage section20, the acceleration sensor23, the angular velocity sensor24, and the like are also stored.

In addition, in the head unit202, the camera15for imaging an external situation, the touch sensor19for detecting a pressure applied by a physical action from the user, such as “patting” or “clapping”, the sound input section16for collecting external sounds, the sound output section22for outputting a sound such as a cry, a distance sensor (not shown) for measuring the distance to an object positioned ahead, and the like are arranged at predetermined positions, respectively. The camera15may be provided at a position equivalent to an “eye” of the pet robot1. In addition, the camera15may be arranged in the forehead portion of the head unit202, and a light emitting diode (LED) (not shown) may be arranged at a position equivalent to an “eye”. In addition, an ear unit204is coupled to the head unit202, and is capable of moving in the case where approach of a person or an object is sensed by the ultrasonic sensor17.

Further, actuators and potentiometers of the number of degrees of freedom are placed at a joint portion of each of the leg units201, each coupled portion between each of the leg units201and the body unit200, a coupled portion between the head unit202and the body unit200, a tail coupled portion of the tail unit203, and the like, respectively. For example, the actuator has a servo motor as a component. The leg units201are controlled by driving of the servo motor to transition to a target attitude or operation.

Regarding a specific configuration example of the pet robot1described above, a reference is made to JP 2002-157596A, for example. The entire contents of JP 2002-157596A are hereby incorporated by reference.

In addition, the configuration of the pet robot1described with reference toFIG.2is also applicable to the case in which the pet robot1is a tiny robot. Specifically, a display terminal displaying the tiny robot has components equivalent to a display section, that is, the above-described control section10, the communication section11, the camera15, the sound input section16, the ultrasonic sensor17, the touch sensor19, the storage section20, and the sound output section22. The tiny robot is displayed on the display section, and may interact with the user.

(Functional Configuration)

Next, a functional configuration of the control section10and the storage section20will be described with reference toFIG.3.FIG.3is a block diagram showing a functional configuration example of the control section10and the storage section20according to one embodiment of the present disclosure. In the drawing, a feature quantity analysis section101, a storage control section102, a learning section103, a discrimination section104, and a notification control section105are shown as functions of the control section10of the pet robot1. In addition, in the drawing, a feature quantity database210and a discrimination data storage section220are shown as functions of the storage section20. Hereinafter, the respective structural elements will be described further.

The feature quantity analysis section101continually analyzes signals (ultrasonic signals) sensed and input by the ultrasonic sensor17, and outputs an analysis result (also referred to as feature quantity data) to the storage control section102and the discrimination section104. The method of analyzing signals is not particularly limited, but Fast Fourier Transform (FFT) analysis or cepstrum analysis in several milliseconds to several tens of milliseconds, for example, may be used.

The storage control section102exerts control so as to store the ultrasonic feature quantity data analyzed by the feature quantity analysis section101in the feature quantity database210. Since analyses are conducted continually in the feature quantity analysis section101, and feature quantity data is output as an analysis result, the storage control section102exerts control so as to continually store the feature quantity data in the feature quantity database210.

The learning section103machine-learns feature quantity data indicating approach of an object such as a visitor or a suspicious person, or a moving body (for example, a vehicle, motorbike, bicycle, drone) on the basis of chronological feature quantity data stored in the feature quantity database210, and generates discrimination data for discriminating among these objects. Specifically, in the case where a command (voice command) that “It was a courier delivery. Tell me if he comes next” is input from a user, the learning section103, for example, calls feature quantity data since a time point going back a predetermined time T (for example, 30 seconds) from the feature quantity database210, and generates discrimination data that senses that the courier delivery has come. Such discrimination data may be chronological feature quantity data having been patterned. The learning section103stores the generated discrimination data in the discrimination data storage section220as teacher data. Here, generation of discrimination data based on chronological feature quantity data will be described with reference toFIG.4.

FIG.4is a diagram showing an example of chronological feature quantity data according to the present embodiment. InFIG.4, feature quantity data F1, F2, F3, F4, and F5since a time point going back a predetermined time T1, read out from the feature quantity database210upon receipt of a command from the user, are shown in chronological order. In the feature quantity database210, a cart sound when a courier delivery person comes, a sound of a gate handle, a sound of opening the gate, a sound of stepping on a stone pavement, and a sound of standing in front of the entrance door are stored as chronological feature quantity data. The learning section103generates discrimination data for discriminating the courier delivery on the basis of the feature quantity data F1, F2, and F3since a time point going back the predetermined time T, for example. Accordingly, it may be discriminated by the discrimination section104which will be described later that a courier delivery person has come at the time point when the cart sound (feature quantity data F1), the sound of the gate handle (feature quantity data F2), and the sound of opening the gate (feature quantity data F3) occur successively.

Note that what event each piece of feature quantity data indicates may be classified by analyzing feature quantity data accumulated in the feature quantity database210or discrimination data accumulated in the discrimination data storage section220, or matching with sound event data acquired from a network. For example, the learning section103can classify the feature quantity data F1as the cart sound, the feature quantity data F2as the sound of the gate handle, the feature quantity data F3as the sound of opening the gate, the feature quantity data F4as the sound of stepping on the stone pavement, the feature quantity data F5as the sound of standing in front of the entrance door, and the like in the example shown inFIG.4.

In the case where events can be classified, the control section10is capable of grasping a state transition of a sound event on the basis of accumulated feature quantity data and discrimination data. Here,FIG.5shows an example of an event state transition according to the present embodiment. In the case where a state transition as shown inFIG.5has been grasped, it is determined that an approaching visitor, moving body, or the like is highly likely to be an object A when a sound event transitions in the order of F01, F02, F03, and F04, and that an object B is highly likely to be approaching when the sound event transitions in the order of F01, F11, F12, and F13. In addition, it is determined that an object C is highly likely to be approaching when the sound event transitions in the order of F01, F11, F21, and F22. Note that, in the case where each event is calculated as one point, for example, and the total sum exceeds a threshold value in accordance with the event transition, it may be determined that a corresponding object is highly likely to be approaching. In addition, each event may be weighted.

The discrimination section104compares feature quantity data analyzed by the feature quantity analysis section101and each piece of discrimination data stored in the discrimination data storage section220, determines whether or not there is similarity, and discriminates an approaching object such as a visitor. That is, the discrimination section104detects a feature indicating approach of an object learned in advance from an ultrasonic signal detected by the ultrasonic sensor17. The technique for determining similarity is not particularly limited, but a technique such as the Hidden Markov Model (HMM), for example, for comparing two patterns having different speeds or timings in similarity is used.

In addition, in the case where a state transition of a sound event has been grasped as described with reference toFIG.5, the discrimination section104is also capable of recognizing the sound event on the basis of feature quantity data, and discriminating an approaching object in accordance with the state transition of the sound event.

In the case where it is discriminated by the discrimination section104that there is similarity, the notification control section105exerts control so as to notify the user about a notification content in accordance with a discrimination content. Notification to the user may be performed by sound output from the sound output section22, for example. Information regarding the notification content may be stored in the storage section20in advance, or may be acquired from a network via the communication section11. Alternatively, the information regarding the notification content may be generated by the control section10concurrently when generating discrimination data, and may be stored in the storage section20. Here, an example of discrimination contents and notification contents according to the present embodiment will be described with reference toFIG.6.

FIG.6shows a notification content per discrimination content. In the case of a discrimination content that a courier delivery person is headed toward the entrance door, for example (that is, in the case where currently acquired ultrasonic feature quantity data has similarity to discrimination data indicating that a courier delivery person is headed toward the entrance door) as shown inFIG.6, a notification content such as “A courier delivery person is coming soon” is output by sound. In addition, in the case of a discrimination content that a newspaper carrier is headed toward the entrance door, for example, a notification content such as “A newspaper carrier is coming soon” is output by sound.

The feature quantity database210stores feature quantity data analyzed by the feature quantity analysis section101in chronological order. In addition, the feature quantity database210may fix a storage period of feature quantity data, and the feature quantity data may be overwritten with new feature quantity data after the lapse of the storage period.

The discrimination data storage section220stores discrimination data generated by the learning section103. The discrimination data is used as teacher data when discriminating an approaching object in the discrimination section104.

The functional configuration of the control section10and the storage section20according to the present embodiment has been specifically described above. Next, operation processing of the information processing system according to the present embodiment will be described with reference toFIG.7toFIG.8.

3. OPERATION PROCESSING

<3-1. Generation Processing>

FIG.7is a flowchart showing identification data generation processing according to the present embodiment. As shown inFIG.7, the pet robot1first captures a sound from the ultrasonic sensor17(step S103).

Next, the feature quantity analysis section101performs a feature quantity analysis for a captured ultrasonic signal (step S106).

Then, the feature quantity analysis section101writes the analyzed chronological feature quantity data into the feature quantity database210(step S109).

The above-described sound capturing, analysis of feature quantity data, and writing are performed continuously all the time (for example, every 2 to 3 minutes).

Next, in the case where a command has been issued from the owner (user) (step S115/Yes), the learning section103reads out feature quantity data since a time point going back the predetermined time T from the feature quantity database210(step S118). The command from the owner is based on, for example, voice such as “Tell me if he comes next” or a specific gesture. The control section10analyzes the user uttered voice acquired by the sound input section16, for example, to recognize the command. The learning section103reads out feature quantity data since 1 minute to 10 seconds or the like before the timing when a previously visited visitor, for example, presses the doorbell in accordance with the command.

Then, the learning section103generates discrimination data for discriminating the visitor when the same visitor comes next on the basis of the read-out feature quantity data (step S121). The generated discrimination data is stored in the discrimination data storage section220.

Next, notification processing through use of the generated discrimination data will be described with reference toFIG.8.

<3-2. Notification Processing>

FIG.8is a flowchart showing notification processing according to the present embodiment. As shown inFIG.8, the pet robot1first captures a sound from the ultrasonic sensor17(step S133).

Next, the feature quantity analysis section101analyzes a feature quantity for a captured ultrasonic signal (step S136).

Then, the discrimination section104compares feature quantity data as the analysis result and discrimination data stored in the discrimination data storage section220to discriminate a visitor (step S139).

The above-described feature quantity data analysis and discrimination are performed continuously all the time (for example, every 2 to 3 minutes), and in the case where there is similarity to any piece of discrimination data, the discrimination section104outputs a discrimination result to the notification control section105.

Next, the notification control section105notifies the owner (user) on the basis of the discrimination result (step S145). For example, in the case where it is discriminated that a courier delivery person has come, the notification control section105controls the sound output section22to output by sound that “A courier delivery person has come!”.

Next, in the case where the owner has made a positive evaluation (step S145/Yes), the learning section103updates teacher data (step S148). Examples of the positive evaluation from the owner include an utterance indicating that the notification was correct, a praising action, a specific gesture, and the like, such as saying that “OK, it was a courier delivery” and patting the pet robot1saying “Thank you for your help”. In this case, the learning section108regenerates and improves discrimination data used for discrimination on the basis of this feature quantity data.

Note that such user feedback is not limited to a positive evaluation, but a negative evaluation is also expected. Learning in the case where a negative feedback is given will be described below.

(Learning Based on Feedback)

Change in Notification Timing

For example, in the case where the timing when the pet robot1notified that “A courier delivery person is coming soon” was late when the courier delivery person came, the user gives the pet robot1feedback that “Tell me a little earlier”.

The pet robot1answers “OK”, and improves discrimination data by means of the learning section103to be capable of notifying the user at an earlier timing. Specifically, for example, the learning section103generates discrimination data on the basis of feature quantity data F0, F1, and F2since a time point going back a predetermined time T2, which is longer than the predetermined time T1, as shown inFIG.9. Accordingly, in the example described with reference toFIG.4, the user is notified at the time point when the feature quantity data F3is sensed, while in the example shown inFIG.9, a notification can be given at an earlier time point when the feature quantity data F2is sensed.

In the Case of Incorrect Notification

For example, in the case where the pet robot1notified that “A courier delivery person is coming soon”, but it was actually a newspaper carrier, the user gives the pet robot1feedback that “It was different”, “It was a newspaper carrier”, or the like.

In this case, the pet robot1refrains from performing the update processing shown in the above-described step S148. As described above, by regenerating discrimination data only in the case where positive feedback is given, it is possible to improve the accuracy of discrimination data.

4. APPLICATION EXAMPLES

In the above-described embodiment, the case of sensing ultrasound to discriminate a visitor or the like has been described, whilst the present embodiment is not limited to this, but it is also possible to sense approach of a visitor or the like by means of radio waves or a smell. Hereinafter, specific description will be provided with reference toFIG.10toFIG.17.

<4-1. Notification Processing Through Use of Radio Waves>

It is assumed that a person visiting on business, such as a courier delivery person or a newspaper carrier, usually carries electronic equipment related to his/her business. It is assumed that he/she carries a mobile phone, smartphone, or tablet terminal, and in the case of a courier delivery person, equipment equivalent to a Point Of Sales (POS) terminal with a barcode scanner for managing deliveries. These types of electronic equipment are connected to a communication network directly or by way of a mobile phone terminal or smartphone using a wireless communication system such as Bluetooth, ANT, or Wi-Fi. It is assumed that even a suspicious person or an intruder carries some electronic equipment. In addition, even electronic equipment that does not actively emit radio waves discharges electro magnetic interference (EMI) pulses, referred to as unnecessary radiation, specific to the equipment.

The pet robot1according to the present embodiment is capable of discriminating a visitor by sensing WAN, Bluetooth, ANT, Wi-Fi, or unnecessary radiation emitted by such electronic equipment.

Specifically, for example, describing Wi-Fi as an example, Wi-Fi equipment has a terminal-specific ID referred to as a MAC address, and terminal equipment issues a signal referred to as a Probe Request in order to connect to an access point. Wi-Fi radio waves propagate a distance of usually about 10 m to 30 m depending on the environment, and the distance to the terminal equipment can be estimated by the strength of radio waves (RSSI).

The pet robot1according to the present embodiment is capable of receiving Wi-Fi radio waves with the communication section11, acquiring a MAC address to be checked against an existing MAC address list, and distinguishing whether a stranger has just passed in front of the house, or a courier delivery person or an acquaintance has come.

Here,FIG.10shows an example of the MAC address list according to the present embodiment.FIG.10shows that a terminal holder whose MAC address is “12:34:AB:CD” is a “courier delivery person a”, a terminal holder whose MAC address is “56:78:EF:EF” is a “mail carrier”, and a terminal holder whose MAC address is “98:76:FE:DC” is “husband”. The MAC address list may be stored in the storage section20in advance, or may be externally acquired via the communication section11.

The pet robot1according to the present embodiment is capable of checking the MAC address acquired by receiving Wi-Fi radio waves against the MAC address list as shown inFIG.10to discriminate a visitor and notifying the user.

(Discrimination Processing Through Use of Radio Waves and Ultrasound)

In addition, the pet robot1according to the present embodiment is capable of narrowing down visitors using the MAC addresses, and then discriminating a visitor using ultrasound to increase the accuracy of discrimination. Since the MAC address is a specific ID, the likelihood of a discriminated person becomes absolutely high in the case where the discriminated person can be identified by Wi-Fi radio waves. In addition, radio waves are less attenuated as compared with ultrasound, and can be sensed remotely. Therefore, the accuracy can be increased by initially sensing a visitor by Wi-Fi radio waves useful in the case where the distance is far, and when the visitor approaches, identifying him/her by means of ultrasound. Such operation processing in the case of discriminating a visitor using both radio waves and ultrasound and notifying the user will be described below with reference toFIG.11.

FIG.11is a flowchart showing notification processing through use of radio waves and ultrasound according to the present embodiment. In the example shown inFIG.11, Wi-Fi is used as an example of received radio waves.

As shown inFIG.11, first, the communication section11of the pet robot1scans Wi-Fi radio waves (step S203).

Next, in the case where the strength of Wi-Fi radio waves is more than or equal to a threshold value (step S206/Yes), the control section10checks a MAC address acquired by scanning against the MAC address list (step S209).

Then, in the case where the acquired MAC address is included in the list (step S212/Yes), the control section10includes the acquired MAC address on a visitor candidate list (step S215).

Next, the pet robot1captures a sound from the ultrasonic sensor17(step S218).

Then, the feature quantity analysis section101analyzes a feature quantity for a captured ultrasonic signal, and compares the analyzed feature quantity data and discrimination data stored in the discrimination data storage section220(step S221). In the comparison with the discrimination data, a determination of similarity (calculation of similarity) between the analyzed feature quantity data and the discrimination data may be performed.

Next, in the case where the similarity to the discrimination data is more than or equal to a threshold value (step S224/Yes), the discrimination section104performs visitor identification by means of ultrasound (step S227). That is, the discrimination section104discriminates a visitor on the basis of discrimination data whose similarity is more than or equal to the threshold value.

Next, in the case where the visitor identified by the discrimination section104is the same as the above-described candidate list (is included in the candidate list) (step S230/Yes), the notification control section105exerts control so as to notify the owner about the visitor (step S233). As described above, since Wi-Fi radio waves can be sensed more remotely than ultrasound which is a sound event, it is not clear if a person possessing an electronic terminal having the acquired MAC address just passes in front of the house, visits a neighboring house, or comes to the house where the pet robot1is present. Therefore, by sensing the person approaching the house with the ultrasonic sensor17and identifying him/her, the discrimination accuracy can be increased further.

The case of complementarily utilizing both radio waves and ultrasound to increase the accuracy of discrimination has been described above. Note that the method of complementarily utilizing the both is not limited to the example of using a MAC address (terminal-specific ID) described above, but it is also possible to increase the accuracy of discrimination by learning a co-occurrence relationship between an ultrasonic feature and radio wave signal detection, for example. Specific description will be provided below with reference toFIG.12.

FIG.12is a flowchart showing processing of collecting chronological data of ultrasound and radio waves according to the present embodiment. As shown inFIG.12, the pet robot1first captures a sound from the ultrasonic sensor17(step S243).

Next, the feature quantity analysis section101analyzes a feature quantity for a captured ultrasonic signal (step S246).

Then, radio waves are received by the communication section11(step S249), and the control section10performs a radio wave analysis such as a frequency analysis, signal strength analysis, and ID detection (step S252).

Next, the control section10writes the analyzed chronological feature quantity data and the radio wave analysis result into the feature quantity database210in association with each other (step S255).

The above-described sound capturing, analysis of feature quantity data, radio wave reception, radio wave analysis, and writing are performed continuously all the time (for example, every 2 to 3 minutes).

Next, in the case where a command has been issued from the owner (user) (step S258/Yes), the learning section103reads out feature quantity data and a radio wave analysis result since a time point going back the predetermined time T from the feature quantity database210(step S261).

Then, the learning section103generates discrimination data for discriminating a visitor when the same visitor comes next on the basis of the read-out chronological feature quantity data and radio wave analysis result (step S2641). The generated discrimination data is stored in the discrimination data storage section220.

In the discrimination and notification processing through use of discrimination data generated in this manner, the acquired chronological ultrasonic feature quantity data and radio wave analysis result are compared with the discrimination data to distinguish whether or not there is similarity, and a visitor is discriminated, similarly to the case of using ultrasound described with reference toFIG.8. In the present embodiment, by increasing the types of chronological patterns to be used for discrimination data and using more parameters in discrimination, the discrimination accuracy can be increased.

<4-2. Notification Processing Through Use of Smell>

Next, the case of sensing a person to be discriminated using a smell will be described with reference toFIG.13toFIG.14. Although the smell often depends on the environment (season, whether the window is open or closed, wind direction) or the like, it is possible to discriminate a visitor when the pet robot1senses the smell and checks a smell list provided in advance.

FIG.13is a block diagram showing an example of a configuration of a pet robot1A according to an application example of the present embodiment. As shown inFIG.13, the pet robot1A has the control section10, the communication section11, the camera15, the sound input section16, the ultrasonic sensor17, the driving section18, the touch sensor19, the storage section20, the sound output section22, the acceleration sensor23, the angular velocity sensor24, and a smell sensor25.

The smell sensor25senses a surrounding smell, and outputs a sensing result to the control section10.

In addition, a smell list is stored in the storage section20. Here, an example of the smell list is shown inFIG.14.

FIG.14shows that the case of the smell of uniform cleaning and the smell of a vehicle is “a courier delivery person”, and the case of the smell of some sweat and the smell of eau de cologne for disguising the smell of sweat is “husband”.

The control section10is capable of checking the smell list on the basis of the smell sensed by the smell sensor25, discriminating a visitor, and notifying the user about the visitor from the sound output section22or the like.

In addition, the control section10is also capable of increasing the accuracy by discriminating the visitor using both ultrasound and a smell. Specifically, the control section10, for example, is capable of identifying a visitor using ultrasound, and identifying the visitor further on the basis of a smell.

<4-3. Operation Processing in Accordance with Emotion>

Next, operation processing in accordance with an emotion of a user (owner) will be described. The pet robot1according to the present embodiment may recognize the user's face from a captured image captured by the camera15, for example, to sense a user's emotion from a facial expression, or may sense a user's emotion from user's voice collected by the sound input section16. The pet robot1is capable of performing automatic learning in accordance with the sensed user's emotion. Specific description will be provided below with reference toFIG.15toFIG.17.

(Configuration)

Although detailed description will be omitted since an essential configuration of the pet robot1capable of sensing emotions is as shown inFIG.2, a functional configuration different from the above-described embodiment will be described below with reference toFIG.15.

FIG.15is a block diagram showing a functional configuration example of a control section10B and the storage section20of the pet robot1according to the present embodiment. In the drawing, the feature quantity analysis section101, the storage control section102, the learning section103, the discrimination section104, the notification control section105, and an emotion sensing section106are shown as functions of the control section10B of the pet robot1. In addition, in the drawing, the feature quantity database210and the discrimination data storage section220are shown as the functions of the storage section20.

The emotion sensing section106senses (estimates) a user's emotion on the basis of data acquired by various sensors. Specifically, for example, the emotion sensing section106distinguishes an expression of the user's face on the basis of a captured image captured by the camera15, and senses a user's emotion. In addition, the emotion sensing section106distinguishes a voice tone and utterance contents on the basis of the user's voice collected by the sound input section16, and senses a user's emotion.

Operation processing performed by the control section10B in accordance with the user's emotion sensing result obtained by such an emotion sensing section106will be specifically described below.

(Operation Processing)

FIG.16is a flowchart showing processing of changing a notification content in accordance with a user's emotion according to the present embodiment. As shown inFIG.16, the emotion sensing section106of the pet robot1first senses an owner's emotion when answering a visitor (step S303). The emotion sensing section106is capable of analyzing a captured image captured by the camera15to sense a user's emotion (anger, disgust, fear, happiness, sorrow, or surprise) from the user's expression. For example, the pet robot1notifies about the visitor that “A newspaper carrier is coming soon” and then images, with the camera15, an expression when the user is answering the visitor and monitors the expression to sense what face (expression) is held.

Next, in the case where the user's emotion is “anger” or “disgust” (step S306/Yes), the learning section103adds a predetermined content to the notification content (step S309). For example, in the case where the user holds a disgusted face at the time of answering, the expression of “disgust” is sensed, and the learning section103adds a notification such as “There is a choice of not answering the door” with the user's emotion taken into consideration to the notification content that “A newspaper carrier is coming soon” when the same visitor is discriminated (update of the notification content). Accordingly, when the newspaper carrier comes next, the pet robot1is capable of notifying the user that “A newspaper carrier is coming soon. There is a choice of not answering the door.

FIG.17is a flowchart showing automatic learning processing in accordance with a user's emotion according to the present embodiment. As shown inFIG.17, the emotion sensing section106of the pet robot1first senses an owner's emotion when answering a visitor (step S323). The emotion sensing section106is capable of analyzing a captured image captured by the camera15to sense a user's emotion (anger, disgust, fear, happiness, sorrow, or surprise) from the user's expression.

Next, in the case where a negative emotion such as “anger”, “disgust”, “fear”, “sorrow”, or the like is sensed as the user's emotion (step S306/Yes), the learning section103learns ultrasonic feature quantity data so as to be capable of notifying the user in advance from the next time the same visitor comes (step S309). Specific processing of learning is as described above with reference toFIG.7(step S118and step S121inFIG.7). In the example described with reference toFIG.7, learning is performed in the case where there is a command from the user (owner), but can be performed automatically in accordance with the user's emotion in the present embodiment. In this manner, learning is performed so as to notify the user in advance in the case of a visitor who brings a negative emotion, and a prior notification can be made from the next time, so that the user can avoid being surprised at a sudden visit, or can prepare himself/herself when answering a person the user dislikes, for example.

On the other hand, in the case where the user's emotion is not a negative emotion (for example, in the case of a happy emotion or lack of emotion) (step S326/No), the pet robot1does not particularly perform learning since there is no need to give a notification in advance.

5. CONCLUSION

As described above, the information processing system according to an embodiment of the present disclosure makes it possible to sense approach of an object and notify a user in advance.

More specifically, the pet robot1according to the present embodiment can sense a feel of a person or an object in the vicinity with the passive ultrasonic sensor17and notify an owner. In addition, the pet robot1is capable of sensing a feel using radio waves or a smell complementarily to ultrasound. In addition, the pet robot1can also sense an owner's emotion against a visitor to automatically perform learning.

The preferred embodiment(s) of the present disclosure has/have been described above with reference to the accompanying drawings, whilst the present disclosure is not limited to the above examples. A person skilled in the art may find various alterations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present disclosure.

For example, it is also possible to generate a computer program for causing the functions of the pet robot1to be exerted in hardware such as a CPU, ROM, and RAM included in the pet robot1of the above-described information processing system. In addition, a computer-readable storage medium having the computer program stored thereon is also offered.

In addition, in the above-described embodiment, the pet robot1alone performs discrimination of a visitor, emotion sensing, and the like, whilst the present embodiment is not limited to this, but it is also possible to achieve the above-described processing on the server side. That is, in an information processing system including the pet robot1and a server (not shown), the pet robot1may transmit an ultrasonic signal sensed by the ultrasonic sensor17from the communication section11to the server, and feature quantity analysis, storage control, learning, discrimination, and notification control may be performed on the server side. In addition, the pet robot1may transmit a captured image captured by the camera15from the communication section11to the server, and a user's emotion may be sensed on the server side. In this manner, a configuration in which the server side has at least part of the functional configuration of the pet robot1described inFIG.3andFIG.15may be adopted.

Further, the effects described in this specification are merely illustrative or exemplified effects, and are not limitative. That is, with or in the place of the above effects, the technology according to the present disclosure may achieve other effects that are clear to those skilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1)

An information processing device including:a passive ultrasonic sensor;a notification section configured to notify a user; anda control section configured to, when detecting a feature indicating approach of an object learned in advance from sensor data detected by the ultrasonic sensor, control the notification section to notify the user.
(2)

The information processing device according to (1), further including:a command acquisition section configured to acquire a command from the user, in whichwhen recognizing that a specific command has been acquired by the command acquisition section, the control section exerts control to generate teacher data for recognizing an approaching object on a basis of sensor data detected since a time point going back a predetermined period and to store the teacher data in a storage section.
(3)

The information processing device according to (2), in whichthe control section detects a feature similar to the teacher data from the sensor data detected by the ultrasonic sensor, and controls the notification section to notify the user that an object approaches, andwhen recognizing that a command indicating a positive evaluation has been acquired by the command acquisition section in response to the notification, the control section updates the teacher data on a basis of the sensor data detected by the ultrasonic sensor.
(4)

The information processing device according to (2) or (3), in whichthe teacher data is data indicating a feature of chronological sensor data since the time point going back the predetermined period.
(5)

The information processing device according to (4), in whichwhen recognizing that a command requesting a still earlier notification has been acquired by the command acquisition section in response to the notification, the control section updates the teacher data on a basis of data indicating a feature of chronological sensor data since a time point going back further than the predetermined period.
(6)

The information processing device according to (4) or (5), in whichwhen features similar to the teacher data are successively detected from the chronological sensor data detected by the ultrasonic sensor, the control section controls the notification section to notify the user that an object approaches.
(7)

The information processing device according to any one of (4) to (6), in whicha feature detected from the sensor data is a state transition or a waveform.
(8)

The information processing device according to any one of (1) to (6), further including:a radio wave receiving section, in whichwhen detecting a feature indicating approach of an object, learned in advance, in the sensor data detected by the ultrasonic sensor and a radio wave signal received by the radio wave receiving section, the control section controls the notification section to notify the user.
(9)

The information processing device according to (8), in whichthe control sectionperforms, after first recognition processing of recognizing an approaching object in accordance with an ID included in the radio wave signal, second recognition processing of recognizing the approaching object in accordance with a feature included in the sensor data detected by the ultrasonic sensor, andexerts control to notify the user about the approaching object in accordance with coincidence between results of the first recognition processing and the second recognition processing.
(10)

The information processing device according to any one of (1) to (9), further including:a smell sensor, in whichwhen detecting a feature indicating approach of an object, learned in advance, from the sensor data detected by the ultrasonic sensor and smell data detected by the smell sensor, the control section controls the notification section to notify the user.
(11)

The information processing device according to (10), in whichsmell data and an approaching object are associated in advance, ultrasonic sensor data and the approaching object are associated in advance, andthe control section exerts control to recognize the approaching object in accordance with the smell data, recognize the approaching object in accordance with the ultrasonic sensor data, and notify the user about the approaching object in accordance with coincidence between the two recognition results.
(12)

The information processing device according to any one of (1) to (11), further including:an imaging section configured to image a face of the user, in whichthe control sectionestimates an emotion of the user from a face image of the user captured by the imaging section when the user answers a visitor serving as the object, andupdates a content to be notified by the notification section in accordance with a result of estimation of the emotion.
(13)

The information processing device according to any one of (1) to (12), further including:an imaging section configured to image a face of the user, in whichthe control section estimates an emotion of the user from a face image of the user captured by the imaging section when the user answers a visitor serving as the object, andin accordance with a result of the estimation after the user answers the visitor,on a basis of sensor data detected since a time point going back a predetermined period from that time point, the control section exerts control to generate teacher data for recognizing the visitor, and to store the teacher data in a storage section.
(14)

A storage medium having a program stored thereon, the program causing a computer to function asa control section configured to control a notification section to notify a user when detecting a feature indicating approach of an object learned in advance from sensor data detected by a passive ultrasonic sensor.

REFERENCE SIGNS LIST

1pet robot10control section11communication section15camera16sound input section17ultrasonic sensor18driving section19touch sensor20storage section22sound output section23acceleration sensor24angular velocity sensor25smell sensor101feature quantity analysis section102storage control section103learning section104discrimination section105notification control section106emotion sensing section210feature quantity database220discrimination data storage section