Patent Description:
A navigation device is known that displays an icon of a point of interest (POI). See Patent Literature <NUM>: <CIT>. The invention disclosed in Patent Literature <NUM> stores a frequency of usage of POIs used by the user, and displays icons of POIs having a high usage frequency. Other examples of navigation devices are disclosed in <CIT>, <CIT>, <CIT> and <CIT>.

The invention disclosed in Patent Literature <NUM>, which simply displays the POIs with the high usage frequency, does not teach any function of recording actions of the user at the POIs.

To solve the conventional problem described above, the present invention provides an information processing device and an information processing method capable of storing actions of a user at POIs.

An information processing device and an information processing method according to the present invention are defined by the appended independent claims. Specific embodiments are defined by the dependent claims. Such a device outputs, from an output device, output data including at least a question regarding an action of a user taken before riding in a vehicle in accordance with positional information of the vehicle when detecting a ride of the user in the vehicle based on a signal acquired from a ride detection device, acquires an answer to the question from the user as input data via an input device, and associates the input data with the positional information of the vehicle or a POI to store the associated data in a storage device.

The present invention can store actions of the user at POIs.

Some embodiments according to the present invention are described below with reference to the drawings. The same elements illustrated in the drawings are denoted by the same reference numerals, and overlapping explanations are not made below.

The entire outline of the present embodiment is described below with reference to <FIG>. As illustrated in <FIG>, a vehicle <NUM> communicates with a computer <NUM> via a communication network <NUM>.

The computer <NUM> includes a central processing unit (CPU) <NUM>, a memory <NUM>, a communication I/F <NUM>, and a storage <NUM>, which are electrically connected to each other via buses (not illustrated). An installed position of the computer <NUM> can be any location determined as appropriate.

The CPU <NUM> reads, into the memory <NUM>, various kinds of programs stored in the storage <NUM>, for example, and executes various types of instructions included in the programs. The memory <NUM> is a storage medium such as a read-only memory (ROM) and a random-access memory (RAM). The storage <NUM> is a storage medium such as a hard disk drive (HDD). The functions of the computer <NUM> may be provided through applications such as software as a service (SaaS) provided on the communication network <NUM>. The computer <NUM> may be a server instead.

The communication I/F <NUM> is implemented by hardware such as a network adopter, various kinds of software for communication, or a combination thereof, and is configured so as to execute wired or wireless communication via the communication network <NUM> and the like.

The communication network <NUM> may be configured in either a wireless or wired mode or in both wireless and wired modes. The communication network <NUM> may also include the Internet. The computer <NUM> and the vehicle <NUM> in the present embodiment are connected to the communication network <NUM> in a wireless communication mode.

An example of a configuration of the information processing device <NUM> mounted on the vehicle <NUM> is described below with reference to <FIG>.

As illustrated in <FIG>, the information processing device <NUM> includes a sensor group <NUM>, a GPS receiver <NUM>, a microphone <NUM>, a storage device <NUM>, a speaker <NUM>, and a controller <NUM>.

The information processing device <NUM> may be mounted on either a vehicle equipped with an automated driving function or a vehicle without being equipped with an automated driving function. The information processing device <NUM> may be mounted on a vehicle capable of switching between automated driving and manual driving. The term "automated driving" as used in the present embodiment refers to a state in which at least any of a brake actuator, an accelerator actuator, and a steering wheel actuator is controlled without the intervention of operation of the user. The automated driving thus can include a state in which other actuators are operated by the user. The automated driving also refers to a state in which any control such as acceleration/deceleration control and lateral positioning control is executed. The term "manual driving" as used in the present embodiment refers to a state in which the user operates the brake pedal, the accelerator pedal, and the steering wheel, for example.

The sensor group <NUM> (a ride detection device) is used to detect a ride of the user in the vehicle. The sensor group <NUM> includes a pressure-sensitive sensor (also referred to as a seating-sensitive sensor) provided at a seat cushion, a camera for capturing the user in the compartment, and a sensor for detecting open and closed states of doors. The controller <NUM> uses signals obtained from the respective sensors so as to detect the ride of the user in the vehicle.

The GPS receiver <NUM> receives radio waves from an artificial satellite so as to detect positional information of the vehicle <NUM> on the ground. The positional information of the vehicle <NUM> detected by the GPS receiver <NUM> includes the latitude information and the longitude information. The GPS receiver <NUM> outputs the detected positional information of the vehicle <NUM> to the controller <NUM>. The means of detecting the positional information of the vehicle <NUM> is not limited to the GPS receiver <NUM>. For example, a means called odometry may be used to estimate the position of the vehicle <NUM>. The odometry is a means of obtaining a moving amount and a moving direction of the vehicle <NUM> in accordance with a rotation angle and an angular velocity of the vehicle <NUM> so as to estimate the position of the vehicle <NUM>.

The microphone <NUM> is used for inputting voice of the user.

The storage device <NUM> is a device different from a memory of the controller <NUM>, and is a hard disk or a solid-state drive, for example. The storage device <NUM> stores a map database 53a, an action history database 53b, and a question database 53c.

The map database 53a stores map information, such as road information and facility information, necessary for a route guidance. The road information refers to information on the number of lanes on a road, boundary lines on a road, and a relation of connection of lanes. The map database 53a outputs the map information to the controller <NUM> in response to the request from the controller <NUM>. While the present embodiment is illustrated with the case in which the information processing device <NUM> includes the map database 53a, the information processing device <NUM> does not necessarily include the map database 53a. The map information may be acquired through vehicle-to-vehicle communications or road-to-vehicle communications. When the map information is stored in a server externally located (for example, the computer <NUM> illustrated in <FIG>), the information processing device <NUM> may acquire the map information from the server through communications as necessary. The information processing device <NUM> may regularly acquire the latest map information from the server so as to update the stored map information.

The map database 53a also stores information regarding points of interest (POIs). The term "POI" as used in the present embodiment refers to data indicating a specific point location. The POI includes at least an attribute and positional information (latitude, longitude) of the location. The attribute refers to information used for classifying POIs, such as a restaurant, a shopping mall, and a park. The POI may also include a name, an address, a phone number, and an icon of the location. The term "name" as used herein refers to, when the attribute is a restaurant, a specific name of the restaurant. The positional information of the POI may include an altitude in addition to a latitude and a longitude.

The action history database 53b stores actions of the user at POIs in association with the positional information of the vehicle <NUM> or the POIs. The actions of the user at the POIs are put in database form as a history.

The question database 53c stores the attribute of the respective POIs associated with questions regarding the actions of the user taken at the POIs.

The controller <NUM> is a general-purpose microcomputer including a central processing unit (CPU), a memory, and an input-output unit. A computer program is installed on the microcomputer so as to function as the information processing device <NUM>. The microcomputer functions as a plurality of information processing circuits included in the information processing device <NUM> when the computer program is executed. While the present embodiment is illustrated with the case in which the software is installed to fabricate the respective information processing circuits included in the information processing device <NUM>, dedicated hardware for executing each information processing as described below can be prepared to implement the information processing circuits. The respective information processing circuits may be composed of individual hardware. The controller <NUM> includes, as examples of the respective information processing circuits, a user identification unit <NUM>, a destination arrival determination unit <NUM>, a voice analysis unit <NUM>, an action history update unit <NUM>, an action estimation unit <NUM>, a question selection unit <NUM>, a ride detection unit <NUM>, and a question output unit <NUM>.

The speaker <NUM> is installed in the compartment of the vehicle <NUM> so as to output voice.

The user identification unit <NUM> identifies the user by use of a face image of the user captured when riding in the vehicle <NUM>. In particular, the user identification unit <NUM> determines whether the captured face image conforms to or is similar to a face image preliminarily registered in the storage device <NUM>. When the captured face image conforms to or is similar to the face image preliminarily registered in the storage device <NUM>, the user identification unit <NUM> identifies the user riding in the vehicle <NUM> as a user preliminarily registered. The face identification described above can also be executed when the ignition is OFF.

Another method of identifying the user may use an ID of an intelligence key (also referred to as a smart key). A system using the intelligence key includes an antenna that transmits radio waves and a receiver that receives the radio waves, which are provided in both the vehicle and the key. When the user presses a switch provided on a door handle or a trunk, for example, the radio waves are transmitted from the antenna of the vehicle, and the key when receiving the radio waves automatically returns the radio waves. The receiver of the vehicle then receives the radio waves, so as to lock or unlock the doors. The radio waves transmitted from the key include an identification cipher that is preliminarily registered in the vehicle. Associating the identification cipher with the user information can identify the user.

The destination arrival determination unit <NUM> determines whether the vehicle <NUM> has arrived at a destination. The term "destination" as used in the present embodiment refers to a target location set by the user through a navigation device (not illustrated). The positional information on the destination is stored in the map database 53a. The destination arrival determination unit <NUM> checks the positional information of the vehicle <NUM> acquired from the GPS receiver <NUM> with the positional information of the destination stored in the map database 53a, and determines that the vehicle <NUM> has arrived at the destination when both information conforms to or substantially conforms to each other.

The voice analysis unit <NUM> analyzes voice of the user input via the microphone <NUM>. The analyzing method as used herein is a conventionally-known method.

The action history update unit <NUM> stores actions that the user takes at POIs in the action history database 53b in association with the positional information of the vehicle <NUM> or the POIs, and updates the action history of the user at the POIs.

The action estimation unit <NUM> estimates the action of the user taken before riding in the vehicle. The term "action of the user taken before riding in the vehicle" as used in the present embodiment refers to an action that the user takes before the user rides in the vehicle <NUM>. In particular, the action of the user taken before riding in the vehicle refers to an action that the user takes at a destination before riding in the vehicle <NUM>.

The action estimation unit <NUM> also acquires schedule data of the user from the computer <NUM> through the communication network <NUM>. The schedule data of the user includes information on an action plan of the user, such as a time "when", a place "where", and a thing "what". The schedule data may be acquired from a terminal (such as a smartphone) that the user holds.

The question selection unit <NUM> refers to the action estimated by the action estimation unit <NUM> and the question database 53c so as to choose a question put to the user. When the question selection unit <NUM> chooses the question, a signal indicating the chosen question is output to the question output unit <NUM>.

The ride detection unit <NUM> detects a ride of the user in the vehicle in accordance with a signal acquired from the sensor group <NUM>. In particular, the ride detection unit <NUM> detects the user riding in the vehicle when acquiring a signal indicating a change in resistance value from the pressure-sensitive sensor. The ride detection unit <NUM> also detects the user riding in the vehicle when detecting the presence of the user in the compartment through the analysis of the camera image. The ride detection unit <NUM> may detect the ride of the user when acquiring the signal indicating the change in the resistance value from the pressure-sensitive sensor after the detection of the open or closed state of the door. When the ride detection unit <NUM> detects the user riding in the vehicle, a signal indicating the detection of the ride is output to the question output unit <NUM>.

The question output unit <NUM> outputs the question chosen by the question selection unit <NUM> when receiving the signal indicating the detection of the ride of the user <NUM>. The question may be output by voice via the speaker <NUM>, or may be output by text information to a display (such as a display of the navigation device). The present embodiment is illustrated with the case in which the question is output by voice via the speaker <NUM>.

An example of a method of storing the action of the user taken at a POI is described below with reference to <FIG>.

The situation illustrated in <FIG> is a case in which the vehicle <NUM> arrives at a destination (a ramen shop <NUM> in this case) set by the user <NUM>, and is parked in a parking space at the ramen shop <NUM>. The user <NUM> gets out of the vehicle <NUM> and enters the ramen shop <NUM>. The user <NUM> then leaves the ramen shop <NUM> and rides in the vehicle <NUM> after finishing things to be done at the ramen shop <NUM>.

When the destination arrival determination unit <NUM> determines that the vehicle <NUM> has arrived at the destination, a signal indicating the determination result is output to the action estimation unit <NUM>. The action estimation unit <NUM> when receiving the signal estimates the action of the user <NUM> taken before riding in the vehicle. As described above, the action of the user <NUM> taken before riding in the vehicle refers to an action that the user <NUM> takes at the destination when the user <NUM> is out of the vehicle <NUM>. Since the destination of the user <NUM> in the case illustrated in <FIG> is the ramen shop <NUM>, the action of the user <NUM> taken before riding in the vehicle refers to the action that the user <NUM> has taken at the ramen shop <NUM>.

A method of estimating the action of the user <NUM> by the action estimation unit <NUM> is described below. The action estimation unit <NUM> first acquires the positional information of the vehicle <NUM> from the GPS receiver <NUM>. In particular, the action estimation unit <NUM> acquires the positional information of the vehicle <NUM> when the vehicle <NUM> is determined to have arrived at the destination by the destination arrival determination unit <NUM>.

The action estimation unit <NUM> checks the positional information of the vehicle <NUM> with the map database 53a, and acquires the POI at the current position (the parked position of the vehicle <NUM>). In particular, as illustrated in <FIG>, the action estimation unit <NUM> acquires the positional information of the POI (the latitude and the longitude) and the attribute of the POI (the ramen shop). In the case illustrated in <FIG>, since the vehicle <NUM> is parked in the parking space at the ramen shop <NUM>, the positional information of the vehicle <NUM> is presumed to conform to the positional information of the POI. The action estimation unit <NUM> acquires the positional information of the POI that conforms to or substantially conforms to the positional information of the vehicle <NUM>. The action estimation unit <NUM> acquires the attribute of the POI associated with the positional information of the POI.

As described above, the action estimation unit <NUM> checks the positional information of the vehicle <NUM> with the map database 53a, so as to recognize that the current position is the ramen shop <NUM>. The action estimation unit <NUM> then refers to table data in which the attribute of the POI is associated with the action type of the user stored in the storage device <NUM> so as to estimate the action of the user <NUM> taken before riding in the vehicle. The action type of the user as used in the present embodiment is a classification of the actions that the user could take at the POI. For example, when the attribute of the POI is a restaurant, two actions of taking a meal and having a meeting are stored in association with the restaurant as the action type of the user. The action type of the user does not necessarily include plural actions, and may include a single action. When the attribute of the POI is a ramen shop as illustrated in <FIG>, the action of taking a meal is only associated with the ramen shop as the action type of the user. The action of the user <NUM> taken before riding in the vehicle in the case illustrated in <FIG> estimated by the action estimation unit <NUM> is thus to take a meal.

A signal indicating the estimated result obtained by the action estimation unit <NUM> is output to the question election unit <NUM>. The question selection unit <NUM> when receiving the signal from the action estimation unit <NUM> refers to the action of the user <NUM> (taking a meal) taken before riding in the vehicle and the question database 53c so as to choose a question put to the user <NUM>. As described above, the question database 53c stores the attribute of the POI in association with the questions regarding the actions of the user at the POI. The question election unit <NUM> refers to the action of the user <NUM> (taking a meal) taken before riding in the vehicle and the question database 53c, so as to choose a question regarding a meal. For example, a question with two choices that the user can answer with either YES or NO, such as "Did you have something for a meal?", is stored as a question regarding a meal. A question asking a particular answer, such as "What did you have for a meal?", is also stored. The question election unit <NUM> can optionally choose any question regarding a meal. When the question election unit <NUM> chooses the question, a signal indicating the chosen question is output to the question output unit <NUM>.

The timing of asking the user the question can be, but not necessarily, immediately after the user <NUM> rides in the vehicle <NUM>. The reason for this is that the probability that the user <NUM> could forget the action taken at the POI increases as the time elapsed after the user <NUM> rides in the vehicle <NUM> is longer. The ride detection unit <NUM> thus detects the ride of the user in accordance with the signal acquired from the sensor group <NUM>. When the ride detection unit <NUM> detects the ride of the user <NUM>, a signal indicating the detection result is output to the question output unit <NUM>.

The question output unit <NUM>, when receiving the signal indicating the detection of the ride of the user <NUM>, asks the user the question chosen by the question selection unit <NUM> by voice via the speaker <NUM>. The user <NUM> in this case is then asked a question, "Did you have something for a meal?", for example.

The user <NUM> answers the question by voice via the microphone <NUM>. The voice analysis unit <NUM> analyzes the voice data of the user <NUM>. When the user <NUM> answers "YES", the user <NUM> is determined to have taken a meal as the action taken at the ramen shop <NUM> through the voice analysis. The action history update unit <NUM> associates the positional information of the vehicle <NUM> with the action indicating that the user <NUM> has taken a meal at the ramen shop <NUM>, and stores the associated information in the action history database 53b. The action history update unit <NUM> may also associate the POI with the action indicating that the user <NUM> has taken a meal at the ramen shop <NUM> and store the associated information in the action history database 53b. The present embodiment thus can store the action of the user <NUM> taken at the POI.

The data stored in the action history database 53b includes the date and time when the answer of the user is input, the positional information of the vehicle <NUM>, the POI (the positional information and the attribute), and the action taken at the POI.

Another example of the method of storing the action of the user taken at a POI is described below with reference to <FIG>.

The situation illustrated in <FIG> is a case in which the vehicle <NUM> arrives at a destination (a shopping mall <NUM> in this case) set by the user <NUM>, and is parked in a parking space at the shopping mall <NUM>. The user <NUM> gets out of the vehicle <NUM> and enters the shopping mall <NUM>. The user <NUM> then leaves the shopping mall <NUM> and rides in the vehicle <NUM> after finishing things to be done at the shopping mall <NUM>.

The action estimation unit <NUM> checks the positional information of the vehicle <NUM> with the map database 53a, and recognizes that the current position is the shopping mall <NUM>, in the same manner as in the case illustrated in <FIG>. The action estimation unit <NUM> then refers to the table data in which the attribute of the POI is associated with the action type of the user <NUM>, so as to estimate the action of the user <NUM>.

The shopping mall <NUM> in the present embodiment is a commercial complex occupied by a plurality of stores, restaurants, beauty salons, travel agencies, and fitness gyms, for example. The shopping mall is also referred to as a shopping center.

The type of the action of the user taken at the shopping mall <NUM> includes a plurality of actions such as taking a meal, doing shopping, waiting for someone, having a haircut, and taking exercise. To accurately estimate the action of the user <NUM> among the plural actions, the action estimation unit <NUM> refers to the schedule data of the user <NUM> acquired from the computer <NUM>. As described above, since the schedule data includes the information on the action plan of the user <NUM>, the action estimation unit <NUM> referring to the schedule data can estimate the action of the user <NUM> taken at the shopping mall <NUM> with a high accuracy. The schedule data in this case is assumed to include the information "taking a meal at the shopping mall <NUM>" as the action plan of the user <NUM>, for example. The action estimation unit <NUM> then refers to the schedule data so as to estimate that the action of the user <NUM> taken at the shopping mall <NUM> is to take a meal.

The question selection unit <NUM> when receiving the signal from the action estimation unit <NUM> refers to the action of the user <NUM> taken before riding in the vehicle and the question database 53c so as to choose a question to be put to the user <NUM>. The question to be put to the user <NUM> in this case is assumed to be a question asking the user "Did you have something for a meal?", as in the case illustrated in <FIG>.

The user <NUM> answers the question by voice via the microphone <NUM>. The voice analysis unit <NUM> analyzes the voice data of the user <NUM>. When the user <NUM> answers "YES", the user <NUM> is determined to have taken a meal at the shopping mall <NUM> through the voice analysis. The action history update unit <NUM> associates the positional information of the vehicle <NUM> with the action indicating that the user <NUM> has taken a meal at the shopping mall <NUM>, and stores the associated information in the action history database 53b. The action history update unit <NUM> may also associate the POI with the action indicating that the user <NUM> has taken a meal at the shopping mall <NUM> and store the associated information in the action history database 53b. Referring to the schedule data thus can store the action of the user <NUM> taken at the POI with a high accuracy if there are plural kinds of actions that the user <NUM> would take at the POI.

The action estimation unit <NUM> may refer to the action history of the user <NUM> stored in the action history database 53b so as to estimate the action of the user <NUM>. For example, the action estimation unit <NUM> may choose an action with a high frequency so as to estimate the action of the user <NUM> among the actions stored in the action history.

An example of operation of the information processing device <NUM> is described below with reference to the flowchart shown in <FIG>.

In step S101, the user identification unit <NUM> identifies the user <NUM> riding in the vehicle <NUM> by use of the face image captured by the camera, the identification cipher transmitted from the intelligence key, and the like. The process proceeds to step S103, and the destination is then set by the user <NUM>.

The process proceeds to step S105, and the destination arrival determination unit <NUM> checks the positional information of the vehicle <NUM> acquired from the GPS receiver <NUM> with the positional information of the destination stored in the map database 53a so as to determine whether the vehicle <NUM> has arrived at the destination. When the vehicle <NUM> is determined to have arrived at the destination (YES in step S105), the process proceeds to step S107. When the vehicle <NUM> is determined not to have arrived at the destination yet (NO in step S105), the process is on standby.

In step S107, the action estimation unit <NUM> estimates the action of the user <NUM> taken before riding in the vehicle. In particular, the action estimation unit <NUM> checks the positional information of the vehicle <NUM> acquired from the GPS receiver <NUM> with the map database 53a, and acquires the POI at the current position (the parked position of the vehicle <NUM>). The action estimation unit <NUM> thus can recognize that the current position is the ramen shop <NUM> (refer to <FIG>) or the current position is the shopping mall <NUM> (refer to <FIG>). The action estimation unit <NUM> refers to the table data in which the attribute of the POI (the ramen shop or the shopping mall) is associated with the type of the action of the user <NUM>, and estimates the action of the user <NUM> taken before riding in the vehicle. The signal indicating the estimation result obtained by the action estimation unit <NUM> is output to the question selection unit <NUM>.

The process proceeds to step S109, and the question selection unit <NUM> when receiving the signal from the action estimation unit <NUM> refers to the action of the user <NUM> taken before riding in the vehicle and the question database 53c so as to choose the question to be put to the user <NUM>. The signal indicating the question chosen by the question selection unit <NUM> is output to the question output unit <NUM>.

The process proceeds to step S111, and the ride detection unit <NUM> detects the ride of the user <NUM> in accordance with the signal acquired from the sensor group <NUM>. When the ride of the user is detected by the ride detection unit <NUM>, the signal indicating the detection of the ride of the use <NUM> is output to the question output unit <NUM>.

The process proceeds to step S113, and the question output unit <NUM>, when receiving the signal indicating the detection of the ride of the user <NUM>, asks the user <NUM> the question chosen by the question selection unit <NUM> by voice via the speaker <NUM>.

When the answer is input by the voice via the microphone <NUM> (YES in step S115), the process proceeds to step S117. When the answer is not returned from the user <NUM> (NO in step S115), the process is on standby. The means of obtaining the answer from the user <NUM> is not limited to the voice, and the answer from the user <NUM> may be obtained through the operation on the touch panel.

In step S117, the voice analysis unit <NUM> analyzes the voice data of the user <NUM>. The action history update unit <NUM> associates the positional information of the vehicle <NUM> with the action that the user has taken at the POI in accordance with the analyzed result obtained by the voice analysis unit <NUM>, and stores the associated information in the action history database 53b. The action history update unit <NUM> may associate the POI with the action that the user <NUM> has taken at the POI and store the associated information in the action history database 53b.

As described above, the information processing device <NUM> according to the present embodiment can achieve the following operational effects.

The information processing device <NUM> includes the controller <NUM>, the ride detection device (the sensor group <NUM>) that detects the ride of the user <NUM> in the vehicle <NUM>, the storage device (the action history database 53b) that stores the action data of the user <NUM>, the output device (the speaker <NUM>) that outputs the question data for requesting the answer from the user <NUM>, and the input device (the microphone <NUM>) that receives the input from the user <NUM>.

The controller <NUM>, when detecting the ride of the user <NUM> by the signal obtained from the ride detection device, outputs the output data including at least the question regarding the action of the user <NUM> taken before riding in the vehicle <NUM> from the output device in accordance with the positional information of the vehicle <NUM>. The controller <NUM> acquires the answer to the question from the user <NUM> as the input data via the input device. The controller <NUM> associates the input data with the positional information of the vehicle <NUM> or the POI, and stores the associated information in the storage device. The controller <NUM> thus can store the action of the user <NUM> taken at the POI with a high accuracy.

The controller <NUM> determines the question depending on the attribute of the POI determined in accordance with the positional information of the vehicle <NUM>. Since the vehicle <NUM> in the case illustrated in <FIG> is parked in the parking space at the ramen shop <NUM>, the attribute of the POI is determined to be the ramen shop <NUM> in accordance with the positional information of the vehicle <NUM>. The storage device (the question database 53c) stores the attribute of the POI (the ramen shop) in association with the question regarding the action of the user (taking a meal) at the POI, so that the controller <NUM> can determine the question in accordance with the attribute of the POI.

The controller <NUM> may estimate the action of the user <NUM> taken before riding in the vehicle in accordance with the action history stored in the action history database 53b or the attribute of the POI. When the attribute of the POI is the ramen shop as illustrated in <FIG>, the controller <NUM> can estimate that the action of the user taken before riding in the vehicle is to take a meal. The controller <NUM> may determine the action having a higher frequency as the estimated action of the user <NUM> among the actions stored in the action history. The controller <NUM> thus can estimate the action of the user <NUM> taken before riding in the vehicle with a high accuracy.

The controller <NUM> may acquire the data regarding the action plan of the user <NUM> (the schedule data of the user <NUM>) through the communication, and refer to the schedule data so as to determine the question. The controller <NUM> thus can determine the appropriate question if there are several types of actions that the user would take at the POI, and can store the action of the user <NUM> at the POI with a high accuracy. The term "communication" as used herein refers to the communication with the computer <NUM> (the server) or the communication with the terminal that the user <NUM> holds.

The input data is data converted from the voice of the user <NUM>, or data generated through the operation made by the user <NUM>. The user <NUM> uses the microphone <NUM> or the touch panel as a means for answering the question. The user <NUM> using the microphone <NUM> or the touch panel can answer the question more easily.

The controller <NUM> detects the user <NUM> getting off the vehicle <NUM> after arriving at the destination set by the user <NUM>, and then outputs the question when detecting the user <NUM> riding in the vehicle <NUM>. The controller <NUM> thus can ask the user <NUM> the question at an appropriate timing.

The POI may be any of a POI located around the positional information of the vehicle <NUM>, a destination set through the navigation device equipped in the vehicle <NUM>, a POI included in the data regarding the action plan of the user <NUM> acquired through the communication, or a POI included in the data regarding the action history of the user <NUM> before riding in the vehicle acquired through the communication.

A modified example of the present embodiment is described below.

As illustrated in <FIG>, an information processing device <NUM> according to the modified example further includes an accuracy calculation unit <NUM>. The accuracy calculation unit <NUM> calculates an accuracy indicating likelihood (probability) of the action of the user <NUM> estimated by the action estimation unit <NUM>. The estimation accuracy may be evaluated according to a three-grade level of a low level, a middle level, and a high level, or may be evaluated according to a probability (<NUM>% to <NUM>%).

An example of a method of storing the action that the user <NUM> has taken at a POI is described below with reference to <FIG>.

<FIG> shows the attribute of the POI illustrating a restaurant, a shopping mall, and a class. The term "class" as used in the present embodiment includes a swimming class a dance class, and a flower-arrangement class. The type of the action that the user takes at a restaurant includes taking a meal and having a meeting. The type of the action that the user takes at a shopping mall includes a taking a meal, doing shopping, waiting for someone, having a haircut, and taking exercise, as described above. The type of the action that the user takes at a class includes taking exercise and taking a lesson.

As described above, since the attribute of the POI is associated with the type of the action of the user, the action estimation unit <NUM> estimates the action of the user <NUM> from the type of the action described above, as shown in <FIG>. The estimated action of the user <NUM> at the shopping mall in the case shown in <FIG> is any of taking a meal, doing shopping, and waiting for someone for illustration purposes.

While the accuracy of the action estimated by the action estimation unit <NUM> (also simply referred to below as an "estimated action) needs to be high when there are several possible actions to be estimated as shown in <FIG>, the accuracy of the estimation is not always high. The reason for requiring the high accuracy for the estimation of the action is that the question put to the user <NUM> would be an inappropriate question if the estimated action is wrong. The action of the user, however, is not always estimated with a high accuracy, as described above.

To deal with this, the information processing device <NUM> according to the modified example calculates the accuracy of the estimated action, and determines the question depending on the calculated accuracy. The action history of the user <NUM> stored in the action history database 53b is used so as to calculate the accuracy of the estimated action. The modified example is illustrated below with two cases in which there is no action history of the user <NUM> stored in the action history database 53b and in which the action history of the user <NUM> stored in the action history database 53b includes actions taken with a predetermined number of times (for example, ten times).

First, the case in which there is no action history of the user <NUM> stored in the action history database 53b is described below.

When the destination of the user <NUM> is a restaurant, the action estimated by the action estimation unit <NUM> is either to take a meal or to have a meeting, as shown in <FIG>. The action estimation unit <NUM> then estimates the action of the user that is either to take a meal or to have a meeting. The action estimation unit <NUM> may estimate, but not necessarily, the action of the user that is typically presumed to have a high probability, for example. An action having a high probability typically taken at a restaurant is to take a meal. This is also stored in the table data. The action estimation unit <NUM> thus may refer to the table data so as to estimate that the action of the user <NUM> is to take a meal when the attribute of the POI is a restaurant.

Alternatively, the action estimation unit <NUM> may refer to the action history of the user <NUM> stored in the action history database 53b so as to estimate the action of the user <NUM>. For example, the action estimation unit <NUM> may choose an action having a high frequency so as to estimate the action of the user <NUM> among the actions stored in the action history. This method cannot be used when there is no action history.

The accuracy calculation unit <NUM> calculates the accuracy of the action of taking a meal estimated by the action estimation unit <NUM>. The accuracy calculation unit <NUM> first refers to the action history database 53b, and acquires the action history of the user <NUM> taken at the restaurant. Since there is no action history of the user <NUM> at the restaurant in this case, the accuracy calculation unit <NUM> determines that the user <NUM> comes to the restaurant for the first time. The accuracy calculation unit <NUM> in this case determines that the probability is low that the action that the user <NUM> has taken at the restaurant is to take a meal. The reason for this is that the determination of whether the action that the user <NUM> takes at the restaurant is to take a meal or have a meeting is difficult. Although the action of taking a meal is typically presumed to have a higher probability, there is still a probability that the action of the user is to have a meeting. The accuracy calculation unit <NUM> then calculates that the accuracy of the action of taking a meal estimated by the action estimation unit <NUM> is low. A signal indicating the calculation result is then output to the question selection unit <NUM>.

The question selection unit <NUM> when receiving the signal from the accuracy calculation unit <NUM> recognizes that the accuracy of the action of taking a meal estimated by the action estimation unit <NUM> is low. This recognition allows the question election unit <NUM> to avoid choosing the question regarding a meal and choose a question for asking the action of the user <NUM> itself. The reason for choosing the question for asking the action of the user <NUM> itself is to accumulate the action history in preparation for a case in which the user <NUM> would come to the restaurant again in the future.

An example of the question for asking the action itself is a question asking "What did you do there?", as shown in <FIG>. The answer to the question from the user <NUM> is associated with the positional information of the vehicle <NUM> or the POI, and is stored in the action history database 53b. Asking the action of the user <NUM> itself when there is no action history of the user <NUM> stored in the action history database 53b can accumulate the action history in preparation for a case in which the user <NUM> would come to the restaurant again in the future. Asking the action of the user <NUM> itself can also avoid asking an inappropriate question.

Next, the case in which the action history of the user <NUM> stored in the action history database 53b includes the actions taken with a predetermined number of times (for example, ten times) is described below.

As in the case in which there is no action history of the user <NUM> stored in the action history database 53b, the destination of the user <NUM> in the following case is assumed to be a restaurant. The accuracy calculation unit <NUM> refers to the action history database 53b, and acquires the action history of the user <NUM> taken at the restaurant. A case is assumed below in which ten times of the actions in the action history include seven times of the actions of taking a meal and three times of the actions of having a meeting. Since the number of times of the actions of taking a meal is greater than the number of times of the actions of having a meeting, the accuracy calculation unit <NUM> calculates to determine that the probability corresponds to the middle level that the action of the user <NUM> taken at the restaurant is to take a meal. In other words, the accuracy calculation unit <NUM> calculates to determine that the accuracy of the action of taking a meal estimated by the action estimation unit <NUM> corresponds to the middle level. A signal indicating the calculation result is then output to the question selection unit <NUM>. The accuracy of the action of taking a meal estimated by the action estimation unit <NUM> can be calculated as a probability. In the case in which ten times of the actions in the action history include seven times of the actions of taking a meal and three times of the actions of having a meeting, the probability can be calculated to be <NUM>% that the action of the user <NUM> taken at the restaurant is to take a meal. The phrase "the accuracy of the action estimated by the action estimation unit <NUM> corresponds to the middle level" conforms to about <NUM>% in terms of the probability.

The question selection unit <NUM> when receiving the signal from the accuracy calculation unit <NUM> determines that the probability that the action of the user <NUM> taken at the restaurant is to take a meal is high, and then chooses the question regarding a meal. Examples of the question regarding a meal in this case include a question asking "Did you eat something?" and a question asking "Did you drink something?", as shown in <FIG>. The answer to the question from the user <NUM> is associated with the positional information of the vehicle <NUM> or the POI, and is stored in the action history database 53b. When ten times of the actions in the action history include three times of the actions of taking a meal and seven times of the actions of having a meeting, the question selection unit <NUM> chooses the question regarding a meeting. Examples of the question regarding a meeting in this case include a question asking "Are you having a meeting for something?" and a question asking "Are you having a meeting with someone?", as shown in <FIG>.

While the above case is illustrated with the predetermined number of times that is ten times, the predetermined number of times is not limited to ten times. The greater number of times of the actions of the user <NUM> in the action history stored in the action history database 53b contributes to an improvement in the accuracy of the estimated action. The accuracy of the estimated action fluctuates as the number of times of the actions of the user <NUM> in the action history is smaller. When the number of times of the actions of the user <NUM> in the action history stored in the action history database 53b is less than five times, the number of times of the actions of the user <NUM> in the action history stored in the action history database 53b may be considered to be zero.

Similarly, when the destination of the user <NUM> is a shopping mall, and the accuracy of the action of the user <NUM> (taking a meal) estimated by the action estimation unit <NUM> is calculated to be the middle level, the question selection unit <NUM> chooses the question regarding a meal. The question regarding a meal is the same as described above, and explanations are not repeated below. When the accuracy of the action of the user <NUM> (doing shopping) estimated by the action estimation unit <NUM> is calculated to be the middle level, the question selection unit <NUM> chooses the question regarding shopping. Examples of the question regarding shopping in this case include a question asking "Did you buy something?" and a question asking "Did you find something?", as shown in <FIG>. When the accuracy of the action of the user <NUM> (waiting for someone) estimated by the action estimation unit <NUM> is calculated to be the middle level, the question selection unit <NUM> chooses the question regarding an appointment. Examples of the question regarding an appointment in this case include a question asking "Are you waiting for someone?" and a question asking "Are you meeting Mr. ?", as shown in <FIG>.

Similarly, when the destination of the user <NUM> is a class, and the accuracy of the action of the user <NUM> (taking exercise) estimated by the action estimation unit <NUM> is calculated to be the middle level, the question selection unit <NUM> chooses the question regarding exercise. Examples of the question regarding exercise include a question asking "Did you take exercise?" and a question asking "Swimming? Yoga? Dancing?", as shown in <FIG>. When the accuracy of the action of the user <NUM> (taking a lesson) estimated by the action estimation unit <NUM> is calculated to be the middle level, the question selection unit <NUM> chooses the question regarding a lesson. Examples of the question regarding a lesson include a question asking "Taking a lesson?" a question asking "Flower arrangement? Tea ceremony? Calligraphy?", as shown in <FIG>.

Calculating the accuracy of the action of the user <NUM> estimated by the action estimation unit <NUM> and determining the question depending on the calculated accuracy as described above can avoid asking an inappropriate question.

A case is assumed below in which ten times of the actions in the action history at the restaurant include ten times of the actions of taking a meal and no actions of having a meeting. Since all of the actions in the action history are to take a meal, the accuracy calculation unit <NUM> calculates to determine that the probability is high that the action of the user <NUM> taken at the restaurant is to take a meal. In other words, the accuracy calculation unit <NUM> calculates to determine that the accuracy of the action of the user <NUM> (taking a meal) estimated by the action estimation unit <NUM> is high. A signal indicating the calculation result is then output to the question selection unit <NUM>. The probability in this case can be calculated to be <NUM>% that the action of the user <NUM> taken at the restaurant is to take a meal.

The question election unit <NUM> when receiving the signal from the accuracy calculation unit <NUM> determines that the probability that the action of the user <NUM> taken at the restaurant is to take a meal is quite high, and then chooses, from the questions regarding a meal, a question delving deeply more than the question asked when the accuracy corresponds to the middle level. Examples of the question delving deeply in this case include a question asking "Was it good?" and a question asking "Do you want to come to have it again?", as shown in <FIG>. The voice analysis unit <NUM> can classify the answer to this question from the user <NUM> into a positive answer and a negative answer. The action history update unit <NUM> can associate the classified result with the POI as classification data so as to store the data into the action history database 53b. Classifying the answer of the user <NUM> into a positive answer and a negative answer can ask the user <NUM> the question delving more deeply next time.

When ten times of the actions in the action history include no actions of taking a meal and ten times of the actions of having a meeting, the question selection unit <NUM> chooses the question delving deeply more than the question asked when the accuracy corresponds to the middle level among the questions regarding a meeting. Examples of the question delving deeply in this case include a question asking "Finished well?" and a question asking "Was Mr. fine?", as shown in <FIG>.

Similarly, when the destination of the user <NUM> is a shopping mall, and the accuracy of the action of the user <NUM> (taking a meal) estimated by the action estimation unit <NUM> is calculated to be high, the question selection unit <NUM> chooses the question delving deeply more than the question asked when the accuracy corresponds to the middle level among the questions regarding a meal. The question delving deeply is the same as described above, and explanations are not repeated below. When the accuracy of the action of the user <NUM> (doing shopping) estimated by the action estimation unit <NUM> is calculated to be high, the question selection unit <NUM> chooses the question delving deeply more than the question asked when the accuracy corresponds to the middle level among the questions regarding shopping. Examples of the question delving deeply in this case include a question asking "Did you find something reasonable?" and a question asking "Could you buy something good?", as shown in <FIG>. When the accuracy of the action of the user <NUM> (waiting for someone) estimated by the action estimation unit <NUM> is calculated to be high, the question selection unit <NUM> chooses the question delving deeply more than the question asked when the accuracy corresponds to the middle level among the questions regarding an appointment. Examples of the question delving deeply in this case include a question asking "Meet all right?" and a question asking "Was Mr. fine?", as shown in <FIG>.

Similarly, when the destination of the user <NUM> is a class, and the accuracy of the action of the user <NUM> (taking exercise) estimated by the action estimation unit <NUM> is calculated to be high, the question selection unit <NUM> chooses the question delving deeply more than the question asked when the accuracy corresponds to the middle level among the questions regarding exercise. Examples of the question delving deeply in this case include a question asking "Sweat well?" and a question asking "Refresh yourself?", as shown in <FIG>. When the accuracy of the action of the user <NUM> (taking a lesson) estimated by the action estimation unit <NUM> is calculated to be high, the question selection unit <NUM> chooses the question delving deeply more than the question asked when the accuracy corresponds to the middle level among the questions regarding a lesson. Examples of the question delving deeply in this case include a question asking "Make progress?" and a question asking "Spiritually awakened?", as shown in <FIG>.

Calculating the accuracy of the action of the user <NUM> estimated by the action estimation unit <NUM> and determining the question depending on the calculated accuracy as described above can stimulate the conversations with the user <NUM>.

The accuracy calculation unit <NUM> may refers to the schedule data of the user <NUM> when calculating the accuracy of the estimated action. Referring to the schedule data can improve the accuracy to be calculated. When there is no action history of the user <NUM> stored in the action history database 53b, the accuracy of the estimated action (taking a meal) is determined to be low, as described above. The action estimation unit <NUM> in this case refers to the schedule data when including the information "taking a meal at a restaurant", so as to calculate the accuracy of the estimated action (taking a meal) to be the middle level or to be high.

Next, an example of a structure of the questions stored in the question database 53c is described below with reference to <FIG>.

As illustrated in <FIG>, the questions are classified into several layers (the first layer to the third layer), and are stored in the question database 53c. The question selection unit <NUM> chooses a question belonging to any of the layers depending on the accuracy of the estimated question calculated by the accuracy calculation unit <NUM>. In particular, the question selectin unit <NUM> chooses a question included in the first layer when the accuracy of the estimated action is smaller than a first predetermined value. As shown in <FIG>, the question included in the first layer is to ask the action of the user <NUM> itself.

The question selection unit <NUM> chooses a question included in the second layer when the accuracy of the estimated action is greater than or equal to the first predetermined value and less than or equal to a second predetermined value (the first predetermined value < the second predetermined value). As shown in <FIG>, the question included in the second layer is to confirm the estimated action.

The question selection unit <NUM> chooses a question included in the third layer when the accuracy of the estimated action is greater than the second predetermined value. The question included in the third layer is to request the user <NUM> to answer an impression or an evaluation regarding the action taken before riding in the vehicle, as shown in <FIG>. The third layer may include a question for requesting the user <NUM> to answer an impression or an evaluation regarding the POI. Examples of the question for requesting the user <NUM> to answer the impression or the evaluation regarding the action taken before riding in the vehicle include a question asking "Was it good?" and a question asking "Do you want to come to have it again?" when the action taken before riding in the vehicle is to take a meal, as shown in <FIG>. An example of the question for requesting the user <NUM> to answer the impression or the evaluation regarding the POI is a question asking "Was the inside clean enough?" when the attribute of the POI is a restaurant, as shown in <FIG>.

The probability calculated by the accuracy calculation unit <NUM> may be used as the first predetermined value and the second predetermined value shown in <FIG>. For example, the first predetermined value may be set to <NUM>%, and the second predetermined value may be set to <NUM>%. The first layer corresponds to the state in which the estimated accuracy is low in the case shown in <FIG>. The second layer corresponds to the state in which the estimated accuracy is at the middle level in the case shown in <FIG>. The third layer corresponds to the state in which the estimated accuracy is high in the case shown in <FIG>.

The question delves more deeply as the classification proceeds from the first layer to the third layer, namely, as the accuracy of the estimated action is higher. The action of the user <NUM> taken at the POI is stored in the action history database 53a so as to accumulate the action history. As the action history is accumulated more, the accuracy of the estimated action calculated by the accuracy calculation unit <NUM> is higher, and the question delves more deeply. The increase in the accumulation of the action history can stimulate the conversations with the user <NUM>. While the questions shown in <FIG> are classified into the three layers, the classification is not limited to this case. The questions may delve more deeply so as to be classified further into the fourth layer and the fifth layer. The respective layers are set depending on the degree of abstraction or specificity of the questions allotted to the respective layers. In other words, the abstraction of the question is increased as the layer is higher, and the specificity of the question is increased as the layer is lower.

Next, a relationship between the action history stored in the action history database 53b and the accuracy of the estimated action calculated by the accuracy calculation unit <NUM> is described below with reference to <FIG>.

As described above, the accuracy of the estimated action is lower as the action history stored in the action history database 53b is smaller. The accuracy of the estimated action is higher as the action history stored in the action history database 53b is greater.

Similarly, the accuracy of the estimated action is lower as the probability of the action history stored in the action history database 53b is lower. The accuracy of the estimated action is higher as the probability of the action history stored in the action history database 53b is greater. The above explanations are made with the case in which the probability that the action of the user <NUM> taken at the restaurant is to take a meal is <NUM>% when ten times of the actions in the action history include seven times of the actions of taking a meal and three times of the actions of having a meeting. The probability of <NUM>% corresponds to the probability of the action history.

The accuracy calculation unit <NUM> does not necessarily need to refer to the action history stored in the action history database 53b upon the calculation of the accuracy of the estimated action. It is difficult to estimate the action that the user <NUM> has taken at the POI when the attribute is a shopping mall as described above, since there are a lot of possible actions that the user <NUM> would take. The accuracy of the estimated action is thus decreased. When the attribute of the POI is a ramen shop, the accuracy of the estimated action is increased since the action that the user <NUM> would take can be specified. When the attribute of the POI is a supermarket, the action that the user <NUM> would take can be substantially limited to shopping, and the accuracy of the estimated action is thus determined to correspond to the middle level. The accuracy calculation unit <NUM> thus can calculate the accuracy of the estimated action only by use of the attribute of the POI without referring to the action history stored in the action history database 53b.

An example of operation of the information processing device <NUM> according to the modified example is described below with reference to the flowchart shown in <FIG>. The processing in steps S201 to <NUM> and <NUM> to <NUM> is the same as the processing in steps S101 to S107 and S111 to S117, and overlapping explanations are not repeated below.

In step S209, the accuracy calculation unit <NUM> calculates the accuracy indicating the likelihood (the probability) of the action of the user <NUM> estimated by the action estimation unit <NUM>.

The process proceeds to step S211, and the question selection unit <NUM> chooses the question belonging to any of the layers depending on the accuracy of the estimated action calculated by the accuracy calculation unit <NUM> (refer to <FIG>). In particular, the question selection unit <NUM> chooses the question for asking the action of the user <NUM> itself when the accuracy of the estimated action is smaller than the first predetermined value (step S213). The question selection unit <NUM> chooses the question for confirming the estimated action when the accuracy of the estimated action is greater than or equal to the first predetermined value and less than or equal to the second predetermined value (in step S215). The question selection unit <NUM> chooses the question for requesting the user <NUM> to answer the impression or the evaluation regarding the action taken before riding in the vehicle or the question for requesting the user <NUM> to answer the impression or the evaluation regarding the POI when the accuracy of the estimated action is greater than the second predetermined value (step S217).

As described above, the information processing device <NUM> according to the modified example can achieve the following operational effects.

The controller <NUM> calculates the accuracy of the estimated action in accordance with the action history stored in the action history database 53b or the attribute of the POI, and determines the question depending on the calculated accuracy of the estimated action. This can avoid asking an inappropriate question.

The questions are classified into the plural layers (refer to <FIG>). The controller <NUM> determines the question belonging to any of the layers as output data depending on the accuracy of the estimated action. The determination of the question depending on the accuracy of the estimated action can stimulate the conversations with the user <NUM>. The controller <NUM> may determine the abstraction of the question depending on the accuracy of the estimated action so as to define the determined question as the output data.

The output data includes the question for requesting the user <NUM> to answer the action taken before riding in the vehicle, the question for requesting the user <NUM> to answer the impression or the evaluation regarding the action taken before riding in the vehicle, and the question for requesting the user <NUM> to answer the impression or the evaluation regarding the POI. Using various kinds of questions can stimulate the conversations with the user <NUM>.

The controller <NUM> classifieds the answer of the user <NUM> into the positive answer and the negative answer, and associates the classified result with the POI to store the classification data in the action history database 53b. The classification of the answer of the user <NUM> into the positive answer and the negative answer can provide the question delving more deeply when asking the user <NUM> next time.

The controller <NUM> may output the classification data associated with the POI in accordance with the positional information of the vehicle <NUM>. In a case in which the answer of the user <NUM> regarding a meal at the ramen shop <NUM> is assumed to be the positive answer indicating "It was good", the controller <NUM> may output the voice saying "The ramen was good, wasn't it?" when the vehicle <NUM> is traveling around the ramen shop <NUM>. This can stimulate the conversations with the user <NUM>.

The respective functions described in the embodiment above can be implemented by one or plural processing circuits. The respective processing circuits include a programed processing device, such as a processing circuit including an electrical circuit. The respective processing circuits also include an application-specific integrated circuit (ASIC) configured to execute the functions described above and a device such as a conventional circuit component.

While the present invention has been described above by reference to the embodiment, it should be understood that the present invention is not intended to be limited to the descriptions and the drawings composing part of this disclosure. Various alternative embodiments, examples, and technical applications will be apparent to those skilled in the art according to this disclosure.

<FIG> illustrates the case in which the vehicle <NUM> is parked in the parking space at the ramen shop <NUM>, and the positional information of the vehicle <NUM> thus can be presumed to conform to the positional information of the POI, as described above. However, the destination does not always provide a parking space. When there is no parking space at the destination, the user <NUM> would try to park the vehicle <NUM> in a coin-operated parking lot around the destination, and then walk toward the destination from the parking lot, as illustrated in <FIG>, for example. The positional information of the vehicle <NUM> in this case does not conform to the positional information of the destination (the positional information of the POI). The positional information of the vehicle <NUM> associated with the action of the user <NUM> taken before riding in the vehicle <NUM> and stored together thus cannot be used as the effective data.

To deal with this, the information processing device <NUM> may associate the positional information of the terminal held by the user (referred to as a "user's terminal") with the action of the user <NUM> taken before riding in the vehicle, and store the information in the action history database 53b. In particular, the information processing device <NUM> acquires the positional information of the user's terminal through the communication when the user arrives at the destination. The information processing device <NUM> is herein presumed to be configured to be able to communicate with the user's terminal. For example, the information processing device <NUM> includes a receiver that receives data transmitted from the user's terminal. The information processing device <NUM> refers to information in a guiding application installed in the user's terminal, so as to determine whether the user <NUM> has arrived at the destination.

Claim 1:
An information processing device (<NUM>) comprising:
a controller (<NUM>);
a ride detection device (<NUM>) configured to detect a ride of a user (<NUM>) in a vehicle (<NUM>);
a storage device (<NUM>) configured to store data regarding points of interest, or POIs, including, for each POI, positional information of the POI and an attribute of the POI in association with an action type in which an action that the user (<NUM>) can take at the POI is classified, and to store action history data of the user (<NUM>);
an output device (<NUM>) configured to output question data for requesting an answer from the user (<NUM>); and
an input device (<NUM>) configured to receive an input from the user (<NUM>),
the controller (<NUM>) being configured to:
refer to the data stored in the storage device (<NUM>) to determine an attribute of a POI in accordance with a positional information of the vehicle (<NUM>) when detecting the ride of the user (<NUM>) in the vehicle (<NUM>) based on a signal acquired from the ride detection device (<NUM>);
estimate an action type of an action that the user (<NUM>) may have taken at the POI before riding in the vehicle depending on the action type associated with the determined attribute of the POI;
calculate an accuracy indicating probability of the estimated action type, depending on the stored action history data and/or the determined attribute of the POI;
determine a question regarding the action of the user (<NUM>) taken before riding in the vehicle (<NUM>) depending on the calculated accuracy of the estimated action type;
output, from the output device (<NUM>), output data including at least the determined question;
acquire an answer to the question from the user (<NUM>) as input data via the input device (<NUM>); and
associate the input data with the positional information of the vehicle (<NUM>) or the POI to store the associated data in the storage device (<NUM>).