Patent ID: 12233714

DESCRIPTION OF EMBODIMENT

An embodiment of the technology disclosed in the present description will hereinafter be described in detail with reference to the drawings.

A. Configuration of Vehicle Control System

FIG.1is a block diagram depicting a configuration example of a general function of a vehicle control system100which is an example of a mobile body control system to which the present technology is applicable.

Note that a vehicle on which the vehicle control system100is provided will be referred to as an own vehicle or an own car in a case of the necessity of distinction of this vehicle from other vehicles.

The vehicle control system100includes an input unit101, a data acquisition unit102, a communication unit103, an in-vehicle apparatus104, an output control unit105, an output unit106, a drive control unit107, a drive system108, a body control unit109, a body system110, a storage unit111, and an autonomous driving control unit112. The input unit101, the data acquisition unit102, the communication unit103, the output control unit105, the drive control unit107, the body control unit109, the storage unit111, and the autonomous driving control unit112are connected to one another via a communication network121. For example, the communication network121includes an in-vehicle communication network, a bus, or the like in conformity with any standard such as a CAN (Controller Area Network), a LIN (Local Interconnect Network), a LAN (Local Area Network), and FlexRay (registered trademark). Note that the respective units of the vehicle control system100are directly connected to one another without using the communication network121in some cases.

In addition, it is assumed that description of the communication network121will hereinafter be omitted in a case where the respective units of the vehicle control system100communicate with one another via the communication network121. For example, in a case where the input unit101and the autonomous driving control unit112communicate with one another via the communication network121, this communication will simply be described as communication between the input unit101and the autonomous driving control unit112.

The input unit101includes a device used by a person on board to input various types of data, instructions, and the like. For example, the input unit101includes an operation device such as a touch panel, a button, a microphone, a switch, and a lever, an operation device capable of inputting voices, gestures, and the like by using a method other than a manual operation, or other devices. Further, for example, the input unit101may be a remote controller using infrared light or other radio waves, or an external connection apparatus such as a mobile apparatus and a wearable apparatus handling operation of the vehicle control system100. The input unit101generates input signals on the basis of data, instructions, or the like input from the person on board, and supplies the generated input signals to the respective units of the vehicle control system100.

The data acquisition unit102includes various types of sensors for acquiring data used for processing by the vehicle control system100, for example, and supplies acquired data to the respective units of the vehicle control system100.

For example, the data acquisition unit102includes various types of sensors for detecting a state or the like of the own vehicle. Specifically, for example, the data acquisition unit102includes a gyro sensor, an acceleration sensor, an inertial measurement unit (IMU), a sensor for detecting an operation amount of an acceleration pedal, an operation amount of a brake pedal, a steering angle of a steering wheel, an engine speed, a motor speed, a rotation speed of wheels, and the like.

Further, for example, the data acquisition unit102includes various types of sensors for detecting external information associated with the own vehicle. Specifically, for example, the data acquisition unit102includes an imaging device such as a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, and other types of cameras. Further, for example, the data acquisition unit102includes an environmental sensor for detecting weather, meteorology, or the like, and an ambient information detection sensor for detecting an object around the own vehicle. For example, the environmental sensor includes a raindrop sensor, a fog sensor, a sunlight sensor, a snow sensor, or the like. For example, the ambient information detection sensor includes an ultrasonic sensor, a radar, a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), a sonar, or the like.

Further, for example, the data acquisition unit102includes various types of sensors for detecting a current position of the own vehicle. Specifically, for example, the data acquisition unit102includes a GNSS (Global Navigation Satellite System) receiver for receiving GNSS signals from GNSS satellites, and the like.

In addition, for example, the data acquisition unit102includes various types of sensors for detecting information inside the vehicle. Specifically, for example, the data acquisition unit102includes an imaging device for imaging a driver, a biosensor for detecting biological information associated with the driver, a microphone for collecting voices in the vehicle interior, and the like. For example, the biosensor is provided on a seat surface, the steering wheel, or the like to detect biological information associated with a person on board sitting on a seat or the driver holding the steering wheel.

The communication unit103communicates with the in-vehicle apparatus104and various apparatuses outside the vehicle, servers, base stations, and the like, and transmits data supplied from the respective units of the vehicle control system100to supply received data to the respective units of the vehicle control system100. Note that a communication protocol supported by the communication unit103is not limited to any particular protocol, and that the communication unit103is allowed to support a plurality of types of communication protocols.

For example, the communication unit103wirelessly communicates with the in-vehicle apparatus104by using a wireless LAN, Bluetooth (registered trademark), NFC (Near Field Communication), a WUSB (Wireless USB), or the like. Further, for example, the communication unit103communicates with the in-vehicle apparatus104by wire via a not-depicted connection terminal (and a cable if necessary) by using a USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), MHL (Mobile High-definition Link), or the like.

Further, for example, the communication unit103communicates with an apparatus (e.g., an application server or a control server) existing on an external network (e.g., the Internet, a cloud network, and a proprietary network of a business operator) via a base station or an access point. In addition, for example, the communication unit103communicates with a terminal existing near the own vehicle (e.g., a terminal carried by a pedestrian or equipped in a store, or an MTC (Machine Type Communication) terminal) by using a P2P (Peer To Peer) technology. Further, for example, the communication unit103performs V2X communication such as vehicle to vehicle (Vehicle to Vehicle) communication, vehicle to infrastructure (Vehicle to Infrastructure) communication, vehicle to home (Vehicle to Home) communication, and vehicle to pedestrian (Vehicle to Pedestrian) communication. Further, for example, the communication unit103includes a beacon reception unit to receive radio waves or electromagnetic waves transmitted from a wireless station or the like installed on a road and to acquire information such as a current position, a traffic jam, a traffic restriction, and a required length of time.

For example, the in-vehicle apparatus104includes a mobile apparatus or a wearable apparatus carried by the person on board, an information apparatus brought into or attached to the own vehicle, a navigation device which searches for a route to any destination, and the like.

The output control unit105controls output of various types of information to the person having boarded the own vehicle or to the outside of the vehicle. For example, the output control unit105generates an output signal containing at least either visual information (e.g., image data) or auditory information (e.g., audio data), and supplies the generated output signal to the output unit106to control output of the visual information and the auditory information from the output unit106. Specifically, for example, the output control unit105generates a bird's eye image, a panorama image, or the like by synthesizing image data captured by different imaging devices of the data acquisition unit102, and supplies output signals containing the generated image to the output unit106. Further, for example, the output control unit105generates audio data containing a warning sound, a warning message, or the like for a danger such as a collision, a contact, and entrance into a dangerous zone, and supplies output signals containing the generated audio data to the output unit106.

The output unit106includes a device capable of outputting visual information or auditory information to the person having boarded the own vehicle or to the outside of the vehicle. For example, the output unit106includes a display device, an instrument panel, an audio speaker, a headphone, a wearable device such as a glass-type display worn by the person on board, a projector, a lamp, or the like. The display device included in the output unit106may be a device displaying visual information within a visual field of the driver, such as a head-up display, a transmissive display, and a device having an AR (Augmented Reality) display function, other than the device having an ordinary display.

The drive control unit107controls the drive system108by generating various types of control signals and supplying the control signals to the drive system108. Further, the drive control unit107supplies control signals to respective units other than the drive system108to give a notification of a control state of the drive system108as necessary, for example.

The drive system108includes various types of devices associated with a drive system of the own vehicle. For example, the drive system108includes a driving force generation device for generating a driving force for driving an internal combustion engine, a driving motor, or the like, a driving force transmission mechanism for transmitting a driving force to wheels, a steering mechanism for adjusting a steering angle, a braking device for generating a braking force, ABS (Antilock Brake System), ESC (Electronic Stability Control), an electric power steering device, and the like.

The body control unit109generates various types of control signals and supplies the control signals to the body system110to control the body system110. Further, the body control unit109supplies control signals to respective units other than the body system110to give a notification of a control state of the body system110as necessary, for example.

The body system110includes various types of devices associated with a body system equipped on a vehicle body. For example, the body system110includes a keyless entry system, a smart key system, a power window device, a power seat, a steering wheel, an air conditioning device, various types of lamps (e.g., head lamps, back lamps, brake lamps, direction indicators, and fog lamps), and others.

For example, the storage unit111includes a magnetic storage device, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like, such as a ROM (Read Only Memory), a RAM (Random Access Memory), or an HDD (Hard Disc Drive). The storage unit111stores various programs, data, or the like used by the respective units of the vehicle control system100. For example, the storage unit111stores such map data as a three-dimensional high-accuracy map such as a dynamic map, a global map having accuracy lower than that of the high-accuracy map but covering a wide area, and a local map containing information associated with surroundings of the own vehicle.

The autonomous driving control unit112performs control associated with autonomous driving, such as autonomous traveling and driving assistance. Specifically, for example, the autonomous driving control unit112performs cooperative control for a purpose of achieving functions of an ADAS (Advanced Driver Assistance System) including avoidance of a collision or easing of a shock of the own vehicle, following traveling based on a distance between vehicles, constant vehicle speed traveling, collision warning for the own vehicle, a lane departure warning for the own vehicle, or the like. Further, for example, the autonomous driving control unit112performs cooperative control for a purpose of achieving autonomous driving for allowing autonomous traveling without the necessity of operation by the driver, for example. The autonomous driving control unit112includes a detection unit131, a self-position estimation unit132, a situation analysis unit133, a planning unit134, and an action control unit135.

The detection unit131detects various types of information necessary for autonomous driving control. The detection unit131includes an exterior information detection unit141, an interior information detection unit142, and a vehicle state detection unit143.

The exterior information detection unit141performs a detection process for detecting information outside the own vehicle on the basis of data or signals received from the respective units of the vehicle control system100. For example, the exterior information detection unit141performs a detection process, a recognition process, and a tracking process for detecting, recognizing, and tracking a body around the own vehicle, and a detection process for detecting a distance to the body. Examples of the body corresponding to a detection target include a vehicle, a human, an obstacle, a structure, a road, a traffic light, a traffic sign, and a road marking. Further, for example, the exterior information detection unit141performs a detection process for detecting an environment around the own vehicle. Examples of the environment around the own vehicle corresponding to a detection target include weather, temperature, humidity, brightness, and a road surface state. The exterior information detection unit141supplies data indicating a result of the detection process to the self-position estimation unit132, a map analysis unit151, a traffic rule recognition unit152, and a situation recognition unit153of the situation analysis unit133, an emergency avoidance unit171of the action control unit135, and others.

The interior information detection unit142performs a detection process for detecting information inside the vehicle on the basis of data or signals received from the respective units of the vehicle control system100. For example, the interior information detection unit142performs an authentication process and a recognition process for authenticating and recognizing the driver, a detection process for detecting a state of the driver, a detection process for detecting the person on board, a detection process for detecting an environment inside the vehicle, and the like. Examples of the state of the driver corresponding to a detection target include a physical condition, a degree of alertness, a degree of concentration, a degree of fatigue, and a visual line direction. Examples of the environment inside the vehicle corresponding to a detection target include a temperature, humidity, brightness, and a smell. The interior information detection unit142supplies data indicating a result of the detection process to the situation recognition unit153of the situation analysis unit133, the emergency avoidance unit171of the action control unit135, and others.

The vehicle state detection unit143performs a detection process for detecting a state of the own vehicle on the basis of data or signals received from the respective units of the vehicle control system100. Examples of the state of the own vehicle corresponding to a detection target include a speed, an acceleration, a steering angle, the presence or absence and details of abnormality, a state of driving operation, a position and an inclination of a power seat, a state of a door lock, and states of other in-vehicle apparatuses. The vehicle state detection unit143supplies data indicating a result of the detection process to the situation recognition unit153of the situation analysis unit133, the emergency avoidance unit171of the action control unit135, and others.

The self-position estimation unit132performs an estimation process for estimating a position, a posture, and the like of the own vehicle on the basis of data or signals received from the respective units of the vehicle control system100, such as the exterior information detection unit141and the situation recognition unit153of the situation analysis unit133. Further, the self-position estimation unit132generates a local map used for estimation of the self-position (hereinafter referred to as a self-position estimation map) as necessary. For example, the self-position estimation map is a high-accuracy map using a technology such as SLAM (Simultaneous Localization and Mapping). The self-position estimation unit132supplies data indicating a result of the estimation process to the map analysis unit151, the traffic rule recognition unit152, and the situation recognition unit153of the situation analysis unit133and others. Further, the self-position estimation unit132stores the self-position estimation map in the storage unit111.

The situation analysis unit133performs an analysis process for analyzing situations of the own vehicle and surroundings. The situation analysis unit133includes the map analysis unit151, the traffic rule recognition unit152, the situation recognition unit153, and a situation prediction unit154.

The map analysis unit151performs an analysis process for analyzing various types of maps stored in the storage unit111by using data or signals received from the respective units of the vehicle control system100, such as the self-position estimation unit132and the exterior information detection unit141, as necessary, and constructs a map containing information necessary for an autonomous driving process. The map analysis unit151supplies the constructed map to the traffic rule recognition unit152, the situation recognition unit153, and the situation prediction unit154, and further to a route planning unit161, a behavior planning unit162, and an action planning unit163of the planning unit134, and others.

The traffic rule recognition unit152performs a recognition process for recognizing a traffic rule around the own vehicle on the basis of data or signals received from the respective units of the vehicle control system100, such as the self-position estimation unit132, the exterior information detection unit141, and the map analysis unit151. For example, a position and a state of a signal around the own vehicle, details of a traffic restriction around the vehicle, a travelable lane, and the like are recognized by the recognition process. The traffic rule recognition unit152supplies data indicating a result of the recognition process to the situation prediction unit154and others.

The situation recognition unit153performs a recognition process for recognizing a situation associated with the own vehicle on the basis of data or signals received from the respective units of the vehicle control system100, such as the self-position estimation unit132, the exterior information detection unit141, the interior information detection unit142, the vehicle state detection unit143, and the map analysis unit151. For example, the situation recognition unit153performs a recognition process for recognizing a situation of the own vehicle, a situation around the own vehicle, a situation of the driver of the own vehicle, and the like. Further, the situation recognition unit153generates a local map used for recognition of the situation around the own vehicle (hereinafter referred to as situation recognition map) as necessary. For example, the situation recognition map is an occupancy grid map (Occupancy Grid Map).

Examples of the situation of the own vehicle corresponding to a recognition target include a position, a posture, and movement (e.g., speed, acceleration, and moving direction) of the own vehicle and the presence or absence and details of abnormality. Examples of the situation around the own vehicle corresponding to a recognition target include a type and a position of a surrounding still body, a type, a position, and movement (e.g., speed, acceleration, and moving direction) of a surrounding dynamic body, a configuration of a surrounding road and a road surface state, ambient weather, ambient temperature, ambient humidity, and ambient brightness. Examples of the state of the driver corresponding to a detection target include a physical condition, a degree of alertness, a degree of concentration, a degree of fatigue, movement of a visual line, and a driving operation.

The situation recognition unit153supplies data indicating a result of the recognition process (including a situation recognition map as necessary) to the self-position estimation unit132, the situation prediction unit154, and others. Further, the situation recognition unit153stores the situation recognition map in the storage unit111.

The situation prediction unit154performs a prediction process for predicting a situation associated with the own vehicle on the basis of data or signals received from the respective units of the vehicle control system100, such as the map analysis unit151, the traffic rule recognition unit152, and the situation recognition unit153. For example, the situation prediction unit154performs a prediction process for predicting a situation of the own vehicle, a situation around the own vehicle, a situation of the driver, and the like.

Examples of the situation of the own vehicle corresponding to a prediction target include a behavior of the own vehicle, occurrence of abnormality, and a travelable distance. Examples of the situation around the own vehicle corresponding to a prediction target include a behavior of a dynamic body around the own vehicle, a state change of a signal, and a change of an environment such as weather. Examples of the situation of the driver corresponding to a prediction target include a behavior and a physical condition of the driver.

The situation prediction unit154supplies data indicating a result of the prediction process to the route planning unit161, the behavior planning unit162, and the action planning unit163of the planning unit134, and others, together with data received from the traffic rule recognition unit152and the situation recognition unit153.

The route planning unit161plans a route to a destination on the basis of data or signals received from the respective units of the vehicle control system100, such as the map analysis unit151and the situation prediction unit154. For example, the route planning unit161sets a route to a designated destination from a current position on the basis of a global map. Further, for example, the route planning unit161changes the route as appropriate on the basis of a situation such as a traffic jam, an accident, a traffic restriction, and construction, a physical condition of the driver, and the like. The route planning unit161supplies data indicating the planned route to the behavior planning unit162and others.

The behavior planning unit162plans a behavior of the own vehicle for safe traveling of the route planned by the route planning unit161within a planned period on the basis of data or signals received from the respective units of the vehicle control system100, such as the map analysis unit151and the situation prediction unit154. For example, the behavior planning unit162plans a departure, a stop, a traveling direction (e.g., forward movement, backward movement, left turn, right turn, and direction change), a traveling lane, a traveling speed, passing, and the like. The behavior planning unit162supplies data indicating the planned behavior of the own vehicle to the action planning unit163and others.

The action planning unit163plans an action of the own vehicle for achieving the behavior planned by the behavior planning unit162on the basis of data or signals received from the respective units of the vehicle control system100, such as the map analysis unit151and the situation prediction unit154. For example, the action planning unit163plans acceleration, deceleration, a traveling track, and the like. The action planning unit163supplies data indicating the planned action of the own vehicle to an acceleration and deceleration control unit172and a direction control unit173of the action control unit135, and others.

The action control unit135controls an action of the own vehicle. The action control unit135includes the emergency avoidance unit171, the acceleration and deceleration control unit172, and the direction control unit173.

The emergency avoidance unit171performs a detection process for detecting an emergency, such as a collision, a contact, entrance into a dangerous zone, abnormality of the driver, and abnormality of the vehicle, on the basis of a detection result obtained by the exterior information detection unit141, the interior information detection unit142, and the vehicle state detection unit143. In a case of detection of an emergency, the emergency avoidance unit171plans an action of the own vehicle for avoiding the emergency, such as a sudden stop and a sharp turn. The emergency avoidance unit171supplies data indicating the planned action of the own vehicle to the acceleration and deceleration control unit172, the direction control unit173, and others.

The acceleration and deceleration control unit172performs acceleration and deceleration control for achieving the action of the own vehicle planned by the action planning unit163or the emergency avoidance unit171. For example, the acceleration and deceleration control unit172calculates a control target value of a driving force generation device or a braking device for achieving a planned acceleration, a planned deceleration, or a planned sudden stop, and supplies a control command indicating a calculated control target value to the drive control unit107.

The direction control unit173performs direction control for achieving the action of the own vehicle planned by the action planning unit163or the emergency avoidance unit171. For example, the direction control unit173calculates a control target value of a steering mechanism for achieving a traveling track or a sharp turn planned by the action planning unit163or the emergency avoidance unit171, and supplies a control command indicating the calculated control target value to the drive control unit107.

B. Display Control by Vehicle

FIG.2schematically depicts a structure of a vehicle interior200of the vehicle. Note that a vehicle interior of a passenger car having four doors is assumed in the figure. Wall surfaces on the four sides of the vehicle interior200include a windshield201, a rear window202, and side windows203to206provided for respective seats (driver's seat, passenger seat, and left and right rear seats). Note that the vehicle includes a not-depicted sunroof in some cases.

According to the present embodiment, it is assumed that a device capable of switching between a transmissive state and a non-transmissive state, such as a liquid crystal shutter, or capable of gradually or successively switching transmittance is attached to at least a part of the windows201to206(preferably each of the windows201to206). Further, it is not assumed that switching between the transmissive state and the non-transmissive state or switching of the transmittance is uniformly achieved throughout the one entire window frame, but is assumed that the transmissive state and the non-transmissive state or switching of the transmittance is partially achievable for each region. Specifically, a liquid crystal shutter partially drivable for each grid of a predetermined size is attached to each of the windows201to206.

Each of the windows201to206of the vehicle is constituted not by transparent glass or the like, but by a combination of a flat display such as an LED (Loght Emittinf Diode) and an OLED (Organic Light Emitting Diode) and an in-vehicle camera each disposed on a window frame. A view through each of the windows201to206may be a video see-through image presented on the flat display. In such a case, the transmissive state may be considered as a state where a video see-through image is displayed, and the non-transmissive state may be considered as a state where a video see-through image is not displayed (or is replaced with another image).

In a case of a car to which autonomous driving or ADAS is applied, the driver or another occupant is not required to constantly gaze at an outside scenery through the windows of the four faces of the vehicle. Accordingly, even if some or all of the windows201to206is switched to the non-transmissive state, no problem is particularly caused in view of safe traveling of the vehicle.

Only a thing desired to be displayed is highlighted by switching only a partial region of the windows201to206to the transmissive state and performing a scenery masking process in such a manner as to limit a visual field. As a result, the visual line of the person on board is guidable to an object located outside the vehicle and visible through the transmissive region of the windows201to206.

For example, the visual line of the person on board is guidable to an object outside the vehicle, such as a scenery point, by forming a transmissive region in a part of the windows201to206such that the scenery point enters the visual field of the person on board when the traveling vehicle approaches the scenery point.

Needless to say, the person on board is capable of visually recognizing the scenery point if transmittance is set to 100% for the entire region of the windows201to206. In this case, however, other various things enter the visual field. Accordingly, time and effort are required to search for the scenery point in the wide visual field. The vehicle passes through the scenery point while the person on board is searching for the scenery point. In this case, the person on board may miss the scenery point.

Further, by switching a part or the entire region of the windows201to206to the non-transmissive state in a manner opposite to the above, a scenery viewed enough by the person on board, a thing uncomfortable for the person on board when viewed (e.g., an obstacle blocking a scenery), or the like can be made invisible for the person on board to maintain comfort of the vehicle interior.

In short, the vehicle interior200which includes the windows201to206each capable of switching between the transmissive state and the non-transmissive state or switching transmittance as depicted inFIG.2is adoptable as a display device capable of presenting information such as visual line guidance. Further, an operation of the display device of such a type is controllable, under the control by the vehicle control system100depicted inFIG.1.

An operation principle for controlling visual line guidance or scenery display with use of the vehicle interior200of the vehicle will be described with reference to the drawings.

FIG.3depicts a state where an object303corresponding to a superb view point appears in a traveling direction while a vehicle301is traveling straight at a constant speed on a straight road302.

In such a case, the object303is allowed to enter a visual field range of the person on board through the windshield201as depicted inFIG.4by bringing the entire windshield201into the transmissive state. Accordingly, the object303is observable by the person on board.

On the other hand, the visual line of the person on board is automatically guided toward a transmissive region501to find the object303, by bringing not the entire windshield201but only a partial region501where the object303is visible into the transmissive state and a remaining region502into the non-transmissive state as depicted inFIG.5. In other words, the person on board is capable of searching for the scenery point in the wide visual field without costing time and effort. Note that the non-transmissive state in the region502is not limited to a completely light shielded state, and includes a state where a scenery is visible to a certain extent even at lowered transmittance (the same applies hereinafter).

If the target object303is a scenery point such as a POI (Point OF Interest), the location of the object303can be checked on the basis of map information used by car navigation or the like. Needless to say, any scenery information describing the scenery point other than the map information may be referred to as well. In addition, a current position and a traveling direction of the vehicle are detectable by using GNSS signals (described above) or the like. Accordingly, the condition that the vehicle301is approaching the object303and from which of the windows201to206of the vehicle interior200and from which place in the window the object303is visible can be calculated and obtained on the basis of the detected current position and traveling direction of the vehicle.

Further,FIG.6depicts a state where an object603corresponding to a superb view point appears in a traveling direction while a vehicle601is traveling straight at a constant speed on a straight road602. However, an obstacle604is present between the vehicle601and the object603. For example, the obstacle604is a large-sized vehicle such as a truck and a trailer, a structure such as a building, a roadside tree, or a signboard provided on a road shoulder.

In such a case, the visual field is blocked by the obstacle604even when the windshield201is in the transmissive state. Accordingly, the object603is invisible for the person on board. Further, when the windshield201is switched to the transmissive state, the person on board is not allowed to view a beautiful scenery, and is only made to feel uncomfortable by the obstacle604entering the visual line as an unsightly object.

If the obstacle is a structure, the location of the obstacle can be checked on the basis of the map information. Further, the obstacle such as a surrounding vehicle, a roadside tree, and a signboard is recognizable on the basis of sensor information obtained by an in-vehicle external recognition sensor such as an in-vehicle camera and a radar. It is thus determinable whether the obstacle604is present between the vehicle601and the object303on the basis of the location of the object603obtained by the map information or the like, the current position of the vehicle601, and the recognition result of the obstacle604. In a case of a determination that the object603is less visible by the presence of the obstacle604, visual line guidance is given up, and the windshield201is maintained in the non-transmissive state. The person on board is not allowed to view the scenery point, but simultaneously not made uncomfortable by the obstacle604also invisible.

Further,FIG.7schematically depicts a relation between a vehicle speed of a vehicle701and the visual field of the person on board. It is known that a visual field of a human changes according to a moving speed. A visual field of a human is wide during a low moving speed, but becomes narrower with an increase in the moving speed.FIG.7depicts a viewing angle during a low moving speed and a viewing angle during a high moving speed to which reference numbers711and712are given, respectively.

Accordingly, in a case where the vehicle701travels on a road702while changing a vehicle speed, the vehicle701controls the transmissive state or transmittance of the windows201to206according to whether or not an object703of a scenery point is located within a range of a viewing angle corresponding to the vehicle speed at the time when the vehicle701approaches the object703.

The vehicle speed and the steering angle of the vehicle701are detectable by the vehicle state detection unit143. The range of the viewing angle corresponding to the vehicle speed can be calculated in the traveling direction of the vehicle701on the basis of a detection result of the vehicle speed and the steering angle. Thereafter, it is determined whether the object703whose location has been specified on the basis of the map information or the like is contained in the range of the viewing angle corresponding to the vehicle speed at the current position of the vehicle701.

While the vehicle701is traveling at a low speed Vlow, the object703is present within the range of the viewing angle711. Accordingly, a partial region where the object703is visible in the windshield201is brought into the transmissive state to guide the visual line of the person on board to the object703.

On the other hand, while the vehicle701is traveling at a high speed Vhigh, the object703is located out of the range of the viewing angle712. In such a case, the visual line of the person on board is not guided. Accordingly, the windshield201is not switched to the transmissive state even at the event that the vehicle701approaches the object703.

Further, the visual field range viewed through the windshield201also changes according to a shape of a road such as a curve and a slope as well as the vehicle speed. For a curved road, an object not shifting in the up-down direction and the left-right direction is preferably selected as an object to which the visual line is guided. Particularly, a scenery rapidly shifts (at a high angular speed) outside a curve. Accordingly, the visual line may be guided to an object at a long distance, but should not be guided to an object along a road (at a short distance). Even for a straight road, an object located along the road (at a short distance) shifts at a high speed in the visual field range. Accordingly, the visual line should not be guided to this object.

FIGS.8to12each depict an example where visual line guidance is performed using the windshield201which includes a transmissive region and a non-transmissive region.

FIG.8depicts a situation where a circular region containing an object801corresponding to a scenery point is brought into a transmissive state. In this case, a peripheral remaining region is brought into a non-transmissive state.

A non-transmissive region803has no pattern or has a dry and tasteless pattern to make a scenery invisible. The visual field range other than the scenery point is masked by the non-transmissive region803, and things other than the object801are made invisible for the eyes of the person on board. The visual line of the person on board is automatically guided to the visible object801. In this manner, the person on board is made more interested in the object801.

When a circular transmissive region802is cut out within the non-transmissive region803which widely covers the visual field range, a pop-up visual effect of the object801is produced within the transmissive region802depending on cases. This effect strongly draws attention of the person on board. In addition, it is preferable that the size (diameter) of the circular transmissive region802containing the object801is set such that an area of a region where the object801is visible has a predetermined ratio or smaller. This size is preferable for the following reasons. When the visual field range has a size substantially identical to the size of the object801, only the object801is visible. In this case, the entire image of the object801is difficult to recognize. Conversely, when the occupation ratio of the object801within the transmissive region802is excessively small, the object801becomes relatively small in appearance. In this case, visual line guidance is difficult to achieve.

Further, the non-transmissive region803is available not only for the purpose of masking things around the object801, but also for an indication place of guidance information. By displaying guidance information (not depicted inFIG.8) in the non-transmissive region803, description of the scenery point can be presented without blocking a superb view.

The guidance information may include description associated with the object801(e.g., geographical or topographical characteristics and historical trails), information associated with a driving schedule or a moving route of the vehicle (e.g., destination, arrival scheduled time to the next scenery point or the like, required length of time, and available time of facilities located adjacent to the route), and further information irrelevant to driving of the vehicle. Further, the guidance information may be displayed using characters, drawings, images, or the like.

FIG.9depicts a situation of the windshield201vertically divided into two parts. A right region containing an object901corresponding to a scenery point is brought into a transmissive state, and a left region is brought into a non-transmissive state. According to the example depicted inFIG.9, the windshield201is divided into substantially equal left and right parts.

A non-transmissive region903on the left side has no pattern or has a dry and tasteless pattern to make a scenery invisible. The visual field range other than the scenery point is masked by a non-transmissive region903, and things other than the object901are made invisible for the eyes of the person on board. The visual line of the person on board is automatically guided to the object901appearing in a transmissive region902on the right side. In this manner, the person on board is made more interested in the object901.

For example, in a case where the visual line is desired to be guided to the object901which is long in the vertical direction, such as a high mountain and a high-rise building, a larger portion of the object901can be inserted into the visual field range of the person on board by vertically dividing the windshield201and forming the transmissive region902in the divided part. In addition, it is preferable that a position of vertical division of the windshield201is determined such that an area of a region where the object901is visible has a predetermined ratio or smaller. This position is preferable for the following reasons. When the transmissive region902has a horizontal width substantially equivalent to the length of the object901, only the object901is visible. In this case, the entire image of the object901is difficult to recognize. Conversely, when the occupation ratio of the object901within the transmissive region902is excessively small, the object901becomes relatively small in appearance. In this case, visual line guidance is difficult to achieve.

Further, the non-transmissive region903is available not only for the purpose of masking things around the object901, but also for an indication region of guidance information (the same as above). The guidance information may be displayed using characters, drawings, images, or the like.

FIG.10depicts a situation of the windshield201horizontally divided into two parts. An upper region containing an object1001corresponding to a scenery point is brought into a transmissive state, and a lower region is brought into a non-transmissive state. According to the example depicted inFIG.10, the windshield201is divided into substantially equal upper and lower parts.

A non-transmissive region1003on the lower side has no pattern or has a dry and tasteless pattern to make a scenery invisible. The visual field range other than a scenery point is masked by the non-transmissive region1003, and things other than the object1001are made invisible for the eyes of the person on board. The visual line of the person on board is automatically guided to the object1001on the upper side. In this manner, the person on board is made more interested in the object1001.

For example, in a case where the visual line is desired to be guided to the object1001which is long in the horizontal direction, such as a chain of mountains (mountain range), a coast, a lake, and a long bridge, a larger portion of the object1001can be inserted into the visual field range of the person on board by horizontally dividing the windshield201and forming a transmissive region1002in the divided part. In addition, it is preferable that a position of horizontal division of the windshield201is determined such that an area of a region where the object1001is visible has a predetermined ratio or smaller. This position is preferable for the following reasons. When the transmissive region1002has a vertical width substantially equivalent to the height of the object1001, only the object1001is visible. In this case, the entire image of the object1001is difficult to recognize. Conversely, when the occupation ratio of the object1001within the transmissive region1002is excessively small, the object1001becomes relatively small in appearance. In this case, visual line guidance is difficult to achieve.

Further, the non-transmissive region1003is available not only for the purpose of masking things around the object1001, but also for an indication region of guidance information (the same as above). The guidance information may be displayed using characters, drawings, images, or the like.

FIG.11depicts a situation of the windshield201horizontally divided into three parts. A central region containing an object1101corresponding to a scenery point is brought into a transmissive state, and each of regions at upper and lower ends is brought into a non-transmissive state.

Each of non-transmissive regions1103and1104at the upper and lower ends has no pattern or has a dry and tasteless pattern to make a scenery invisible. The visual field range other than the scenery point is masked by the non-transmissive regions1103and1104, and things other than the object1101are made invisible for the eyes of the person on board. The visual line of the person on board is automatically guided to the object1101near the center. In this manner, the person on board is made more interested in the object1101.

For example, in a case where the visual line is desired to be guided to the object1101which is long in the horizontal direction and visible near the center of the windshield201, such as a chain of mountains (mountain range), a coast, a lake, and a long bridge, a larger portion of the object1101can be inserted into the visual field range of the person on board by horizontally dividing the windshield201and forming a transmissive region1102at the center. In addition, it is preferable that a position of horizontal division of the windshield201is determined such that an area of a region where the object1101is visible has a predetermined ratio or smaller. This position is preferable for the following reasons. When the transmissive region1102has a vertical width substantially identical to the height of the object1101, only the object1101is visible. In this case, the entire image of the object1101is difficult to recognize. Conversely, when the occupation ratio of the object1101within the transmissive region1102is excessively small, the object1101becomes relatively small in appearance. In this case, visual line guidance is difficult to achieve.

Further, each of the non-transmissive regions1103and1104is available not only for the purpose of masking things around the object1101, but also for an indication region of guidance information (the same as above). The guidance information may be displayed using characters, drawings, images, or the like.

FIG.12depicts a situation of the windshield201which includes a triangular region formed by cutting an upper right corner of the windshield201and brought into a transmissive state. A transmissive region1202containing an object1201of a scenery point is formed in the triangular region, and the remaining region is brought into a non-transmissive state.

A non-transmissive region1203in a region other than the upper right portion has no pattern or has a dry and tasteless pattern to make a scenery invisible. The visual field range other than the scenery point is masked by the non-transmissive region1203, and things other than the object1201are made invisible for the eyes of the person on board. The visual line of the person on board is automatically guided to the object1201on the right side. In this manner, the person on board is made more interested in the object1201.

In a case where the object1201of the scenery point appears at an offset place such as a location near a peripheral edge of the windshield201, the object1201can be inserted into the visual field range of the person on board by cutting a corner of the windshield201and forming the transmissive region1202in the cut corner. In addition, it is preferable that a cutting position of the corner and the size of the windshield201are determined such that an area of a region where the object1201is visible has a predetermined ratio or smaller. These position and size are preferable for the following reasons. When the transmissive region1202in the cut corner has an area substantially equivalent to the area of the object1201, only the object1201is visible. In this case, the entire image of the object1201is difficult to recognize. Conversely, when the occupation ratio of the object1201within the transmissive region1202is excessively small, the object1201becomes relatively small in appearance. In this case, visual line guidance is difficult to achieve.

Further, the non-transmissive region903is available not only for the purpose of masking things around the object901, but also for an indication region of guidance information (the same as above). The guidance information may be displayed using characters, drawings, images, or the like.

FIGS.8to12each depict the example where the transmissive region and the non-transmissive region each having any of various shapes and sizes, such as a circular shape, a quadrangular shape, and a triangular shape, are defined in the windshield201. However, these examples are not required to be adopted. The transmissive region may have an elliptical shape or a polygonal shape having more than four corners, such as a pentagonal shape, a shape cut out along the contour of the object to which the visual line is guided, or a random shape. In what kind of shapes and sizes the transmissive region and the non-transmissive region are allowed to be drawn also depends on resolution or the like of the device used for switching between transmissive and non-transmissive, such as a liquid crystal shutter.

Further,FIGS.8to12each depict the example where the visual line of the person on board is guided using the windshield201. However, the visual line of the person on board can similarly be guided using the windows202to206other than the windshield201.

Note that the “non-transmissive” of the non-transmissive region is assumed to include not only a case where transmittance of the windows is set to 0%, but also a case where transmittance is reduced to a low value to lower visual recognizability of a scenery. Further, the “transmissive” of the transmissive region is not limited to a case where transmittance of the windows is set to 100%. For example, the transmissive region is assumed to include a transmissive region having transmittance adjusted according to brightness of a visual line point corresponding to a target of visual line guidance, such as direct light, slanting light (side light), and backlight.

C. Visual Line Guidance by Vehicle According to Road Shape

It is extremely rare that a car travels on a flat and straight road. Many roads include left and right curves and slopes. Accordingly, the scenery viewable through the windshield201and the other windows202to206also changes according to the shapes of the roads such as curves and slopes.

FIG.13illustrates a shape of a road on which a car travels, and a state of a change of a scenery viewable through a windshield of the car for each place on the road in conjunction with the shape of the road.

A road1300depicted in the figure passes through a bridge1302built over a lake1301, a sightseeing boat boarding point1303on a lakeshore, a sea of trees park1304, a plateau1305, and a high mountain1306in this order while meandering, and reaches a mountain site1307. Further, the road1300includes a first curve1311on the bridge1302, and a second curve1312near the sea of trees park1304. The first curve1311has a right curve and a left curve following the right curve, while the second curve1312has a clockwise curve. It is assumed herein that each of the bridge1302, the sightseeing boat boarding point1303on the lakeshore, the sea of trees park1304, the plateau1305, the high mountain1306, and the mountain site1307becomes an object at a scenery point. An order of priority is given to each of the plurality of objects on the basis of a score value or the like calculated by a filtering process (described below).

In a case where the vehicle travels on a straight road, a vanishing point of the road viewed through the windshield201is located substantially at the center of the windshield201. On the other hand, in a case where the vehicle travels on a curve, the vanishing point of the road shifts leftward or rightward from the center of the windshield201. Specifically, the vanishing point shifts leftward in the left curve, and shifts rightward on the right curve. A shift amount is dependent on a curvature of the curve.

While not depicted in the figures, in a case of traveling on a slope such as an upslope and a downslope, the vanishing point of the road shifts upward or downward from the center of the windshield201according to an inclination of the slope.

FIGS.14to18each depict a scenery viewable through the windshield201of the car traveling on the road1300depicted inFIG.13. Note that each ofFIGS.14to18depicts a scenery through the windshield201in a state where visual line guidance by masking is not applied.

FIG.14depicts a scenery viewable through the windshield201of the vehicle traveling on a straight road before the first curve1311for each predetermined frame period. The scenery viewed through the windshield201during this period includes the bridge1302and the high mountain1306as objects of scenery points.

Further,FIG.15depicts a scenery viewable through the windshield201of the vehicle traveling near an entrance of the first curve1311for each predetermined frame period. The scenery viewed through the windshield201during this period includes the bridge1302and the high mountain1306as objects of scenery points.

Further,FIG.16depicts a scenery viewable through the windshield201of the vehicle traveling within the first curve1311for each predetermined frame period. The scenery viewed through the windshield201during this period includes the sightseeing boat boarding point1303on the lakeshore, the plateau1305, and the high mountain1306as objects of scenery points. The sightseeing boat boarding point1303disappears from the visual field range in the middle of traveling, and the sea of trees park1304appears instead of the sightseeing boat boarding point1303.

Besides,FIG.17depicts a scenery viewable through the windshield201of the vehicle traveling within the second curve1312following the first curve1311for each predetermined frame period. The scenery viewed through the windshield201during this period includes the sea of trees park1304as an object of a scenery point. The plateau1305and the high mountain1306appear in the middle of traveling.

Further,FIG.18depicts a scenery viewable through the windshield201of the vehicle traveling on a straight road after passing through the second curve1312for each predetermined frame period. The scenery viewed through the windshield201during this period includes the plateau1305, the high mountain1306, and the mountain site1307as objects of scenery points.

In short, it is obvious fromFIGS.14to18that the appearance positions where the objects of the respective scenery points appear in the windshield201change from moment to moment according to the shape of the road. Needless to say, the size of each of the objects appearing in the windshield201also increases or decreases according to the distance from the vehicle to the object of the scenery point. Accordingly, visual line guidance for the person on board is achievable in a suitable manner by dynamically controlling the position, the size, and the shape of the transmissive region for allowing the person on board to view the object, or the position and the size of the non-transmissive region masked to make the region not containing the object invisible for the eyes of the person on board, on the basis of the shape of the road or the distance to the object.

FIG.19depicts an example of a scenery viewable through the windshield201of the vehicle traveling within the second curve1312following the first curve1311. The scenery depicted inFIG.19includes the high mountain1306as one of scenery points appearing near the upper right of the windshield201.

Further,FIG.20depicts a state where visual line guidance is applied to the windshield201of the vehicle at the same traveling position as that of the case ofFIG.19. As depicted in the figure, a transmissive region2001includes the high mountain1306near the upper right of the windshield201as the object of the scenery point such that the area of the portion where the high mountain1306in the transmissive region2001is viewed has a predetermined ratio or smaller (e.g., approximately 20%). In addition, the remaining region is designated as the non-transmissive region2002, and things other than the high mountain1306are masked to guide the visual line of the person on board toward the high mountain1306.

When the transmissive region2001is cut out within the non-transmissive region2002which widely covers the visual field range, a pop-up visual effect of the high mountain1306within the transmissive region2001is produced depending on cases. This effect strongly draws attention of the person on board. In addition, it is preferable that the size of the transmissive region2001containing the high mountain1306is set such that an area of a region where the high mountain1306is visible has a predetermined ratio or small. This size is preferable for the following reasons. When the visual field range has a size substantially identical to the size of the high mountain1306, only the high mountain1306is visible. In this case, the entire image of the high mountain1306is difficult to recognize. Conversely, when the occupation ratio of the high mountain1306within the transmissive region2001is excessively small, the high mountain1306becomes relatively small in appearance. In this case, visual line guidance is difficult to achieve.

According to the vehicle control system100, appearance timing and an appearance position of an object of each scenery point on the windshield201are constantly calculated on the basis of position information associated with the object and obtained from a current position of the vehicle, a vehicle speed and a steering angle of the vehicle, map information, or the like. A liquid crystal shutter disposed on each of the windows201to206is drive-controlled on the basis of a calculation result, to achieve visual line guidance for the person on board.

Further, because countless scenery points are widely spread on the ground, the vehicle control system100filters the scenery points. Initially, the vehicle control system100acquires position information associated with the respective scenery points from map information or the like, and extracts, as targets of visual line guidance, things likely to enter the visual field range of the person on board through the windows201to206such as the windshield201, on the basis of a result of comparison between a current position and a traveling direction of the vehicle and position information associated with the scenery points.

In a case where a large number of scenery points still remain as candidates only by the comparison with the position information, the vehicle control system100performs selection on the basis of information associated with the person on board and a matching process for matching with the respective scenery points. Specifically, the scenery points are selected on the basis of a boarding purpose, schedule information, and background information such as the gender, the age, hobbies, and histories of the person on board. Further, the matching process may calculate a score of each of the scenery points by using a method such as CF (Collabolative Filtering) and CBF (Content Based Filtering), and selects a predetermined number of scenery points having high scores as targets of visual line guidance, or give priority to each scenery point in a descending order of the score. For example, in a case where plural scenery points are simultaneously present in the visual line direction of the person on board of the vehicle, priority may be given to a scenery point located closer, or a scenery point having a higher score may be selected.

Further, the order of priority given to each of the objects of the scenery points is not required to be fixed, and may be changed from moment to moment. For example, the order of priority may be dynamically switched between the objects on the basis of the number of times of visual line guidance performed for the objects in the past, the interval from previous execution of visual line guidance, or the latest state of the person on board (including a physical condition and a temper).

FIGS.21to25each depict a state where the visual line guidance is applied to the windshield201of the car traveling on the road1300depicted inFIG.13. It is assumed that the vehicle control system100constantly calculates appearance timing and an appearance position of an object of each scenery point on the windshield201during traveling of the vehicle on the road1300. Note that traveling sections depicted inFIGS.21to25correspond to those depicted inFIGS.14to18, respectively. Further,FIG.28is a bird's eye view of a state of a shift of visual line guidance during traveling on the road1300.

FIG.21depicts a state where visual line guidance is applied to the windshield201of the vehicle traveling on the straight road before the first curve1311for each predetermined frame period.

At a traveling position of the vehicle corresponding to a reference number2101, the high mountain1306corresponding to one of the objects of the scenery points is scheduled to appear in a direction near the upper right of the windshield201. In this case, the high mountain1306is invisible behind a building adjacent to the right side of the road, and is scheduled to appear after ten seconds. Accordingly, the windshield201is temporarily brought into a non-transmissive state, and a transmissive region2111is prepared near the upper right beforehand. Further, looking ahead the appearance position of the high mountain1306, guidance information “superb view is appearing soon” is generated to give advance notice that the high mountain1306will appear in ten seconds. This notice information is displayed on a non-transmissive region2112of the windshield201.

Subsequently, as indicated by a reference number2102, the high mountain1306corresponding to the object of the scenery point appears in the transmissive region2111of the windshield201as announced. Then, guidance information “Mt. - - -” indicating the name of the high mountain1306is generated. This guidance information is displayed on the non-transmissive region2112of the windshield201.

Note that the bridge1302corresponding to an object of another scenery point also appears in the visual field range of the windshield201at this point of time. However, visual line guidance to the bridge1302is not started for reasons such as difficulty in simultaneous execution of visual line guidance together with visual line guidance already started toward the high mountain1306and higher priority given to the high mountain1306than the end1302.

Further, as indicated by a reference number2103, guidance information describing details of the high mountain1306, i.e., “beautiful volcano best-known and symbolic in Japan, registered as world heritage in 2013- - - ,” is generated and displayed while being overlapped on the non-transmissive region2112of the windshield201.

After the visual line of the person on board is guided to the high mountain1306in such a manner, the entire window of the windshield201is brought into a transmissive state by cancelling the non-transmissive state of the non-transmissive region2112as indicated by a reference number2104. As a result, the person on board is allowed to enjoy the scenery throughout a wide visual field range with awareness of the high mountain1306.

Further,FIG.22depicts a state where visual line guidance is applied to the windshield201of the vehicle traveling near the entrance of the first curve1311for each predetermined frame period.

At a traveling position of the vehicle corresponding to a reference number2201, the bridge1302and the high mountain1306simultaneously exist as objects of scenery points within the visual field of the windshield201. At this time, visual line guidance to the high mountain1306has already been completed. Accordingly, visual line guidance to the bridge1302as guidance not started before (described above) is started. Specifically, the windshield201is horizontally divided into three parts. A central region2211containing the object1101corresponding to a scenery point is brought into a transmissive state, and each of regions2212and2213at upper and lower ends is brought into a non-transmissive state. In this manner, the visual line of the person on board is guided to the bridge1302within the transmissive region2211. Further, guidance information “Lake - - - Long Bridge” indicating the name of the bridge1302is generated. This guidance information is displayed on the non-transmissive region2212in an upper part of the windshield201.

The vehicle traveling on the road1300in this manner approaches the lakeshore, and reaches the sightseeing boat boarding point1303corresponding to the object of the next scenery point. Then, an appearance position of the sightseeing boat boarding point1303is looked ahead, and guidance information associated with the bridge1302and including notice information associated with the sightseeing boat boarding point1303, i.e., “you can enjoy view on lake for total length of 2 km, with sightseeing boats in service” is generated. This guidance information is displayed on the non-transmissive region2212in the upper part of the windshield201as indicated by a reference number2202.

After the visual line of the person on board is guided to the bridge1302, the entire window is brought into a transmissive state by cancelling the non-transmissive state of the regions2212and2213of the windshield201as indicated by a reference number2203. In this manner, the person on board is allowed to enjoy a view of the lake1301during traveling on the bridge1302.

After visual line guidance to the bridge1302is completed as described above, visual line guidance to the lake1301is subsequently started. The sightseeing boat boarding point1303is scheduled to appear in a left part of the windshield201. Accordingly, as indicated by a reference number2214, a view of the lake1301on the left side of the bridge1302is provided by bringing a triangular region cut at an upper left corner of the windshield201into a transmissive state. Further, guidance information “Lake - - -” indicating the name of the lake1301is displayed using a non-transmissive region2215.

Further, with approach of the sightseeing boat boarding point1303, an appearance position of the sightseeing boat boarding point1303is looked ahead. Thereafter, as indicated by a reference number2205, detailed guidance information including notice information associated with the sightseeing boat boarding point1303, i.e., “you can enjoy view on lake for total length of 2 km, with sightseeing boats in service” is displayed using a non-transmissive region2215.

After visual line guidance of the person on board and display of the notice information associated with the sightseeing boat boarding point1303are completed in such a manner, the entire window is brought into a transmissive state to provide a view of the lake1301by cancelling the non-transmissive state of the non-transmissive region2215of the windshield201as indicated by a reference number2206.

Further,FIG.23depicts a state where visual line guidance is applied to the windshield201of the vehicle traveling within the first curve1311for each predetermined frame period.

At a traveling position of the vehicle corresponding to a reference number2301, the sightseeing boat boarding point1303as the object of the next scenery point appears in a small size in a left part of the windshield201. Accordingly, visual line guidance to the sightseeing boat boarding point1303is started. Specifically, the windshield201is vertically divided into two parts. A left region2311containing the sightseeing boat boarding point1303is brought into a transmissive state, and a right region2312is brought into a non-transmissive state. In this manner, the visual line of the person on board is guided to the sightseeing boat boarding point1303within the transmissive region2311. Further, guidance information indicating a remaining distance to the sightseeing boat boarding point1303, a service schedule and a use situation of sightseeing boats, i.e., “sightseeing boat boarding point, 500 m, 15 minutes until departure, crowded” is generated. This guidance information is displayed on the non-transmissive region2312of the windshield201.

When the vehicle travels on the road1300in this condition and further approaches the sightseeing boat boarding point1303, information such as numerical values is updated. In addition, detailed guidance information associated with use of sightseeing boats such as fares, a congestion situation, and a required length of time of the sightseeing boats, i.e., “sightseeing boat boarding point, 100 m, 14 minutes until departure, crowded, lake round tour in 90 minutes, 1,200 yen for adults and 600 yen for elementary school students and above” is generated. Display of the guidance information on the non-transmissive region2312in the right half part of the windshield201is switched to this guidance information as indicated by a reference number2302. The person on board is allowed to determine whether to use the sightseeing boat by checking the fares and the required length of time before arrival at the sightseeing boat boarding point1303. It is assumed herein that the person on board has decided to board the sightseeing boat.

When the vehicle subsequently arrives at the sightseeing boat boarding point1303, guidance information including information such as the arrival, the congestion situation of the sightseeing boats, and the schedule, i.e., “arrival, departure time: 13:00, crowded” is generated as indicated by a reference number2303. This guidance information is displayed on a non-transmissive region2312in the right half of the windshield201.

After completion of a cruise on the lake1301by using the sightseeing boat, the person on board rides into the vehicle. The vehicle restarts traveling on the road1300.

A traveling position of the vehicle corresponding to a reference number2304comes out of the lake1301at a position near an end edge of the first curve1311. At this position, the sea of trees park1304corresponding to the object of the next scenery point appears in a small size at a far distance within the visual field range of the windshield201. Accordingly, the windshield201is horizontally divided into two parts. An upper region2313containing the sea of trees park1304is brought into a transmissive state, and a lower region2314is brought into a non-transmissive state. Further, guidance information indicating the name of the sea of trees park1304, the remaining distance, and the congestion situation, i.e., “- - - Sea of trees park 2 km ahead, extremely crowded” is generated. This guidance information is displayed on the non-transmissive region2314in a lower part of the windshield201.

After the visual line guidance of the person on board and display of the guidance information announcing the arrival at the sea of trees park1304are completed in such a manner, the entire window is brought into a transmissive state to provide a view of the sea of trees park1304by cancelling the non-transmissive state of the non-transmissive region2214of the windshield201as indicated by a reference number2305.

Further,FIG.24depicts a state where visual line guidance is applied to the windshield201of the vehicle traveling within the second curve1312following the first curve1311for each predetermined frame period.

At the traveling position of the vehicle corresponding to a reference number2401, the vehicle has already arrived at the sea of trees park1304, and the sea of trees park1304appears in a large size in a right part of the windshield201. Accordingly, the windshield201is vertically divided into two parts. A right region2411containing the sea of trees park1304is brought into a transmissive state, and a left region2412is brought into a non-transmissive state. In this manner, the visual line of the person on board is guided to the sea of trees park1304within the transmissive region2411. Further, guidance information “- - - sea of trees park” indicating the name of the sea of trees park1304is generated, and is displayed on the non-transmissive region2412in the left part of the windshield201.

A similar scenery continues for a while in the windshield201during traveling within the second curve1312. Detailed guidance information associated with the sea of trees park1304, i.e., “- - - Sea of trees park, forest extending through - - - Prefecture - - - lake-cho, - - - mura, and designated as national park located on southwest of Mt. - - - ,” is newly generated. Thereafter, as indicated by a reference number2402, this guidance information is displayed on the non-transmissive region2412in the left part of the windshield201. At this time, the visual line guidance to the sea of trees park1304is assumed to have been completed.

At a subsequent traveling position of the vehicle corresponding to a reference number2403, there exists a superb view which contains the plateau1305corresponding to an object of a scenery point and appearing in a direction near the upper right of the windshield201and the high mountain1306appearing at the back of the plateau1305. However, this superb view is invisible behind trees of the sea of trees park1304, and is scheduled to appear in ten seconds. Accordingly, appearance positions of the plateau1305and the high mountain1306are looked ahead, and a transmissive region2413is prepared near the upper right of the windshield201beforehand. Further, guidance information announcing appearance of a spot of a superb view to the person on board, i.e., “superb view is appearing soon” is generated. This guidance information is displayed on a non-permissive region2414of the windshield201.

After the visual line of the person on board is guided in the direction where the superb view appears in such a manner, the entire window is brought into a transmissive state by cancelling the non-transmissive state of the non-transmissive region2112of the windshield201as indicated by a reference number2404. As a result, the person on board is allowed to enjoy the superb view produced by the plateau1305and the high mountain1306in a wide visual field range without missing the view.

Further,FIG.25depicts a state where visual line guidance is applied to the windshield201of the vehicle traveling on a straight road after passing through the second curve1312for each predetermined frame period.

At a traveling position of the vehicle corresponding to a reference number2501, the superb view where both the plateau1305and the high mountain1306at the back are simultaneously visible appears in the right half of the windshield201. Accordingly, the windshield201is vertically divided into two parts. A right region2511containing the superb view constituted by the plateau1305and the high mountain1306is brought into a transmissive state, and a left region2512is brought into a non-transmissive state. In this manner, the visual line of the person on board is guided to the superb view within the transmissive region2511. In addition, guidance information “- - - Plateau and Mt. - - -” indicating the respective names of the plateau1305and the high mountain1306constituting the superb view is generated. This guidance information is displayed on the non-transmissive region2512of the windshield201.

Note that the plateau1305is a scenery point appearing for the first time, and is determined as a target of visual line guidance. Further, the high mountain1306is an object for which visual line guidance has already been performed (during traveling of the vehicle in the vicinity before the first curve1311). However, a combination of the plateau1305and the high mountain1306are designated as a target of visual line guidance on the basis of the fact that a period of time has elapsed from the previous visual line guidance to the high mountain1306and the fact that the composition where the high mountain1306is visible at the back of the plateau1305is a superb view.

Thereafter, guidance information “- - - Plateau and Mt. - - - , view from - - - Plateau in evening is selected as one of standard 30 spots” indicating details of the plateau1305and the high mountain1306is generated. This guidance information is displayed on the non-transmissive region2512as indicated by a reference number2502.

After the visual line of the person on board is guided in the direction where the superb view appears in such a manner, the entire window is brought into a transmissive state by cancelling the non-transmissive state of the non-transmissive region2512of the windshield201as indicated by a reference number2503. As a result, the person on board is allowed to enjoy the superb view produced by the plateau1305and the high mountain1306in a wide visual field range without missing the view.

Thereafter, at a traveling position of the vehicle corresponding to a reference number2504, the mountain site1307corresponding to one of the objects of the scenery points appears in a small size at a far distance within the visual field range of the windshield201. Accordingly, the windshield201is horizontally divided into two parts. An upper region2513containing the mountain site1307is brought into a transmissive state, and a lower region2514is brought into a non-transmissive state. Further, guidance information describing the name of the mountain site1307and details of the mountain site1307, i.e., “Mt. - - - , active volcano having caused small eruption in 2018” is generated. This guidance information is displayed on the non-transmissive region2514in the lower part of the windshield201.

D. Configuration of Visual Line Guidance System

FIG.26depicts a functional configuration example of a visual line guidance system2600. The visual line guidance system2600is configured to execute visual line guidance using the windows201to206of the vehicle as described in paragraph C described above.

The visual line guidance system2600includes a state acquisition unit2601, a display unit2602, an information retrieval unit2603, an information generation unit2604, and a control unit2605. The visual line guidance system2600basically operates on the vehicle control system100depicted inFIG.1. At least some of constituent elements of the visual line guidance system2600are implemented by using constituent elements included in the vehicle control system100. Alternatively, some of the constituent elements of the visual line guidance system2600may be implemented by using an information terminal such as a smartphone and a tablet-type terminal carried into the vehicle interior by the person on board, or other types of information apparatuses. In addition, it is assumed that the respective constituent elements of the visual line guidance system2600are capable of achieving bidirectional data communication between each other via a bus or by using interprocess communication.

The state acquisition unit2601basically acquires information associated with a state of the vehicle, and may further acquire information associated with a state of a person on board having boarded the vehicle.

The state acquisition unit2601acquires a current position, a vehicle speed, and a steering angle of the vehicle as a state of the vehicle. These items of information correspond to information necessary for calculating an object of a scenery point entering the visual field range of each of the windshield201and the other windows20to206, and an appearance position of the object in the windows201to206. For example, the current position of the vehicle is acquirable on the basis of a GNSS signal received by the data acquisition unit102including a GNSS receiver. Further, the vehicle speed and the steering angle are acquirable from a detection result obtained by the vehicle state detection unit143on the basis of a rotation speed of the wheels and a steering angle of the steering wheel.

Further, the state acquisition unit2601acquires a level of autonomous driving of the autonomous driving control unit112as a state of the vehicle. As also apparent fromFIGS.21to25, visual line guidance is achieved by masking at least a part of the visual field range of each of the windows201to206. During complete autonomous driving or high-level autonomous driving of the vehicle, safe traveling is achievable without any problems even if at least a part of the visual field of the windshield201or the like is lacking. On the other hand, after switching to low-level autonomous driving or manual driving, safe traveling may be put at risk if at least a part of the visual field of each of the windows201to206such as the windshield201is lacking.

In addition, the state acquisition unit2601performs personal authentication of each person on board using face authentication or biometric authentication as a state of each person on board, and acquires a boarding purpose, schedule information, and background information such as the gender, the age, hobbies, and histories of the identified person on board. The personal authentication may be performed on the basis of vehicle owner information. Further, the boarding purpose, the schedule information, and the background information associated with the person on board may be managed inside the vehicle by using a database, may be read from an information terminal such as a smartphone carried by the person on board, or may be acquired from the outside of the vehicle by using V2X communication or the like. The object of the visual line guidance or the order of priority of the objects may be switched according to the boarding purpose, the schedule information, and the background information associated with the person on board. For example, the visual line should be actively guided to scenery points if boarding is made for a traveling purpose. However, the level of the necessity of visual line guidance decreases if boarding is made for a commuting purpose. Further, the visual line need not be repeatedly or frequently guided to an identical or similar scenery point. Further, the tendency of scenery points desired to be viewed differs for each hobby, history, or the like of the person on board.

In addition, the state acquisition unit2601may acquire a physical condition, a degree of alertness, a degree of concentration, a degree of fatigue, and the like of the person on board on the basis of a detection result obtained by the interior information detection unit142. It may be unpreferable to perform visual line guidance applied to the windows201to206by a scenery masking process when the person on board feels sick or feels tired.

The display unit2602includes a liquid crystal shutter attached to at least a part of the windows201to206of the vehicle. The liquid crystal shutter is a device capable of switching between a transmissive state and a non-transmissive state or gradually or successively switching transmittance in units of grid having a predetermined size, and drives in a frame period of approximately 30 fps (frame per second) or 60 fps, for example. For example, a scenery masking process for bringing only a partial region of the windshield201into a transmissive state is performed by operating the liquid crystal shutter. In this manner, a desired object is visually highlighted to guide the visual line of the person on board to this object.

Further, the display unit2602is capable of displaying guidance information by using a non-transmissive region arranged on the windows201to206such as the windshield201. The guidance information is character-based information, but may include figures or images. The display unit2602may draw the guidance information by driving the liquid crystal shutter, or may project the guidance information in the non-transmissive region by using a projector or a head-up display additionally equipped.

The information retrieval unit2603retrieves information associated with an object of a scenery point to which the visual line is to be guided. The information retrieval unit2603refers to map information and scenery information to achieve this information retrieval. At least a part of the map information and the scenery information used by the information retrieval unit2603may be accumulated in a storage device included in the vehicle control system100or equipped in the vehicle, such as the storage unit111. Alternatively, the information retrieval unit2603may access the map information and the scenery information accumulated in a system outside the vehicle via V2X communication.

The information retrieval unit2603searches for a scenery point to which the visual line is guided, on the basis of position information. Specifically, after acquiring position information associated with respective scenery points from the map information and the scenery information, the information retrieval unit2603extracts things likely to enter the visual field range of the person on board through the windows201to206such as the windshield201as targets of visual line guidance, on the basis of a result of comparison between a current position and a traveling direction of the vehicle and the position information associated with the scenery points.

Further, the information retrieval unit2603may select (filter) the scenery points extracted on the basis of the position information by performing a matching process for matching information associated with the person on board and the respective scenery points. This selection is made because a mere comparison with the position information could leave a large number of scenery points remaining as candidates. For example, the information retrieval unit2603calculates a score of each scenery point by performing the matching process using a method such as CF and CBF, on the basis of a boarding purpose, schedule information, and background information associated with the person on board and acquired by the state acquisition unit2601, and selects a predetermined number of scenery points each having a high score as targets of visual line guidance, or gives priority to each scenery point in a descending order of the score.

The information generation unit2604generates guidance information associated with the object of the scenery point found by the information retrieval unit2603. The guidance information is displayed by the display unit2602by using the non-transmissive region arranged for the scenery masking process in the windows201to206such as the windshield201as described above. Examples of the guidance information include simple information simply indicating the name of the object, information associated with an access to the object, such as a distance and a required length of time, information associated with use of the object such as an entrance fee, a usage fee, and a congestion situation, and information describing a historical trail and a view of the object. The information generation unit2603generates guidance information associated with the object, on the basis of scenery information accumulated in the storage unit111, and scenery information accessible using V2X communication. Further, the information generation unit2604generates plural pieces of guidance information associated with one object in some cases.

The guidance information generated by the information generation unit2604is displayed by the display unit2602using the non-transmissive region arranged in the windows201to206such as the windshield201. In a case where the information generation unit2604generates plural pieces of guidance information associated with one object, display of the plural pieces of guidance information is sequentially switched according to traveling processes of the vehicle, the distance between the vehicle and the object, or the like. For example, in a stage where the object is viewed in a small size from the vehicle still located away from the scenery point, or in a stage of notice of appearance of the object invisible behind another thing, simple guidance information indicating the name of the target is simply displayed. Thereafter, with approach to the object, display is gradually switched to guidance information including information associated with an access to the object, such as a distance and a required length of time, more detailed guidance information including information associated with use of the object, such as a congestion situation, and information describing a history and a view of the object.

The control unit2605controls overall operations of the respective units within the visual line guidance system2600.

For example, the control unit2605controls a filtering process for filtering a plurality of scenery points searched for by the information retrieval unit2603.

Further, the control unit2605controls a scenery masking process, i.e., a process performed by the display unit2602for achieving optimum arrangement of the transmissive region and the non-transmissive region within the windows201to206, on the basis of an appearance position of an object of an object of a scenery point on the windows201to206such as the windshield201, for example. Specifically, the position, the size, the shape, and the like of the transmissive region are determined such that a desired object is visible in a highlighted manner in a state where a scenery is masked by the non-transmissive region set in the windshield201or the like.

Further, the control unit2605controls a generation process performed by the information generation unit2604for generating guidance information according to selection of an object of a scenery point and selection switching.

E. Process Operation of Visual Line Guidance System

Described in this paragraph is an operation example of the visual line guidance system2600described in paragraph D.FIG.27presents a processing procedure in the form of a flowchart, performed by the visual line guidance system2600for achieving visual line guidance.

Initially, the state acquisition unit2601acquires a current position, a vehicle speed, and a steering angle of the vehicle (step S2701).

In this processing step S2701, the state information acquisition unit2601may also acquire information associated with other states of the vehicle, such as an autonomous driving level of the vehicle, and information associated with the person having boarded the vehicle. The information associated with the person on board includes a boarding purpose, schedule information, and background information such as the gender, the age, hobbies, and histories of the person on board, for example.

Subsequently, the information retrieval unit2603retrieves information associated with an object of a scenery point to which the visual line is to be guided (step S2702). Specifically, after acquiring position information associated with the respective scenery points from map information and scenery information, the information retrieval unit2603detects things likely to enter the visual field range of the person on board through the windows201to206such as the windshield201, and extracts these things as targets of visual line guidance on the basis of a result of comparison between a current position and a traveling direction of the vehicle and position information associated with the scenery points.

Thereafter, the information retrieval unit2603performs a filtering process for a search result obtained in previous step S2702(step S2703).

In this processing step S2703, for example, the information retrieval unit2603calculates a score of each scenery point by performing the matching process using a method such as CF and CBF, on the basis of a boarding purpose, schedule information, and background information associated with the person on board and acquired by the state acquisition unit2601, and selects a predetermined number of scenery points each having a high score as targets of visual line guidance, or gives priority to each scenery point in a descending order of the score. For example, in a case where plural scenery points simultaneously appear in the windshield201, the number of the scenery points to which visual line guidance is applied is reduced to a predetermined number or smaller (e.g., one) on the basis of the order of priority.

Further, in this processing step S2703, the filtering process may be performed on the basis of a state of the person on board. For example, the filtering process is performed on the basis of a boarding purpose, the number of times of visual line guidance performed for the objects before, the interval from previous visual line guidance, the latest state of the person on board (including a physical condition and a temper). If boarding is made for a commuting purpose or the like, frequent execution of visual line guidance is troublesome. In this case, the filtering process may be performed in such a manner as to reduce the number of times of execution. Further, if a scenery point is famous but viewed by the person on board many times before, the degree of necessity of visual line guidance is low. Accordingly, this scenery point may be filtered. Further, it may be unpreferable to perform visual line guidance applied to the windows201to206by a scenery masking process when the person on board feels sick or feels tired. In this case, the filtering process is performed on the basis of a physical condition, a degree of alertness, a degree of concentration, a degree of fatigue, or the like of the person on board.

Thereafter, the control unit2605performs a scenery mask optimization process (step S2704). In this processing step2604, display positions of the transmissive region and the non-transmissive region are determined according to an appearance position of the object of the scenery point to which the visual line is guided within the windshield201(or the other windows20to206). Further, the shape of the transmissive region is determined according to the contour or the like of the object of the scenery point, and the size of the transmissive region is determined such that an area of a portion where the object is visible becomes a predetermined ratio or smaller.

Further, the information generation unit2604generates guidance information associated with the object of the scenery point to which the visual line is determined to be guided after the filtering process (step S2705). The information generation unit2604may generate plural pieces of guidance information associated with one object. Examples of the guidance information include simple information simply indicating the name of the object, guidance information associated with an access to the object, such as a distance and a required length of time, guidance information associated with use of the object such as an entrance fee, a usage fee, and a congestion situation, and guidance information describing a historical trail and a view of the object.

Thereafter, the control unit2605controls the display unit2602in such a manner that the transmissive region and the non-transmissive region located at the positions and having the shapes each determined in step S2704are displayed on the windows201to206such as the windshield201, and execute scenery masking is executed (step S2706).

However, the non-transmissive region displayed on the windshield201blocks a part of the visual field range of the driver. Accordingly, the control unit2605may determine whether to execute visual line guidance and actually display the non-transmissive region depending on whether or not the vehicle is in an autonomous driving state. During complete autonomous driving or high-level autonomous driving of the vehicle, safe traveling is achievable without any problems even if at least a part of the visual field of the windshield201or the like is lacking. On the other hand, after switching to low-level autonomous driving or manual driving, safe traveling may be put at risk if at least a part of the visual field of each of the windows201to206such as the windshield201is lacking. During low-level autonomous driving or manual driving, the control unit2605may prohibit display of the non-transmissive region on the windows201to206such as the windshield201, or may prohibit a start of visual line guidance processing itself.

Further, the control unit2605controls the display unit2602in such a manner that the guidance information generated in step S2705is displayed using the non-transmissive region displayed in step S2706(step S2707).

In a case where the information generation unit2604generates plural pieces of guidance information associated with one object, display of the plural pieces of guidance information is sequentially switched according to traveling processes of the vehicle, the distance between the vehicle and the object, and the like in this processing step S2707. For example, in a stage where the object is viewed in a small size from the vehicle still located away from the scenery point, or in a stage of notice of appearance of the object invisible behind another thing, simple guidance information indicating the name of the object is simply displayed. Thereafter, with approach to the object, display is gradually switched to guidance information including information associated with an access to the object, such as a distance and a required length of time, more detailed guidance information including information associated with use of the object, such as a congestion situation, and information describing a history and a view of the object.

During execution of visual line guidance, the visual line guidance system2600repetitively starts the processing procedure presented inFIG.27for each predetermined cycle. Every time this processing procedure is executed, the scenery mask and display of the guidance information applied to the windows201to206such as the windshield201are refreshed.

INDUSTRIAL APPLICABILITY

The technology disclosed in the present description has been described above in detail with reference to the specific embodiment. However, it is obvious that those skilled in the art can make corrections or substitutes of this embodiment without departing from the subject matters of the technology disclosed in the present description.

According to the present description, the embodiment where the technology disclosed in the present description is applied to scenery display on the vehicle traveling on the road has been chiefly described. However, the subject matters of the technology disclosed in the present description are not limited to this example. The technology disclosed in the present description is also applicable to scenery display on a window of a railroad vehicle, an airplane, or a vessel, or on a window of other various types of mobile bodies or the like boarded by a human.

Further, according to the present description, the embodiment where the visual line of the person on board is guided using the window of the vehicle has been chiefly described. However, the technology disclosed in the present description may be practiced using other devices. For example, the technology disclosed in the present description can similarly be practiced using an in-house video display device such as a car navigation screen or a tablet terminal carried by a person on board.

In short, the technology disclosed in the present description has been described by presenting only an example. It is therefore not intended that contents described in the present description should be interpreted in a limited manner. The claims should be taken into consideration to determine the subject matters of the technology disclosed in the present description.

Note that the technology disclosed in the present description may also have the following configurations.

(1)

An information processing apparatus including:

a detection unit that detects a region where a predetermined object outside a mobile body appears within a window of the mobile body;

a display unit that displays a transmissive region and a non-transmissive region within the window; and

a control unit that controls the display unit on the basis of a detection result obtained by the detection unit.

(2)

The information processing apparatus according to (1) described above, in which the control unit controls the display unit in such a manner that a transmissive region in the region where the object appears is displayed or a non-transmissive region that masks a region where the object is absent is displayed.

(3)

The information processing apparatus according to (1) or (2) described above, further including:

a state acquisition unit that acquires state information associated with the mobile body, in which

the detection unit detects the object on the basis of the state information.

(4)

The information processing apparatus according to (3) described above, in which

the state acquisition unit acquires position information associated with the mobile body, and

the detection unit detects the object on the basis of a current position of the mobile body.

(5)

The information processing apparatus according to (3) described above, in which

the state acquisition unit acquires a current position of the mobile body and a moving speed and a traveling direction of the mobile body, and

the detection unit detects the region where the object appears in the window, on the basis of the current position of the mobile body, the moving speed and the traveling direction of the mobile body, and position information associated with the object.

(6)

The information processing apparatus according to any one of (1) to (5) described above, in which the control unit controls at least either a shape or a size of the transmissive region on the basis of at least either a size of the object appearing in the window or the region where the object appears.

(7)

The information processing apparatus according to any one of (1) to (6) described above, in which

the display unit includes a liquid crystal shutter disposed on the window, and

the control unit arranges the transmissive region and the non-transmissive region in the window by drive-controlling the liquid crystal shutter.

(8)

The information processing apparatus according to any one of (1) to (7) described above, further including:

a search unit that searches for the object.

(9)

The information processing apparatus according to (8) described above, in which the search unit searches for an object of a scenery point appearing within the window.

(10)

The information processing apparatus according to (9) described above, in which the search unit searches for the object on the basis of at least either map information or scenery information that describes information associated with the scenery point.

(11)

The information processing apparatus according to any one of (8) to (10) described above, in which

the search unit further selects an object from plural found objects or gives priority to each of the found objects, and

the control unit controls the display unit in such a manner that a non-transmissive region that masks a region where the selected object or the object to which higher priority has been given is absent is displayed.

(12)

The information processing apparatus according to (11) described above, in which the search unit selects or gives priority to the object on the basis of a result of matching between information associated with a person having boarded the mobile body and each of the objects.

(13)

The information processing apparatus according to any one of (1) to (12) described above, further including:

a state acquisition unit that acquires information associated with a person having boarded the mobile body, in which

the control unit controls display of the non-transmissive region on the basis of the information associated with the person on board.

(14)

The information processing apparatus according to any one of (1) to (13) described above, further including:

a generation unit that generates guidance information associated with the object, in which

the control unit controls the display unit in such a manner that the guidance information is displayed.

(15)

The information processing apparatus according to (14) described above, in which the control unit controls the display unit in such a manner that the guidance information is displayed in the non-transmissive region.

(16)

The information processing apparatus according to (14) or (15) described above, in which

the generation unit generates plural pieces of guidance information for one object, and

the control unit controls the display unit in such a manner that the plural pieces of guidance information are sequentially displayed.

(17)

The information processing apparatus according to (16) described above, in which the control unit switches display of the plural pieces of guidance information on the basis of relative positions of the mobile body and the object.

(18)

An information processing method including:

a detection step of detecting a region where a predetermined object outside a mobile body appears within a window of the mobile body;

a display step of causing a display unit to display a transmissive region and a non-transmissive region within the window; and

a control step of controlling the display unit in such a manner that a transmissive region in the region where the object appears is displayed or a non-transmissive region that masks a region where the object is absent is displayed, on the basis of a detection result obtained by the detection step.

(19)

A mobile body apparatus including:

a main unit of a mobile body, the main unit including one or more windows;

a detection unit that detects a region where a predetermined object outside the mobile body appears within the window;

a display unit that displays a transmissive region and a non-transmissive region within the window; and

a control unit that controls the display unit on the basis of a detection result obtained by the detection unit, in which

the control unit controls the display unit in such a manner that a transmissive region in the region where the object appears is displayed or a non-transmissive region that masks a region where the object is absent is displayed.

REFERENCE SIGNS LIST

100: Vehicle control system101: Input unit102: Data acquisition unit103: Communication unit104: In-vehicle apparatus105: Output control unit106: Output unit107: Drive control unit108: Drive system109: Body control unit110: Body system111: Storage unit112: Autonomous driving control unit121: Communication network131: Detection unit132: Self-position estimation unit133: Situation analysis unit134: Planning unit135: Action control unit141: Exterior information detection unit142: Interior information detection unit143: Vehicle state detection unit151: Map analysis unit152: Traffic rule recognition unit153: Situation recognition unit154: Situation prediction unit161: Route planning unit162: Behavior planning unit163: Action planning unit171: Emergency avoidance unit172: Acceleration and deceleration control unit173: Direction control unit2600: Visual line guidance system2601: State acquisition unit2602: Display unit2603: Information retrieval unit2604: Information generation unit2605: Control unit